Building out the Economy Class and Beyond Next Generation storage (Featuring Drobo 5N2)
- My name is Kevin and I’m a serial storage hoarder
- The New Drobo build out
- Getting the data over (or how I learned to love ROBOCOPY)
- Backup and such.
- … and a media injection machine?
Welcome to a very different type of write-up and report – where I look into something rather technical (and IT Technical at that). This is a pretty niche subject, but for those who aren’t looking for cloud storage and rather keep their data local, this could provide some hints and tips.
Or at least some hints. Maybe some awful network design work, and definitely some network spaghetti cabling.
And there’s a Drobo for sure.
My name is Kevin, and I’m a serial storage hoarder. Especially in digital imagery, I am a serial hoarder – in terms that I don’t like to delete any images (except in cases of awful blur or autofocus).
And over the years – that’s given me problems in more ways than one. But the biggest issue I run into time and again is long term storage.
Like a lot of photographers, I shoot with digital RAW files – the equivalent of a negative that chews disk space for breakfast. Whilst I have a continually growing data collection, I know it’s going grow and cause further chaos especially now I shoot with a Canon EOS R consuming high amounts of storage per photo.
This thing creates all the megabytes – have a look at my Long Term Review…
So I need storage… both for live photography work and archival work. Believe it or not, I dig more into my archival work than I care to admit (I can count on multiple times I’ve had to dive for stuff a few years old – and more than the odd occasion when I’ve had to go to the early 2000s for stuff.
The project
My project is to build a network accessible storage cube, that acts as a medium-term archival server, with short term working off it where needed (and where disk capacity is constrained on my laptop).
- It’s got to have storage with headroom for growth
- Network accessible
- Easy to install and use
Yes. It’s pretty much a back of a postage stamp book specification. Makes my life a lot easier.
What isn’t in my scope (or what I’m not splashing the cash on)
There are certain things that I won’t put a penny out on this project – and there are good reasons Of course, your mileage may vary.
Certain types of Network Storage and Servers
- Storinators – http://www.45drives.com/products/storage/ – As much as I would love these things (as shown on Linus Tech Tips and so on).
But dear deity – not for the flat. Too loud and too expensive for my tastes. Also: where the hell am I going to stick it? (Although if 45 Drives and Seagate/Western Digital wish to sponsor me, drop me a message – and I’ll make space at the flat).
Storinators. Lovely expensive storage. *drool*
- Enterprise style servers (e.g. Rack Mount/Server Mount HP ProLiant/Dell PowerEdge/Lenovo ThinkServer) – The workload that I’m going to throw at these don’t match my design, as these are designed mostly for actual compute tasks – not storage.
HP Proliant DL380 G10 – Image HPE
And have you seen the mark-up on an HP disk? It’s a fair old whack. That and the noise and power consumption of these things… well – there’s a reason why you keep them locked in a server room. With air conditioning to keep them cool.
- Most Microsoft Windows Servers. If you work in the IT Realm and I said to you the following words “Microsoft Licensing”… need I go on? Yes, there are ways to mitigate the upfront cost (by using Retail Option Kits and Windows Server Essentials Editions), but in some cases, the cost of the software is almost the same exact cost of the base hardware!
Once you go down the rabbit hole of Microsoft Licensing, there’s not coming out. Ever. Even at the lowest end of the market as a Retail Option Kit… and may any deity you worship save you when it comes to Data Centre or if you’re planning to do some Virtualisation – Data – Servers Plus
- LTO (Linear Tape Open) Tape Drives – sadly, I do have more than an unusual amount of random writes. Whilst LTO storage is good for long term archival storage, in terms of day to day use for random retrievals… it isn’t going to work
Quantum External LTO 8 Drive. Interestingly, most drives are made by IBM, HPE, Quantum and Sony – with a lot of rebadging going on – Image, Quantum.
- RDX Removal Storage Drives – Have you sat down to the do the numbers with those things? Whilst RDX is cheaper than tape, you are still paying £130 a disk (multiply out as needed), A drive holder and backup software. Not cheap in the longer term
HP RDX Backup system with RDX Cartridges – Image, HPE
Upgrading the MacBook Pro
- Umm. Good luck squeezing a 6tb spinning disk inside a MacBook Pro. I’ll sit here and wait. And step away from the angle grinder if you’d be so kind.
- I could consider switching to a Gaming Style Laptop or a CAD workstation – the trouble is, these laptops aren’t exactly portable – they’re more luggable. And they chew into the precious hand-baggage allowances if I dare to take it on the road.
Use more External Hard Disks
- No. Just no. Why? Figure one ladies and gentlemen. Figure… one.
There’s about 12Tb of storage here. Roughly. There’s also about another 4TB of camera. I make no apologies for this unholy pile of hard disks.
Cloud Storage is a no-no…
But what about the cloud? Everything is free in the cloud isn’t it – an untapped river of storage!
Except it isn’t. At the Gigabyte level, you can get lots of low-cost and cheap options. When you start hitting Terabytes, things get very pricey, very quickly
- Cloud storage such as Amazon Glacier (which as well as having upload costs, has download costs). https://aws.amazon.com/glacier/. – Whilst it might be US$0.04 per Gb, that works out to US$55 a month for 10TB, with what I would call a reasonable request queue
- BlackBlaze? A mere $727 a year using the B2B backup service. (although its better value than Amazon S3, Azure Storage, or Google Cloud)
So Cloud storage is out, as well as some of the more complex storage mechanisms. So we’re back to Network Storage, and in particular – Network Attached Storage (NAS).
NAS Hardware
With the cheapest Windows-based solution going for £649 (without hard disks), there are other options out there – mostly Network Attached Storage boxes.
Qtap, Theseus, Drobo and Synology are the bigger players here – and when buying in a NAS you’re looking for:
- Ease of setup
- Connectivity
- Transfer rates
- Migration
In my example, I wanted the ease of setup and ease of upgrading in the longer term. I, therefore, chose a Drobo 5N2 from Amazon UK,
Most NAS’s have their own requirements, may require matching disks and so on. It’s best to check before you buy. The beauty of a Drobo is that you don’t need matching hard disks of equal capacity – rather if you have hard drives of different sizes, it can create a working, protected array across them
The base unit costs £484 or so.
It’s gone up since I brought it – mine was £470 or so – https://www.amazon.co.uk/Drobo-DRDS5A31-5N2-Desktop-Enclosure/dp/B06XGN6L4S/ref=sr_1_2_sspa?crid=1CKW3GF2ZMHI6&keywords=drobo+5n2&qid=1558700076&s=gateway&sprefix=drobo%2Caps%2C146&sr=8-2-spons&psc=1
There’s also something important to consider at this point – Data Redundancy. Because the last thing you want is your disk to give up on your – and all the data going *foom*
To RAID
So we are going to be looking at a local solution all being well. One technology that can help me is RAID – Redundant Array of Inexpensive Disks. RAID is nothing new – it’s been around for years but has some good ways of protecting data.
- RAID0 – Offers no disk protection, but stripes data. Good for speed, but offers no protection
- RAID1 – Allows for disk mirroring – the basic level of protection, without parity or striping
- RAID5 – Allows for 1 disk failure with block level parity and striping
- RAID6 – allows for 2 disks of failure – double block level parity and striping
Wikipedia has a great introduction to the RAID levels and how they perform. But here’s a handle table from the article that explains some of the math, and your considerations when designing storage that is fault tolerant to N-1 or N-2.
Standard RAID levels – https://en.wikipedia.org/wiki/Standard_RAID_levels
RAID 1/5/6 works well data protection – especially if your drives are hot-swappable (ie – you can remove a dead disk whilst a device is active – and yes, I’ve done this more than once in my life with HP Proliant ML/DL Enterprise Servers… and I still get nervous when doing it). Some servers will demand its done in a cold-swappable way (ie – the system powered off), so if keeping your data accessible whilst your disks are repairing is important – hot-swappable is the way to go.
Just remember, large hard disks take time equal a large amount of time to rebuild an array though. Unfortunately – this is the real world – and large hard disks aren’t a nicety these days– they’re a requirement when it comes to mass data storage – and during the rebuild time, if another disk fails – your data will go *foom*.
RAID6 is is the ideal with two-disk redundancy levels – let’s not bet around the bush – but at the cost of two disks in your array being use for party. The downside is you lose two disks worth of storage. But in most cases, RAID5 offers enough protection… providing you have a backup.
To reiterate: RAID is not a backup solution.
Wait? Does this thing need a backup too? Yes, it does.
That’s another discussion for another day on how I’m going to back this storage up (and I might have an idea. Ok – I haven’t, but I do have a strategy…. well… maybe).
Also, disks are required to be the same size in a RAID, or else they’ll all format down to the smallest possible disk size. And if you’re growing out a storage platform – that could be a problem down the line (for Drobo Users, BeyondRAID helps in terms of provisioning the space in another way – another reason why I went for Drobo in case I wish to scale to 10TB disks in a few years time).
Disks
Whilst I could buy SATA/M2/NvME Solid State Disks, there is a simple fact – that at the high capacities, they are expensive for what they are. Thus it’s back to spinning tin… I mean traditional mechanical hard disks.
Thankfully, I had a little windfall that allowed me to stack up a lot of storage at once.
At this point, it becomes a rather simple choice of who you trust when it comes to hard disk storage – be it Seagate, Toshiba or Western Digital.
Based on my experience – I have the best luck with Western Digital equipment. I’ve chosen Western Digital Red Hard disks – which are rated for NAS usage (as spinning disks in a box cause a fair bit of vibration)
Five of those please – Scan UK
For those who are purists of one brand or another – you can’t go wrong with any of them in most respects. BackBlaze has its ever interesting reliability stats to help you decide.
I can put in an SSD with my Drobo to accelerate it (akin to a Tiered storage/Cache Accelerator). For now, I’m skipping this as most of the work on the disk will be repeated writes, rather than high-speed reads. Again, if your needs differ, you may want to factor something like that in.
So, with all the Drobo and drives arrived – it’s time to build out the kit. Let’s have some fun!
Next: The Build-Out
Notes:
- I’ve concentrated a lot of HP Enterprise equipment in this article. Mainly as what I’ve used in the past when doing deployments. Dell PowerEdge and Lenovo Think Servers are all probably fine servers. Your enterprise may and will differ. And no, I’m not talking about NetApps or anything like that. This is for a home… not a multi-million-pound enterprise…
- No, I’m not going corporate bin raiding – most of that kit ends up on eBay anyway.
- Don’t skimp on hard disks or go second hand for them. It ends badly every single time. Buy new or manufacture-refurbished.
- Everyone hates Microsoft Licensing. And I mean everyone.
Welcome to Economy Class and Beyond – Your no-nonsense guide to network news, honest reviews, with in-depth coverage, unique research as well as the humour and madness as I only know how to deliver.
Follow me on Twitter at @EconomyBeyond for the latest updates! You can also follow me on Instagram too!
Also remember that as well as being part of BoardingArea, we’re also part of BoardingArea.eu, delivering frequent flyer news, miles and points to the European Frequent Flyer
