A few days ago I wrote to the senior vice president of marketing and business at Seagate, Jeff Fochtman.
... Seagate, a global company, one of — if not the — largest companies in the data storage industry. Seagate is such a superpower that it's safe to say a very large portion of the entire "web", "cloud" or whatever you would like to call it is kept on Seagate storage drives. It is practically impossible for you to go a single day without directly or indirectly using a Seagate product.
... and Jeff Fochtman? He's one of the first people you see when you go to Seagate's "our story" page on their website. This is a man who needs approval from a PR team before he can officially respond to questions publicly.
Long story short: one of Seagate's top executives graciously donated some of his extremely valuable time to answer a few basic questions that a nerd like me had about archiving personal data.
He also asked for absolutely nothing in return. I was given no suggestions or guidance on how to write anything in this post.
I cannot thank Jeff and his team enough, and I cannot recommend Seagate and LaCie products enough.
TL;DR
In short, Jeff says that if you're serious about backing up your data:
- First of all, the basis of any archiving or backing up is a 3-copy system. Keep at least 3 copies of your data: 2 on-site (in your house/office) and 1 off-site (e.g. in the cloud).
- For the copies you plan to keep at home, use an HDD-based storage solution. Jeff suggests the Seagate Expansion External drive if you want to keep things simple (but if you're a bit more "techy", working with bare drives is more economical, see my drive cost comparison spreadsheet).
- Generally speaking, the warranty period for an HDD is a good measure of its longevity.
- Replace your HDDs/storage units before their warranties expire (for most HDDs, that's every 2–5 years).
- If you have a lower risk tolerance, you should replace your drives before their warranties expire (e.g. every 3 years for a drive with a 5-year warranty).
- Don't use SSDs for archiving/backups, they're just too expensive.
The best hard drives for archiving
For archiving 1–18TB of data, your best option is any appropriately-sized Seagate IronWolf Pro HDD.
For archiving 18–100TB or more of data, I recommend the appropriate multiple of the Seagate Exos X16 14TB HDD (it is the most economical HDD for archiving on the market today).
But you cannot go wrong with any appropriately-sized HDD(s) from any of these product lines:
- Seagate Exos X16
- Seagate Exos X14
- Seagate Exos X10
- Toshiba MG Series
- Seagate IronWolf Pro
- Western Digital WD_BLACK Performance
- Western Digital WD Gold Enterprise Class
If you will be buying enterprise HDDs, remember that they often come in SATA and SAS interface versions. If you do not know which to buy, you want to buy the SATA interface version.
According to my research, a good rule of thumb is that the best drives for archiving:
- Have 5-year warranties.
- Cost $20–40 per TB.
Do you really need to replace archive drives?
Yes, replacing archive drives is necessary.
I know we would all love to just fill a drive with data, put it in a closet, and let it sit there in immortality "just in case" for a doomsday scenerio 30, 40, or 100 years from now... but there is no economical, easily-accessable and manipulable storage medium on the market today that can promise that kind of data retention.
You have to just accept drive replacement as part of the archiving process.
As things stand today, HDDs are the most reasonable and economical things you can use to archive moderate (1–100TB) amounts of data.
Replacing all of your archive drives every 2–5 years may seem extreme and expensive. By Jeff's own admission: "an HDD is like a car"... it often lasts well beyond its warranty period... but as also pointed out by Jeff: "storage is relatively cheap", so if you don't want to risk losing priceless photos of your child's first steps or you absolutely cannot afford to lose backups of your clients' data, replace your HDDs regularly.
Jeff also says that if this type of solution seems too expensive for you, you probably have a LOT of data to archive and you should look into more "enterprise-like" solutions.
Is 1 backup enough?
No. You should keep at least 3 copies of all the data you do not want to lose on 3 different storage mediums.
Jeff says this. I say this. Everybody who has ever lost all their family photos because they were kept on a single drive and the drive failed says this.
This post focuses on the drives you should use to keep at least one of those copies.
A solid 3-copy system could look like this:
- Copy 1: a "live" copy on the drive(s) inside your computer, on a NAS, etc. These are the files you work with every day.
- Copy 2: another copy of your data at a cloud storage provider (e.g. Google Drive, iCloud, AWS, etc.) These can/should automatically sync with your "live" data and allow you to quickly get back up and running in case your main storage unexpectedly fails.
- Copy 3: a third copy of all your data on backup drives you keep in your closet/basement/safe/secret room behind a bookcase/etc. These are the drives this blog post is focusing on. These you should manually fill up on a regular basis. These are like a last resort; they probably aren't as up-to-date as the first and second data locations, but they protect you in a doomsday scenerio where somehow your first and second copies are destroyed on the same day.
I mean really, each copy in the 3-copy system is just protecting the other 2 copies.
When you keep 3 copies of your data in 3 distinctly different storage mediums (one of them being off-site), you can sleep easy knowing there is no realistic scenerio that will cause you to lose your data.
If you want to learn more about how to employ a 3-copy (AKA "2+1 " or "3-2-1") backup plan, you can read my blog post about how I back up my data, this Seagate page (geared towards SMBs), or watch this awesome Linus Tech Tips video.
How much does archiving data cost?
$5–14 per TB, per year, per copy (If done correctly).
That's cheap. You can personally archive 2 copies of 14TB of data for the price of a standard Netflix plan.
Your actual costs will depend on how much data you need to archive, what hardware you buy, market conditions, etc. The $5–14 figure assumes you have at least 1TB of data to archive, buy 5-year warrantied HDDs, and just as an extra precaution: replace them 1 year early (every 4 years).
And keep in mind: that's based on today's pricing. I don't predict HDD pricing will be very volatile in the coming years, but you never know what the future holds.
What is the best storage drive for archiving?
For most people, the best drive for archiving will be an appropriately-sized Seagate IronWolf Pro HDD.
By "most people" I mean people who have less than 18TB of data to archive. Seagate consistently provides a solid value across the entire IronWolf Pro line of HDDs.
You cannot go wrong purchasing any IronWolf Pro HDD for archiving less than 18TB of data.
But for more specific recommendations, please read on...
Based on what Jeff said, I made a spreadsheet of over 250 drives currenty for sale from Seagate, Western Digital and Toshiba and compared their prices, storage capacities, and warranties. Then I came up with each drive's cost per TB, per year.
Here is a link to the full spreadsheet. Feel free to make a copy of it and use it however you'd like.
Here are my more specific recommendations, based on how much data you need to archive:
For archiving less than 1TB of data
Get a 1TB drive from any of these product lines:
- Seagate FireCude
- Seagate BarraCuda Pro
- Western Digital WD_Black
For archiving 1–8TB of data
Get an appropriately-sized drive from one of these product lines:
- Seagate IronWolf Pro
- Western Digital Gold
- Western Digital WD_Black
For archiving 8–18TB of data
Get an (or a few) appropriately-sized drive(s) from one of these product lines:
- Seagate IronWolf Pro
- Toshiba MG
- Seagate Exos X18
- Seagate Exos X16
- Seagate Exos X14
- Western Digital WD_Black
- Western Digital Gold
For Archving 18–100TB of data
For this much data, you will probably want to buy some multiple of one of these drives:
- Seagate Exos X16 (14TB or 16TB models)
- Seagate Exos X14 (10TB or 14TB models)
- Toshiba MG (10TB, 12TB, or 14TB models)
LTO Tapes
For completeness, I'm going to mention LTO tapes (if you read Jeff's answers, you'll see that he did too).
But just to be clear: if you are not planning on archiving at least 50TB of data, LTO tapes are probably not worth it.
The 8th generation of LTO technology (LTO-8) is probably the most relevant/realistic generation that you would use for personal archiving today.
LTO-8 tapes are much cheaper than HDDs; they can cost as little as $8.75 per TB (uncompressed storage).
And they last much longer — they are rated for 30 years of data retention.
That's a yearly cost as low as $0.30 per TB!!
$0.30 per TB per year!! The most economical HDD I found in all my research was the Seagate Exos X16 14TB and it came in at $4.06 per TB per year (over 13x more expensive)!!
So why not LTO tapes?
There is really only 1 major disadvantage to LTO tapes for archiving: equipment cost. You need to buy an LTO-8 tape drive to read from and write to LTO tapes. A drive like this can cost upwards of $3000, used. A nice, new "tape library" that stores and manages up to 50 LTO cartridges (600TB of uncompressed data) starts at around $8,000.
That said, if you can bare the cost of an LTO-8 tape drive up-front, the investment can make a lot of sense, especially if you are planning to archive well over 50TB of data.
Data degradation and mechanical HDD failure
Jeff made no mention of data degradation or about HDDs e.g. seizing up due to lack of use. Those were my primary concerns when it comes to my on-site archival data HDDs.
When I asked a follow up question regarding whether the replacement cycle Jeff suggests is to prevent such failures from affecting live data, he said "[it's] more HDD best practices in general and not as a specific action to remedy a specific issue".
I'm sure a part of those best practices exist to avoid these failures... but going off of Jeff's responses, as long as you commit to the regular replacement model: data degradation or mechanical HDD failure should not be something to lose sleep over.
Where should you keep your archive hard drives?
Putting your archive drives Into external enclosures or antistatic bags and keeping them in a Pelican case is a great option.
Pelican cases are great, but keep in mind that Pelican's standard foam is not ESD-safe, so you should not keep raw hard drives (i.e. hard drives that are not in any kind of enclosure) in the standard Pelican "Pick N Pluck" foam. All you need to do is put them in antistatic bags (A.K.A. "ESD bags") first. Antistatic bags are really cheap (you can buy a 100-pack for less than $20 on Amazon at the time of writing this).
Worrying about ESD might be a little bit of overkill, but considering an antistatic bag costs just a few cents, it's worth the peace of mind.
Whether you put them in a Pelican case or not, keep your archive drives:
- in a cool, dry place;
- in antistatic bags or external enclosures; and
- somewhere where they are not at risk of being dropped.
The full interview with Jeff
Here are the questions I sent Jeff and his unadulterated answers:
The "archiving" workload I trying to help people plan out most efficiently looks essentially like this: fill a drive with data -> store it in a closet -> read from it (at most) a few times a year.
What is the best HDD to purchase from Seagate for the workflow mentioned above?
JF – A simple Seagate Expansion desktop external desktop drive is the short answer, when archiving you want to focus on value and that product packs the best value-per-GB. The other thing is that you want to run a three-copy system, for any important file you want three copies, with at least one of the copies at an alternate location (like cloud back-up or a secondary physical archive location). The fact is storage is relatively cheap so focus on value but have a two+one location archive process. An example of this would be to have one back-up drive in current use, one copy on archive in the closet and a third copy in the cloud or at alternate ‘closet’ location. I personally have two-physical backups and one-cloud services backup of my photo and video content.
Assuming I purchased that HDD from you (brand new, today) and filled it completely or nearly completely with data (e.g. photo, video, and text files)... then put the drive into a closet and left it UNTOUCHED... how long should I expect the drive to be fully-functional and all the data on it to still be readable?
JF – Most of our warranties are five-years, that’s a good barometer for lifespan, but an HDD is like a car -- they can usually last longer, and a very, very low percentage may last shorter. If you are serious about archiving or are a professional photographer, I would do three-year rotations and purchase a new archive drive in that timeframe. That may alarm people to hear that buying three fully new archive drives (to fit the two+one archive model above) is my suggestion but the fact is you get better value on GB-to-price over time and the ‘nesting’ of archive files on new drives becomes fairly easy and not overly expensive in most user scenarios. If that model is overly costly it’s probably because you have such a large amount of data and you should be looking into a more ‘enterprise like’ SAN (Storage Area Network) or public cloud backup plan.
If I plugged the drive into a computer every once in a while, and allowed it to operate for a few minutes/hours, would the lifespan of the data and/or HDD be considerably longer? If so, how often should this be done?
JF – Occasional usage can help ensure a longer total lifespan but it’s not necessary for the time windows of usage listed above. Pulling it out at least once a year is a good idea for a lot of reasons. Most importantly I believe you need to be scheduling your archives much more often than once a year so would focus more on your archive intervals than just spinning them up for fun.
If I formatted the drive and re-wrote all the same data to it every once in a while, would the lifespan of the data and/or HDD be considerably longer? If so, how often should this be done?
JF – No, the formatting would not provide any real benefit to lifespan or longevity.
What is the best SSD to purchase from Seagate for the workflow mentioned above?
JF – For archive I would not use SSD, simply because HDD’s are way cheaper and you can build-in redundancy (like having three copies) at a lower cost.
If I purchased that SSD from you (brand new, today)... filled it completely... then put the drive into a closet and left it untouched... how long should I expect the drive to be fully-functional and all the data to still be readable?
JF – The five-year warranty [is] still good [a] barometer. Should last even longer... but any electronic part can break, so why risk it.
If I plugged the drive into a computer every once in a while, and allowed it to operate for a few minutes/hours, would the lifespan of the data and/or SSD be considerably longer? If so, how often should this be done?
JF – Only if you stand on your head while doing so… I’m joking! It’s not something you need to do, so I would focus more on your archiving velocity and rotation.
If I formatted the SSD and re-wrote all the same data to it every once in a while, would the lifespan be considerably longer? If so, how often should this be done?
JF – nope
If you personally had to archive 10TB of data for 100 years for a reasonable price, how would you do it?
JF – I would stick one copy in the cloud (probably on Amazon glacier), and two other copies on 12TB Seagate external drives that I would change every three to five years.
If you personally had to archive 1PB of data for 100 years for a reasonable price, how would you do it?
JF – I would buy two x 1.2PB racks of Seagate EXOS systems and host those in a single full rackmount enclosure in an on-prem[ises] closet or a colocation a datacenter. The other copy I would provision in my enterprise’s chosen archive-tier cloud. You could do a tape archive solution here depending on the SLA (service-level agreement) … but I’ll stick with my first answer.
One more big thank you to Jeff Fochtman, and a big thank you to YOU for reading this post! Both you and Jeff are some really awesome people. 😉
