To be honest, it seems rather silly to go this deep into probability calculations when your N = 4 (hard drives). I can present numerous wildly-differing anecdotes about how single hard drives I have owned have fared. I have some drives that are 20 years old that still apparently work fine. I've had brand new drives silently corrupt sectors after less than a year in use. Trying to put hard numbers to the probability of failures is a fool's errand. Better to simply assume that components will fail over time, and architect your system to handle those failures gracefully.
That said, I actually agree with much of the final architecture decisions made by the author. BTRFS raid1 with an offsite copy in a geographically distant area is a pretty good solution for the average person's personal documents. Possible improvements would be:
- Multiple offsite copies in different locations
- Hardware diversity (differing models of HDD, host PC, power supplies, etc.)
- Staggered hardware upgrades on a 5-10 year cycle
oh it's definitely a bit absurd, but i felt i needed a workout.... hence this post. multiple sites, hardware diversity, staggered upgrades, cold storage, are all really good ways to improve this, as they pull failure events closer to independence
> the single best thing you can do to preserve your data is add redundancy.
The single best thing you can do to preserve the information is to put your data in the hands of a full time engineering team with redundant, on-call support whose sole purpose is to make sure that no bits are ever lost or flipped.
An ability to allocate even 40 hours a week to maintaining a highly durable DIY NAS solution doesn't even show up next to this. Any perception to the contrary is merely an ego trip. Control != durability. Do not conflate these things.
The problem with these extremely unlikely events is that it's really hard to reason about them because we don't have a lot of samples to draw from. My handful of personal samples tell me that lightning is the ultimate equalizer and that otherwise hard drives are mostly immortal. Clearly this is not an accurate depiction of the actual statistical distribution, but it's all I can come up with having N ~= 100. Backblaze and Amazon have owned way more hard drives than I can even imagine. They can get much closer to modeling actual reality.
I understand for certain classes of data (every anime show ever produced by Japan), you might not want to actually pay for the kind of durability guarantees that a big provider can offer. However, what about your other documents? Tax and legal forms? Are we comfortable storing this in the same way across 2 different residential sites?
What a long and convoluted way to arrive at the most obvious conclusion. Use redundant storage (mirrors) for important data and have backups (3-2-1 rule).
Don’t waste money on redundancy if your data can be easily reacquired.
Anyone who mentions SMART and reliability in the one sentence isn't worth reading - sorry OP.
I hve had bazillions of drives with a perfect SMART - which were unusable, and mamy others which had SMART throwing OSHA at me - yet they worked for decades, literally.
iirc i address this in a footnote --- absence of SMART errors has very little predictive power, presence has slightly more but it depends which errors they are and how they accumulate
BTW transfer errors aren't indicating the drive errors. SATA is a quite nimble protocol so CRC errors do indicate the problem between the drive and the controller.
The no capitals is really hard for me. Especially given the font "i" makes me think of the imaginary number and I get mentally derailed even though I know it's not. Or maybe it's just eyestrain
I understand why you might for stylistic reasons, but a toggle would be nice!
The author explicitly mentioned syncing to his parents house. Even the "RAID is not backup" should be more nuanced IMO once you are talking about FS that supports regular snapshots.
RAID is not backup. Snapshots on same storage are not backup. For “the oh I don’t like this edit of my Word doc, do I have a version from yesterday”, sure, snapshots. For momma giving birth photos, you need backup. S3 is $1/tb for backup. Blu-rays are metal not ink now, discs are $2/25gb which might work for older photos. LTO6 drives can be had for under $1000.
> i want to keep my data for my entire life.
NAS has very little part in this conversation. In this conversation, a NAS is the same category a desktop drive: it is better than a laptop that you can lose.
the reason RAID is not backup, aside from snapshotting, is colocated drives (same faulty machine, same flammable location, same malicious thunderhead) have a far higher chance of failing at the same time due to evnironmental effects --- hence the decision to have half the drives stored remotely. having a third copy, stored cold (blu-rays are a good idea) helps, but only if its failure is sufficiently independent af the others to make a real dent in the probabilities (e.g., a safe-deposit box or a friend's house, not where any of the drives are!)
To be honest, it seems rather silly to go this deep into probability calculations when your N = 4 (hard drives). I can present numerous wildly-differing anecdotes about how single hard drives I have owned have fared. I have some drives that are 20 years old that still apparently work fine. I've had brand new drives silently corrupt sectors after less than a year in use. Trying to put hard numbers to the probability of failures is a fool's errand. Better to simply assume that components will fail over time, and architect your system to handle those failures gracefully.
That said, I actually agree with much of the final architecture decisions made by the author. BTRFS raid1 with an offsite copy in a geographically distant area is a pretty good solution for the average person's personal documents. Possible improvements would be:
- Multiple offsite copies in different locations
- Hardware diversity (differing models of HDD, host PC, power supplies, etc.)
- Staggered hardware upgrades on a 5-10 year cycle
- Cold storage (offline) copy
oh it's definitely a bit absurd, but i felt i needed a workout.... hence this post. multiple sites, hardware diversity, staggered upgrades, cold storage, are all really good ways to improve this, as they pull failure events closer to independence
3-2-car is still the best model for your Average Joe.
> the single best thing you can do to preserve your data is add redundancy.
The single best thing you can do to preserve the information is to put your data in the hands of a full time engineering team with redundant, on-call support whose sole purpose is to make sure that no bits are ever lost or flipped.
An ability to allocate even 40 hours a week to maintaining a highly durable DIY NAS solution doesn't even show up next to this. Any perception to the contrary is merely an ego trip. Control != durability. Do not conflate these things.
The problem with these extremely unlikely events is that it's really hard to reason about them because we don't have a lot of samples to draw from. My handful of personal samples tell me that lightning is the ultimate equalizer and that otherwise hard drives are mostly immortal. Clearly this is not an accurate depiction of the actual statistical distribution, but it's all I can come up with having N ~= 100. Backblaze and Amazon have owned way more hard drives than I can even imagine. They can get much closer to modeling actual reality.
I understand for certain classes of data (every anime show ever produced by Japan), you might not want to actually pay for the kind of durability guarantees that a big provider can offer. However, what about your other documents? Tax and legal forms? Are we comfortable storing this in the same way across 2 different residential sites?
What a long and convoluted way to arrive at the most obvious conclusion. Use redundant storage (mirrors) for important data and have backups (3-2-1 rule). Don’t waste money on redundancy if your data can be easily reacquired.
Anyone who mentions SMART and reliability in the one sentence isn't worth reading - sorry OP.
I hve had bazillions of drives with a perfect SMART - which were unusable, and mamy others which had SMART throwing OSHA at me - yet they worked for decades, literally.
iirc i address this in a footnote --- absence of SMART errors has very little predictive power, presence has slightly more but it depends which errors they are and how they accumulate
BTW transfer errors aren't indicating the drive errors. SATA is a quite nimble protocol so CRC errors do indicate the problem between the drive and the controller.
off topic but I really like the design of the site
Instantly recognisable as Tufte
https://edwardtufte.github.io/tufte-css
definitely cribbed from tufte, with a few tweaks.
The overall design is fine, but I find the writing style of “fully justified with no capitals” difficult to read for something of that length.
The no capitals is really hard for me. Especially given the font "i" makes me think of the imaginary number and I get mentally derailed even though I know it's not. Or maybe it's just eyestrain
I understand why you might for stylistic reasons, but a toggle would be nice!
Came here to say this too. :-)
Thanks OP for making it so readable.
A NAS is not a backup device.
The author explicitly mentioned syncing to his parents house. Even the "RAID is not backup" should be more nuanced IMO once you are talking about FS that supports regular snapshots.
RAID is not backup. Snapshots on same storage are not backup. For “the oh I don’t like this edit of my Word doc, do I have a version from yesterday”, sure, snapshots. For momma giving birth photos, you need backup. S3 is $1/tb for backup. Blu-rays are metal not ink now, discs are $2/25gb which might work for older photos. LTO6 drives can be had for under $1000.
> i want to keep my data for my entire life.
NAS has very little part in this conversation. In this conversation, a NAS is the same category a desktop drive: it is better than a laptop that you can lose.
the reason RAID is not backup, aside from snapshotting, is colocated drives (same faulty machine, same flammable location, same malicious thunderhead) have a far higher chance of failing at the same time due to evnironmental effects --- hence the decision to have half the drives stored remotely. having a third copy, stored cold (blu-rays are a good idea) helps, but only if its failure is sufficiently independent af the others to make a real dent in the probabilities (e.g., a safe-deposit box or a friend's house, not where any of the drives are!)