My solution is RAIDZ5 and storing the backup on LTO6 tape with parity/erasure code. I think the fact that scrub times take 24 hours even on 16TB drives is already over the safety margin. If a drive failure happens, the first thing I’ll do to run a manual diff backup which should take a fraction of the time and then run the ZFS resilver.
I’m beginning to see why SSD RAID is being considered now. My guess for HDDs in enterprise is that a RAID 15 (I made this up) would be considered. What I mean is data is stored on two identical servers each running RAID5 or 6. Off the shelf solutions like Gluster exist and that seems to be gaining traction at least according to Linus Tech Tips.
SSD RAID is actually very common outside of home use! And yeah, clustered filesystems help overcome many of these limitations, but tend to be extremely demanding (expensive hardware for comparable performance). Network almost immediately becomes the bottleneck. Even forgetting about latency and other network efficiency concerns, 100 Gbps isn’t that fast when you have individual devices approaching 16 Gbps.
Will raid 6 still be viable at this size or will this require something like raid 10 or even moving beyond raid.
ZFS has triple parity support for RAID-Z (basically RAID-5/RAID-6/RAID-7 with better data safety guarantees), so there’s that.
My solution is RAIDZ5 and storing the backup on LTO6 tape with parity/erasure code. I think the fact that scrub times take 24 hours even on 16TB drives is already over the safety margin. If a drive failure happens, the first thing I’ll do to run a manual diff backup which should take a fraction of the time and then run the ZFS resilver.
I’m beginning to see why SSD RAID is being considered now. My guess for HDDs in enterprise is that a RAID 15 (I made this up) would be considered. What I mean is data is stored on two identical servers each running RAID5 or 6. Off the shelf solutions like Gluster exist and that seems to be gaining traction at least according to Linus Tech Tips.
SSD RAID is actually very common outside of home use! And yeah, clustered filesystems help overcome many of these limitations, but tend to be extremely demanding (expensive hardware for comparable performance). Network almost immediately becomes the bottleneck. Even forgetting about latency and other network efficiency concerns, 100 Gbps isn’t that fast when you have individual devices approaching 16 Gbps.