In the late 1990s the standard rule of thumb for building a new server was to put the operating system onto its own, small, RAID 1 array and separate out applications and data into a separate RAID 5 array.\u00a0 This was done for several reasons, many of which have swirled away from us, lost in the sands of time.\u00a0 The main driving factors were that storage capacity was extremely expensive, disks were small, filesystems corrupted regularly and physical hard drives failed at a very high rate compared to other types of failures.\u00a0 People were driven by a need to protect against physical hard drive failures, protect against filesystem corruption and acquire enough capacity to meet their needs.<\/p>\n
Today the storage landscape has changed.\u00a0 Filesystems are incredibly robust and corruption from the filesystem itself is almost unheard of and, thank to technologies like journalling, can almost always be corrected quickly and effectively protecting the end users from data loss.\u00a0 Almost no one worried about filesystem corruption today.<\/p>\n
Modern filesystem are also able to handle far more capacity than they could previously.\u00a0 It was not uncommon in the late 1990s and early 2000s to have the ability to easily make a drive array larger than any single filesystem could handle. Today that is not reasonably the case as all common filesystems handle many terabytes at least and often petabytes, exabytes or more of data.<\/p>\n
Hard drives are much more reliable than they were in the late 1990s.\u00a0 Failure rates for an entire drive failing are very low, even in less expensive drives.\u00a0 So low, in fact, that array failures (data loss in the entire RAID array) is concerned with failing arrays primarily, rather than the failure of hard drives.\u00a0 We no longer replace hard drives with wild abandon.\u00a0 It is not unheard of for large arrays to run their entire lifespans without losing a single drive.<\/p>\n
Capacities have scaled dramatically.\u00a0 Instead of 4.3GB hard drives we are installing 3TB drives.\u00a0 Nearly one thousand times more capacity on a single spindle compared to less than fifteen years ago.<\/p>\n
These factors come together to create a need for a dramatically different approach to server storage design and a change to the “rule of thumb” about where to start when designing storage.<\/p>\n
The old approach can be written RAID 1 + RAID 5.\u00a0 The RAID 1 space was used for the operating system while the RAID 5 space, presumably much larger, was used for data and applications.\u00a0 This design split the two storage concerns putting maximum effort into protecting the operating system (which was very hard to recover in case of disaster and on which the data relied for accessibility) onto highly reliable RAID 1.\u00a0 Lower cost RAID 5, while somewhat riskier, was chosen, typically, for data because the cost of storing data on RAID 1 was too high in most cases.\u00a0 It was a tradeoff that made sense at the time.<\/p>\n
Today, with our very different concerns, a new approach is needed, and this new approach is known as “One Big RAID 10” – meaning a single, large RAID 10 array with operating system, applications and data all stored together.\u00a0 Of course, this is just what we say to make it handy, in a system without the needs of performance or capacity beyond a single disk we would say “One Big RAID 1”, but many people include RAID 1 in the RAID 10 group so it is just easier to say the former.<\/p>\n
To be even handier, we abbreviate this to OBR10.<\/p>\n
Because the cost of storage has dropped considerably and instead of being at a premium is typically in abundance today, because filesystems are incredibly reliable, because RAID 1 and RAID 10 share performance characteristics and because non-disk failure triggered array failures have moved from background noise to primary causes of data loss the move to RAID 10 and to eliminate array splitting has become the new standard approach.<\/p>\n
With RAID 10 we now have the highly available and resilient storage previously held only for the operating system available to all of our data.\u00a0 We get the benefit of mirrored RAID performance plus the benefit of extra spindles for all of our data.\u00a0 We get better drive capacity utilization and performance based on that improved utilization.<\/p>\n
Even the traditional splitting of log files normally done with databases (the infamous RAID 1 + RAID 5 + RAID 1 approach) is no longer needed because RAID 10 keeps the optimum performance characteristics across all data.\u00a0 With RAID 10 we eliminate almost all of the factors that once caused us to split arrays.<\/p>\n
The only significant factor, that has not been mentioned, for which split arrays were traditionally seen as beneficial is access contention – the need for different processes to need access to different parts of the disk at the same time causing the drive head to move around in a less than ideal pattern reducing drive performance.\u00a0 Contention was a big deal in the late 1990s when the old rule of thumb was developed.<\/p>\n
Today, drive contention still exists but has been heavily mitigated by the use of large RAID caches.\u00a0 In the late 90s drive caches were a few megabytes at best and often non-existent.\u00a0 Today 256MB is a tiny cache and average servers are deployed with 1-2GB of cache on the RAID card alone.\u00a0 Some systems are beginning to integrate additional solid state drive based caches to add a secondary cache beyond the memory cache on the controller.\u00a0 These can easily add hundreds of gigabytes of extremely high speed cache that can buffer nearly any spindle operation from needing to worry about contention.\u00a0 So the issue of contention has been solved in other ways over the years but has, like other technology changes, effectively freed us from the traditional concerns requiring us to split arrays.<\/p>\n
Like array contention, another, far less common reason for splitting arrays in the late 1990s was to improve communications bus performance because of the limitations of the era’s SCSI and ATA technologies.\u00a0 These, too, have been eliminated with the move to serial communications mechanisms, SAS and SATA, in modern arrays.\u00a0 We are no longer limited to the capacity of a single bus for each array and can grow much larger with much more flexibility than previously.\u00a0 Bus contention has been all but eliminated.<\/p>\n
If there is a need to split off space for protection, such as log file growth, this can be achieved through partitioning rather than through physical array splitting.\u00a0 In general you will want to minimize partitioning as it increases overhead and lowers the ability of the drives to tune themselves but there are cases where it is the better approach.\u00a0 But it does not require that the underlying physical storage be split as it traditionally was.\u00a0 Even better than partitioning, when available, is logical volume management which makes partition-like separations without the limitations of partitions.<\/p>\n
So at the end of the day, the new rule of thumb for server storage is “One Big RAID 10.”\u00a0 No more RAID 5, no more array splitting.\u00a0 It’s about reliability, performance, ease of management and moderate cost effectiveness.\u00a0 Like all rules of thumb, this does not apply to every single instance, but it does apply much more broadly than the old standard ever did.\u00a0 RAID 1 + RAID 5, as a standard, was always an attempt to “make due” with something undesirable and to make the best of a bad situation.\u00a0\u00a0 OBR10 is not like that.\u00a0 The new standard is a desired standard – it is how we actually want to run, not something with which we have been “stuck”.<\/p>\n
When designing storage for a new server, start with OBR10 and only move away from it when it specifically does not meet your technology needs.\u00a0 You should never have to justify using OBR10, only justify not<\/em> using it.<\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" In the late 1990s the standard rule of thumb for building a new server was to put the operating system onto its own, small, RAID 1 array and separate out applications and data into a separate RAID 5 array.\u00a0 This was done for several reasons, many of which have swirled away from us, lost in … Continue reading One Big RAID 10 – A New Standard in Server Storage<\/span>