It is becoming increasingly common to forgo traditional, proprietary storage devices, both NAS and SAN, and instead using off the shelf hardware and installing a storage operating system on it for, what many call, “do it yourself” storage servers.\u00a0 This, of course, is a misnomer since no one calls a normal file server “do it yourself” just because you installed Windows yourself.\u00a0 Storage has a lot of myth and legend swirling around it and people often panic when the they think of installing Windows and calling it NAS rather than calling it a file server.\u00a0 So, if it makes you feel better, use terms like file server or storage server rather than NAS and SAN – problem solved.\u00a0 This is a part of the “open storage” movement – moving storage systems from proprietary to standard.<\/p>\n
Choosing the right operating system for a storage server is important and not always that easy.\u00a0 I work extensively in this space and people often ask me what I recommend and the recommendations vary, based on scenario, and often seem confusing.\u00a0 But the factors are actually relatively easy, if you just know the limitations that create the choices and paths in the decision tree.<\/p>\n
Before choosing an OS we must stop and consider what our needs are going to be.\u00a0 Some areas that need to be considered are: capacity, performance, ease of administration, budget, connection technology, cost and clustering.\u00a0 There are two main categories of systems that we will consider as well, standard operating system or storage appliance operating system.\u00a0 The standard operating systems are Windows, Linux, Solaris and FreeBSD.\u00a0 The storage appliance operating systems are FreeNAS,\u00a0OpenFiler and NexentaStor.\u00a0 There are others in both categories but these are the main players currently.<\/p>\n
The first decision to be made is whether or not you or your organization is comfortable supporting a normal operating system operating in a storage server role.\u00a0 If you are looking at NAS then simply ask yourself if you could administer a file server.\u00a0 Administrating a block storage server (SAN) is a little more complex or, at least, unusual, so this might induce a small amount of concern but is really in line with other administration tasks.\u00a0 If the answer is yes, that using normal operating system tools and interfaces is acceptable to you, then simply rule out the “appliance” category right away.\u00a0 The appliance approach adds complexity and slows development and support cycles, so unless necessary is undesirable.<\/p>\n
Storage appliance operating systems exist only to provide a pre-packaged, “easy to use” view into running a storage server.\u00a0 In concept this is nice, but there are real problems with this method.\u00a0 The biggest problems come from the packaging process which pulls you a step away from the enterprise OS vendors themselves making your system more fragile, further behind in updates and features and less secure than the traditional OS counterparts.\u00a0 It also leaves you at the mercy of a very small company for OEM-level support when something goes wrong rather than with a large enterprise vendor with a massive user base and community.\u00a0 The appliancization process also strips features and options from the systems by necessity.\u00a0 In the end, you lose.<\/p>\n
Appliances are nice because you get a convenient web interface from which “anyone” can administer your storage.\u00a0 At least in theory.\u00a0 But in reality there are two concerns.\u00a0 The first is that there is always a need to drop into the operating system itself and fix things every once in a while.\u00a0 Having the custom web interface of the appliance makes this dramatically harder than normal so at the time when you most need the appliance nature of the system is when you do not have it.\u00a0 The second is that making something as critical as storage available for “anyone” to work on is a terrifying thought.\u00a0 There are few pieces of your infrastructure where you want more experience, planning and care taken than in storage.\u00a0 Making the system harder to use is not always a bad thing.<\/p>\n
If you are in need of the appliance system then primarily you are looking at FreeNAS and OpenFiler.\u00a0 NexentaStor offers a compelling product but it is not available in a free version and the cost can be onerous.\u00a0 The freely downloadable version appears to be free for the first 18TB of raw storage but the license states otherwise making this rarely the popular choice.\u00a0 (The cost of NexentaStor is high enough that purchasing a fully supported Solaris system would be less costly and provides full support from the original vendor rather than Nexenta which is essentially repackaging old versions of Solaris and ZFS.\u00a0 More modern code and updates are available less expensively from the original source.)<\/em><\/p>\n FreeNAS, outside of clustering, is the storage platform of choice in an appliancized package.\u00a0 It has the much touted ZFS filesystem which gives it flexibility and ease of use lacking in OpenFiler and other Linux-based alternatives.\u00a0 It also has a working iSCSI implementation so you can use FreeNAS safely as either a NAS or a SAN.\u00a0 Support for FreeNAS appears to be increasing with new developments being made regularly and features being retained.\u00a0 FreeNAS offers a large range of features and supported protocols.\u00a0 It is believed that clustering will be coming to FreeNAS in the future as well as this has recently been added to the underlying FreeBSD operating system.\u00a0 If so, FreeNAS will completely eliminate the need for OpenFiler in the marketplace.\u00a0 FreeNAS is completely free.<\/p>\n OpenFiler lacks a reliable iSCSI SAN implementation (unless you pay a fortune to have that part of the system replaced with a working component ) and is far more out of date than its competitors but does offer full block-level real-time replication allowing it to operate in a clustered mode for reliability .\u00a0 The issue here being that the handy web interface of the NAS appliance does not address this scenario and if you want to do this you will need to get your hands dirty on the command line, very dirty indeed.\u00a0 This is expert level stuff and anyone capable of even considering a project to make OpenFiler into a reliable cluster will be just as comfortable, and likely far more comfortable, building the entire cluster from scratch on their Linux distribution of choice.\u00a0 OpenFiler is built on the rather unpopular, and now completely discontinued, rPath Linux using the Conary packaging system both which are niche players, to say the least, in the Linux world.\u00a0 You’ll find little rPath support from other administrators and many packages and features that you may wish access to are unavailable.\u00a0 OpenFiler’s singular advantage of any significance is the availability of DRBD for clustering, which as stated above, in nonsensical.\u00a0 Support for OpenFiler appears to be waning with new features being non-existant and, in fact, key features like the AFP have been dropped rather than new features having been added.\u00a0 OpenFiler is free but key features, like reliable iSCSI, are not.\u00a0 Recent reports from OpenFiler users are that even non-iSCSI storage has become unstable in the latest release and losing data is a regular occurrence.\u00a0 OpenFiler remains very popular in the mindshare of this industry segment but should be avoided completely.<\/p>\n If you do not need to have your storage operating system appliancized then you are left with more and better choices, but a far more complex decision tree.\u00a0\u00a0\u00a0 Unlike the appliance OS market which is filled with potholes (NexentaStor has surprise costs, OpenFiler appears to support iSCSI but causes data loss, features get removed from new versions) all four operating systems mentioned here are extremely robust and feature rich.\u00a0 Three of them have OEM vendor support which can be a major deciding factor and all have great third party support options far broader than what is available for the appliance market.<\/p>\n The first decision is whether or not Windows only features, notably NTFS ACLs, are needed.\u00a0 It is common for new NAS users to be surprised when the SMB protocol does not provide all of the granular filesystem control that they are used to in Windows.\u00a0 This is because those controls are actually handled by the filesystem, not the network protocol, and Windows alone provides these via NTFS.\u00a0 So if that granular Windows file control is needed, Windows is your only option.<\/p>\n The other three entrants, Linux, Solaris and FreeBSD, all share basic capabilities with the notable exception of clustering.\u00a0 All have good software RAID, all have powerful and robust filesystems, all have powerful logical volume management and all provide a variety of NAS and SAN connection options.\u00a0 Many versions of Linux and FreeBSD are available completely freely.\u00a0 Solaris, while free for testing, is not available for free for production use.<\/p>\n The biggest differentiator between these three OS options is clustering.\u00a0 Linux has had DRBD for a long time now and this is a robust filesystem clustering technology.\u00a0 FreeBSD has recently (as of 9.0) added HAST to serve the same purpose.\u00a0 So, in theory, FreeBSD has the same clustering options as Linux but this is much newer and much less well known.\u00a0 Solaris lacks filesystem clustering in the base OS and requires commercial add-ons to handle this at this time.<\/p>\n Solaris and FreeBSD share the powerful and battle tested ZFS filesystem.\u00a0 ZFS is extremely powerful and flexible and has long been the key selling point of these platforms.\u00a0 Linux’ support for filesystems is more convoluted.\u00a0 Nearly any Linux distribution (we care principally about RHEL\/CentOS, Oracle Unbreakable Linux, Suse\/OpenSuse and Ubuntu here) supports EXT4 which is powerful and fast but lacks some of the really nice ZFS features.\u00a0 However, Linux is rapidly adopting BtrFS which is very competitive with ZFS but is nascent and currently only available in Suse and Oracle Linux distros.\u00a0 We expect to see it from the others soon for production use but at this time it is still experimental.<\/p>\n Outside of clustering, likely the choice of OS of these three will come down primarily to experience and comfort.\u00a0 Solaris is generally known for providing the best throughput and FreeBSD the worst.\u00a0 But all three are quite close.\u00a0 Once BtrFS is widely available and stable on Linux, Linux will likely become the de facto choice as it has been in the past.<\/p>\n Without external influence, my recommendation for storage platform are FreeBSD and then Linux with Solaris eliminated on the basis that rarely is anyone looking for commercial support and so it is ruled out automatically.\u00a0 This is based almost entirely on the availability of Copy-on-Write filesystems and assuming no clustering which is not common.\u00a0 If clustering is needed then Linux first then FreeBSD and Solaris is ruled out, again.<\/p>\n Linux and FreeBSD are rapidly approaching each other in functionality.\u00a0 As BtrFS matures on Linux and HAST matures on FreeBSD they seem to be meeting in the middle with the choice being little more than a toss up.<\/p>\n There is no single, simple answer.\u00a0 Choosing a storage OS is all about balancing myriad factors from performance, resources, features, support, stability, etc.\u00a0 There are a few factors that can be used to rule out many contenders and knowing these hard delimiters is key.\u00a0 Knowing exactly how you plan to use the system and what factors are important to you are important in weeding through the available options.<\/p>\n Even once you pick a platform there are many decisions to make.\u00a0 Some platforms include multiple file systems.\u00a0 There is SAN and NAS.\u00a0 There are multiple SAN and NAS protocols.\u00a0 There is network bonding (or teaming, the Windows world.)\u00a0 There is Multipathing.\u00a0 There are snapshots, volumes, RAID.\u00a0 The list goes on and on.<\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" It is becoming increasingly common to forgo traditional, proprietary storage devices, both NAS and SAN, and instead using off the shelf hardware and installing a storage operating system on it for, what many call, “do it yourself” storage servers.\u00a0 This, of course, is a misnomer since no one calls a normal file server “do it … Continue reading Choosing an Open Storage Operating System<\/span>