6
« on: May 21, 2017, 04:47:38 am »
Hi all,
I've recently added support for Oracle's ZFS filesystem to LinuxMCE. The support is integrated in the same place as the RAID setup in the web admin, and is selected as another type of RAID template. ZFS offers a wide range of benefits over the traditional md software RAID that has been present with LinuxMCE since the beginning, namely:
* The underlying block storage system and filesystem know about each other, and keep track of what's actually stored. This means, no lengthy array reconstruction times!
* The filesystem natively supports compression to increase disk space.
* The filesystem supports snapshotting for backing up, restoring, or transferring pool contents.
* For RAID-Z levels, the underlying parity information is kept with each block, rather than with each stripe, avoiding the RAID pairty hole problem.
* In addition to modes that correspond to RAID-0 (pooling), RAID-1 (mirroring), and RAID-5 (RAID-Z1), there are modes to provide a double parity RAID (akin to RAID-6), and triple parity RAID (akin to what would be in essence, RAID-7)
* The ability to mix and match all of these modes, together, in various configurations to create storage pools that best suit performance and reliability requirements.
It's not all roses, there are some drawbacks, most notably fragmentation due to the use of copy-on-write for everything, but ZFS mitigates this using very clever algorithms, so in practice with media storage, this should be of minor consequence. I will provide a way to defragment the resulting pools via the web admin, when I can.
As before with traditional RAID, you select your disks, and select Create Array to create the resulting storage pools. Pluto Storage Devices will immediately appropriate the storage, and mark it for use (if use automatically is selected).
What versions?
-----------------
Currently, Trusty (14.04), and Xenial (16.04) and later have this functionality. I had done the necessary work for Precise (12.04) but for some reason, it's not installing correctly (due to a sysv-rc version mismatch, which is VERY strange!), so the linked dependency to the zfs tools from the lmce-core package was removed until I can determine what the problem is.
The following ZFS Pool types are available:
* ZFS Pool - Akin to RAID-0, but with blocks dynamically striped across the device for better overall performance.
* ZFS Mirror - RAID-1 like functionality, again with dynamic striping
* ZFS Raid Z1 - Single Parity RAID (RAID-5 like)
* ZFS Raid Z2 - Double Parity RAID (RAID-6 like)
* ZFS Raid Z3 - Triple Parity RAID (RAID-7 like)
What's done:
* Array creation (with or without hot spares)
* Deletion
* integration into pluto storage devices
What's to be done:
* Array growing (there will be no deletion of members from pools, as this isn't possible)
* Hybrid pools (mixtures of the above storage pools, needs a lot of new UI in the web admin)
* snapshotting (need to think of a proper way to present this to the user)
* Better status reporting (hooking further into zed)
Getting it:
New installs of 14.04 and later include the necessary ZFS support. If you are already running 14.04, then an apt-get upgrade should pull in the new ZFS packages, as well as the new device templates for the ZFS storage pools. After this point, you should be able to create new storage pools.
A note:
You may want to wait until posde pushes my latest updates (as of 2017-05-21), before trying, as I did a LOT of bugfixing over the last day. I will tell in this thread when those changes have been pushed into the repo.
Enjoy,
-Thom
Show Posts
