SnapRAID and MergerFS on OpenMediaVault

I am running SnapRAID and MergerFS. I want my data to be protected by SnapRAID and simplify the storage management across the SnapRAID disks array via MergeFS. OpenMediaVault (OMV) made this easier by having these two available as plugins. These can be installed via OMVExtras Plugins.

After installing the OMVExtras, search for these two plugins in the System > Plugins

• openmediavault-snapraid
• openmediavault-unionfilesystem

Some of the bullets below are not required, but this is what I have done to my disks. I am starting this post from scratch with an assumption of newly installed disks. If some of these do not apply to you, that is fine as long as you have the last three bullets.

• Completed badblock and SMART test scans
• Wiped the drives
• Encrypted the drives
• Created a filesystem
• Mounted the disks
• SnapRAID and UnionFS (mergerFS) plugins already installed

SnapRAID

This is from SnapRAID website:

SnapRAID is a backup program for disk arrays. It stores parity information of your data and it recovers from up to six disk failures.

https://www.snapraid.it/

SnapRAID does more than what I have quoted. I would recommend for anyone viewing this post to visit SnapRAID’s website especially the comparison between Unraid, SnapRAID, ZFS, etc.

We are going to start setting up the SnapRAID’s parity drive. Navigate to :

• Services > SnapRAID click on the Drives (tab)
  1. Click Add
  2. Select the drive that is going to be a parity drive under the Drive drop-down menu
  1. Give the drive a name
  2. Enable Parity
  3. Click on Save
  4. Click Apply
  5. Then confirm by clicking Yes
• Repeat steps 1 through 5 if you have multiple parity drives

Once the parity drive has been set-up, we would need to configure the data/content drive(s).

• Services > SnapRAID click on the Drives (tab)
  1. Click Add
  2. Select the drive that is going to be the data drive under the Drive drop-down menu
  3. Give the drive a name
  4. Enable Content
  5. Enable Data
  6. Click on Save
  7. Click Apply
  8. Then confirm by click Yes
• Repeat step 1 through 8 for the rest of the data/content drives

SnapRAID Scrub

The purpose of scrubbing is to find the error between the data on the data drive(s) and the data on the parity drive(s). My use case is to scrub to 100% in a month. My scrub is going to run weekly.

• Navigate to Services > SnapRAID > Settings
  1. Under the Settings section
  2. Set the Scrub percentage from 12 to 25. This is the value for manual scrub
  3. Under the Diff Scripts Settings
  4. Change the Scrub Percentage from 100 to 25
  5. Enable the Send Mail if you have the notification configured # optional
  6. Click Save
  7. Click Apply
  8. Enable the Scheduled diff. This will automatically create an entry in Scheduled Jobs
  9. Click Apply

SnapRAID Rules

There are plenty of data types that you would not want to sync to the parity. Such data types are temporary files, media metadata, Docker containers configuration, virtual machines or anything that is not too important and that constantly changing. If you want to sync this type of data then you can skip this section.

I have my Docker containers’ /config and virtual machines on an SSD. Since this SSD is part of my SnapRAID array, I have to make some exclusion for these folders /sharedfolders/appdata/

• Navigate to Services > SnapRAID > Settings
  1. Click on Rules tab
  2. Click on Add
  3. Select the volume from the Volume drop-down menu
  4. Select the directory of what you want to exclude from Rule field
  5. Click Save
  6. Click Apply
  7. Click Yes to confirm
• Repeat steps 1 through 7 for each exclusion rule

SnapRAID Scheduled Jobs

Before setting up the cronjobs, run the snapraid sync or scrub commands or run it via the web UI manually just to get an idea of how long it would take. Once you have an idea, you can specify the proper schedule to automate the process.

# Run this command for the first time
snapraid sync

# Run this command after the sync is completed
snapraid scrub

# Run this command for status
snapraid status

To run the above commands using the web UI:

• To manually sync, navigate to Services > SnapRAID > Drives
  1. Select the sync option from the Tools tab
  2. Click Start to start the sync

• To manually scrub, navigate to Services > SnapRAID > Drives
  1. Select the scrub option from the Tools tab
  2. Click Start to start the scrub

• To manually check the status, navigate to Services > SnapRAID > Drives
  1. Select the status option from the Info tab
  2. Click Start to start

If everything goes well, you should get a similar output as shown in Figure 5.

I have posted an OMV post about Scheduling jobs. Instead of writing the same information, please check that post. The basic function here is the sync, touch, and scrub.

sync – Updates the parity information. All the modified files in the disk array are read, and the corresponding parity data is updated.

touch – Sets arbitrarely the sub-second timestamp of all the files that have it at zero. This improves the SnapRAID capability to recognize moved and copied files as it makes the timestamp almost unique, removing possible duplicates.

scrub – This command verifies the data in your array comparing it with the hash computed in the “sync” command.

https://www.snapraid.it/manual

Creating MergerFS Pool

MergerFS basically would spread the data across the data disks that are members of the MergerFS pool. For the person perspective, who uses the system, there is only one folder. The files are spread across all the disks. How is MergerFS spreading the files is depending on the policy that is applied to the pool.

Here is the description from the GitHub page:

mergerfs is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to mhddfs, unionfs, and aufs.

https://github.com/trapexit/mergerfs

To create a mergerFS pool, navigate to

• Storage > Union Filesystems
  1. Click Add
  2. Give the pool a name in the Name field
  3. In the Branches box, select all the SnapRAID data drive(s) or the datas that you would like to be a part of this pool and make sure that the parity drive(s) is not selected
  4. Under Create policy‘s drop-down menu, select the Most free space
  5. *The Minimum free space field, I put in 100G.
  6. The Option field leave it at default
  7. Click Save
  8. Click Apply
  9. Click Yes to confirm

*In step 5, the value will depends on the size of your disk. I am not sure how relevant this post, but it is worth reading it. If you notice some performance degration, increasing this value might help, but I cannot say for sure.

There are reports that the Docker container does not like to work well when its /config spread across the MergerFS pool. It is a good idea to keep the Docker containers’ /config to a single location or disk.

Navigate to Storage > Filesystems to verify the pool exist and the total storage amount as shown in Figure 8.

Directories

To use the data drives as the storage and simplify access to the user data, we would need to create directories on the MergerFS pool that was created earlier.

Every directory that is created via the web UI as shown below will be located in /srv/hhhhhhhh-hhhh-hhhh-hhhhhhhhhhhh where h is a hex value.

In addition, every disk that is mounted to OpenMediaVault is also located in /srv. The default permission is root:users; however, if creating a directory via CLI using the root account, only the root can use this directory. To make directory available to the web UI or any application, you must change the group of the directory. This is outside of the scope of this post.

• Access Rights Management
  • Shared Folders
    • Shared Folders (tab)
      1. Give the folder a name
      2. Select the MergerFS pool that was created earlier from the Device drop-down menu
      3. Click Save
      4. Click Apply
      5. Click Yes to confirm
• Repeat steps 1 through 5 for each directory

This is pretty much it. We configured our OpenMediaVault with SnapRAID and configured MergerFS.

I hope you find this helpful. Cheers!