This guide is designed to give you the quickest way to get a "net new" RAID setup on Ubuntu with a minimum of 3 or more (ideally) matching drives.
At my house, I run a small media server for the local network. From backups of my various devices to running small test projects or archiving my retro game ROM collections, we use a LOT of storage. The server in reference is a purpose-built computer with a smattering of various sized WD Red's. We will be adding NEW drives to the computer to build the POOL/RAID.
Step 1 - Install Drives
In my case, I had 4 fresh 12TB WD Red's, so I shut down the box, popped the case and installed the new drives. I took special effort to make sure that the new drives were clustered at the end of the SATA controller. My motherboard has 2 M.2 slots and 6 SATA ports. Boot is an M.2 and I had 3 other SATA (0 index) drives, so the new ones were SATA 3, 4, & 5.
Once the drives are installed, power the box back on, log in and use the lsblk command to display the drives and their assigned drive letter.
❯ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 3.6T 0 disk
└─sda1 8:1 0 3.6T 0 part /media/disk1
sdb 8:16 0 3.6T 0 disk
└─sdb1 8:17 0 3.6T 0 part /media/disk2
sdc 8:32 0 10.9T 0 disk
sdd 8:48 0 10.9T 0 disk
sde 8:64 0 10.9T 0 disk
sdf 8:80 0 10.9T 0 disk
nvme0n1 259:0 0 476.9G 0 disk
├─nvme0n1p1 259:1 0 1G 0 part /boot/efi
├─nvme0n1p2 259:2 0 2G 0 part /boot
└─nvme0n1p3 259:3 0 473.9G 0 part
└─ubuntu--vg-ubuntu--lv 252:0 0 100G 0 lvm /
Notice the 3 matching entries under sdd, sde, sdf. Don't do anything else to the drives - no partitioning, no formatting, nothing... Just crack the package and pop it in.
Step 2 - Create the ZFS Pool
Now the fun begins. We are now going to create a ZFS pool that in the example is names MediaPool, but you can change that to whatever you like.
❯ sudo zpool create -f MediaPool raidz1 /dev/sdd /dev/sde /dev/sdfAnd then check the status to make sure everything looks right.
❯ sudo zpool status
pool: MediaPool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
MediaPool ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
sdf ONLINE 0 0 0
errors: No known data errors
The zpool create command will also mount your MediaPool to the /MediaPool address taken from the name of the pool itself.
❯ df -h
Filesystem Size Used Avail Use% Mounted on
...
MediaPool 32T 0T 32T 0% /MediaPool
Step 3 - Enable ZFS Compression
LZ4 is the default compression algorithm for ZFS because it delivers a great balance of speed, efficiency, and storage savings with negligible overhead. This makes it a solid choice for ZFS RAID pools, particularly in environments where performance and responsiveness are key concerns.
❯ sudo zfs get compression MediaPool
NAME PROPERTY VALUE SOURCE
MediaPool compression on default
❯ sudo zfs set compression=on MediaPool❯ sudo zfs get compression MediaPool
NAME PROPERTY VALUE SOURCE
MediaPool compression on localStep 4 - Increase Block Size
Since I generally work with large files that do extended, sequential reads and writes, so adjusting the block size to 1M is suggested.
❯ sudo zfs get recordsize MediaPool
NAME PROPERTY VALUE SOURCE
MediaPool recordsize 128K default❯ sudo zfs set recordsize=1M MediaPool❯ sudo zfs get recordsize MediaPool
NAME PROPERTY VALUE SOURCE
MediaPool recordsize 1M localIntermission - Start Loading Data
Now that we have a striped pool, it's time to start putting some files on it. I am starting with about 8TB of files, so this is an overnight operation.
Write Caching and SLOG
Write caching is also know as L2ARC on ZFS. L2ARC is a secondary read cache in ZFS that extends the primary ARC (Adaptive Replacement Cache), which resides in RAM, by using a fast storage device such as an SSD or NVMe drive.
In ZFS, the SLOG (Separate Intent Log) is an optional dedicated device used to store the ZFS Intent Log (ZIL). The ZIL temporarily holds synchronous write operations before they are committed to the main storage pool.
For both of these cases, I used a 500GB NVME drive split evenly for that purpose. Here's how I set it up.
- Install the hardware, boot the machine and identify the device ID, such as
/dev/nvme1n1. - Use
partedto split the disk in half:
❯ parted /dev/nvme1n1
(parted) mklabel gpt
(parted) mkpart primary 0% 50%
(parted) mkpart primary 50% 100%
(parted) quit(parted) print in between those commands to verify that things are going as planned if you need.After that, you should be left with nvme1n1p1 and nvme1n1p2 or similar.
- Assign those partitions to their respective operations:
❯ sudo zpool add -f MediaPool cache nvme1n1p1❯ sudo zpool add -f MediaPool log nvme1n1p2Assuming those didn't throw errors, you can verify that these new functions are active by running the following command:
❯ sudo zpool statusYou should see an output similar to below:
pool: MediaPool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
MediaPool ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
logs
nvme1n1p2 ONLINE 0 0 0
cache
nvme1n1p1 ONLINE 0 0 0
errors: No known data errors
And with that, you have a L2ARC caching and SLOG.
Monitoring and Alerting
Now that you have this fancy ZFS Raid and storage pool, you probably need to know when something goes wrong, and knowing about it quickly can mean the difference between being able to recover everything easily or losing the data that you have stored.
I have done this with a simple Python script I wrote that is attached to a CRON and ran periodically. I am generally a regular on Discord, and I use Email for ongoing reporting, so my scripts use both mechanisms to get your attention. Click the link below to get started:
cmivxx
Scrubbing the Pool
Zpool scrubbing is a maintenance process in ZFS that scans all data in a storage pool to verify its integrity. It checks each block against its checksum to detect any silent data corruption. If an error is found and the pool has redundancy (like mirrors or RAID-Z), ZFS will automatically repair the corrupted data. Regular scrubbing helps ensure the health and reliability of stored data over time.
I have my scrubbing set to run on the 1st of every month. The contab config below represents that.
Start by opening your crontab for editing.
❯ sudo crontab -eThe add the following lines at the bottom:
# Add the following line
0 0 1 * * /sbin/zpool scrub MediaPoolcrontab's and for help changing the frequency, please visit: The Crontab GuruComing Soon
- Snapshot Information
Bonus Tips
zpool import and follow the on screen instructions.
