Install Ubuntu onto ZFS

For installing Arch Linux into a ZFS root, refer to Install Arch into ZFS Root

Ubuntu, Linux Mint and Zorin OS can be installed to and booted from a ZFS data set. This article details the manual process of installing Ubuntu to a new ZFS pool created from a blank or formatted hard disk drive. ZFS supports transparent compression, data integrity and snapshots, of which are welcomed features when setting up network attached storage for servers which depend on redundancy and continuity. ZFS is resource intensive, however. The recommended requirements for ZFS include a minimum of 8 Gigabytes of system memory, ideally with error correction support and Ubuntu 16 or later.

The process of installing Ubuntu into a ZFS data pool is a multi-part process:

1. Prepare Live Environment by adding ZFS support
2. Provision ZFS volume by formatting an existing hard disk.
3. Optionally encrypt the partition housing the ZFS pool for additional confidentiality
4. Create new ZFS pool and filesystems for housing new installation
5. Install Ubuntu into a (non-booteable) ZFS volume
6. Copy the installed Ubuntu instance onto a mounted ZFS root
7. Configure Ubuntu
8. Reconfigure the boot loader to boot off the new ZFS pool

Advantages

ZFS provides the following advantages:

• Improved performance due to on-the-fly compression of data
• Reduced disk space usage due to compression
• Easier to spin up a clone of the installed instance

Moreover, since ZFS supports a RAID setup, the data set where Ubuntu is installed is transparent to the actual disk where Ubuntu is installed. This allows for redundancy, for instance, a RAID mirroring setup would allow the failure of a disk without impacting the running of the system.

1. Setup Environment and Install ZFS

Setup Live Environment

Setup the live environment by following this guide

Install ZFS

The package, zfs-initramfs is necessary to create a modified initramfs image that can boot from the ZFS data set. Please type these commands to install ZFS utilities needed for accessing ZFS volumes on a hard disk:

sudo apt-get install -y zfs zfs-initramfs

2. Format Hard Disks

Prior to provisioning a ZFS data pool, it is first necessary to appropriately create and format the partitions. The ZFS data pool will be created on top of a so-called root data partition. Partitioning one or more local storage mediums is necessary to prepare a ZFS data pool for this kind of application. Even though, an entire disk can be used for ZFS, this setup will not be bootable. Partitioning a disk offers a number of advantages:-

• The ability to specify an UFI partition for booting UEFI-enabled systems
• The ability to use LUKS encryption to protect the ZFS installation.
• The ability to have a dual booted setup where partition X is ZFS and partition Y is another system such as Windows

In theory, partitions from multiple physical disks can be added to a single ZFS pool to provide enhanced performance, increased space or redundancy. However, the long term implications of this setup remain to be tested.

Step 1: Install SGDisk Utility

The Ubuntu Live CD comes with a graphical partitioning tool called GParted, which may be used to create appropriate partitions. However, to reduce problems, the sgdisk utility should be used.

Type:

sudo apt-get install -y gdisk 

Please Note: GParted cannot be used on non-graphical systems such as Ubuntu Server (which lack a desktop environment).

Step 2: Wipe Partition Table on physical medium

Ideally, the existing partition table (and any data currently residing on the disk) should be destroyed before attempting to create a valid partition setup. Doing so will render data inaccessible with no guaranteed chance of successful recovery. You should back any sensitive data before proceeding. Otherwise, it will be deleted and may become corrupted. If the disk is new and has never been formatted or used before, then you can skip this step, since no partition table will be present.

Type the command below to wipe the existing partition table.

sudo sgdisk --zap-all /dev/disk/by-id/<physical-disk> 

Step 3A. Create Partition for UEFI Booting

Modern computer systems, including many tablet-based computers such as the Microsoft Surface support Unified Extensible Firmware Interface designed to standardize disk booting. This is option is recommended if your computer supports it.

To create a UEFI-enabled boot partition, type:

sudo sgdisk -n3:1M:+512M -t3:EF00 /dev/disk/by-id/<physical-disk>

Step 3B. Create Partition for Legacy Booting

If your computer does not have support for UEFI or if you booted the Ubuntu live CD in legacy mode, then you should provision a boot partition that supports booting in legacy mode.

sudo sgdisk -a1 -n2:34:2047 -t2:EF02 /dev/disk/<physical-disk>

3. Setup Logical Partitions

In this step, a data partition will be created that will house the ZFS pool. There are are two different types of data partitions that can be created:

1. Encrypted ZFS Pool: A ZFS pool is created on a LUKS-encrypted data partition
2. Unencrypted ZFS Pool: A ZFS pool is created on an unencrypted partition

Method 1: Encrypted ZFS Pool using LUKS

1): The first step is to create a new, LUKS partition, which will house the ZFS pool information. To do this, type the following commands:

sudo sgdisk -n4:0:+512M -t4:8300 /dev/disk/by-id/<physical-disk>
sudo sgdisk     -n1:0:0      -t1:8300 /dev/disk/by-id/scsi-SATA_disk1

2): Format the new partition using the LUKS filesystem:

sudo cryptsetup luksFormat -c aes-xts-plain64 -s 256 -h sha256 \
      /dev/disk/by-id/<physical-disk>-part1

Note: You may be prompted to enter a passphrase. This is a personal "password" designed to protect the volume. You will also have to re-enter it every time the machine boots. For more information and alternate solutions, please see Full Disk Encryption.

3): Open the newly formatted LUKS volume and map to /dev/mapper/luksVol:

sudo cryptsetup luksOpen /dev/disk/by-id/<physical-disk>-part1 luksVol

4): Enter the same passphrase entered in step 2.

4. Setup up ZFS Pool

The ZFS pool was created using our data partition as the so called source disk. The pool will be stored within the data partition, which allows other partitions to simultaneously reside on the disk. If encryption is being used, the data partition is the LUKS container set up in the previous section. GitHub recommends using the ashift=12 flag to support 4K sector sized disks.

Step 1. Find Appropriate Block Device

The command below needs to know to which block device (disk, partition or virtual device) ZFS is being set up on. This will vary, according to your system and needs.

• If using encryption, you will want to find the mapped LUKS device from phase 3, step 3 e.g. /dev/disk/by-id/<physical-disk>-part1.
• If not using encryption, you will want to find the partition that is mapped to a device alias. Aliases can be listed using:

sudo ls -lhsa /dev/disk/by-id

2. Provision ZFS Pool

Setup the ZFS pool by typing:

sudo zpool create -o ashift=12 -O atime=off -O canmount=off -O compression=lz4 -O normalization=formD -O mountpoint=/ -R /mnt <pool-name> <partition> 

3. Create Data Sets

A new root data set needs to be created inside the new ZFS pool to house the Ubuntu installation instance. You can vary the following command as required to create a valid ZFS data set.

sudo zfs create -o canmount=off -o mountpoint=none <pool-name>/ROOT
sudo zfs create -o canmount=noauto -o mountpoint=/ <pool-name>/ROOT/ubuntu
sudo zfs mount <pool-name>/ROOT/ubuntu

5. Create Additional Data Sets (Optional)

You may want to additionally create more data sets, for instance, for storing user data in a separate data set from your main system. This is important when using snapshots because reverting a broken system will not accidentally erase user data.

For example, to create a data set for storing home directories, you could do:

sudo zfs create -o setuid=off <pool-name>/home

You may want to create a cache directory that does not have snapshots. You can disable snapshots on a created ZFS data set with this command.

sudo zfs create -o com.sun:auto-snapshot=false <pool-name>/cache

You may use these commands to create data sets for your var and log directories.

sudo zfs create -o canmount=off -o setuid=off -o exec=off <pool-name>/var
sudo zfs create <pool-name>/var/log
sudo zfs create <pool-name>/var/spool
sudo zfs create -o com.sun:auto-snapshot=false -o exec=on  <pool-name>/var/tmp

6. Install Base System

The current ZFS data sets are empty and do not have any files on them. The Ubuntu installer does not support directly installing to a ZFS data set. There are two approaches you can take to install Ubuntu:

1. Install Ubuntu to an alternate location: Ubuntu can be installed to an alternate location, such as a supported unbootable ZFS volume and later copied over to the newly created ZFS data set.
2. Install a minimal, base system of Ubuntu directly into ZFS: On non-graphical environments, Ubuntu can be installed directly into the ZFS data set but only a minimal system will be installed.

Typically, the second method can be used when a graphical environment is not available, for instance, when installing Ubuntu server. The first method is slower than the second method but is easier to perform for novices since Ubuntu will configured and nearly ready to go once copied over.

Method 1: Graphical Install

If the Ubuntu live environment has graphics available, the Ubuntu graphical installer can be used to install Ubuntu to an alternate location, and then moved over to the ZFS data set.

1) A small 20GB ZFS volume (known as a ZVOL) is created within the ZFS pool i.e. <pool-name>. You can use this command to create a ZVOL with 20GB of provisioned disk space. You may adjust the space allocation as desired:

sudo zfs create -V 20G <pool-name>/ubuntu-installation

You might be curious to know, whether or not, you can simply boot Ubuntu by installing it here and rebooting. The problem is that the installer fails to install the boot loader. This is not a problem when moving the installation to a ZFS dataset, but the ZVOL itself is not bootable.

The graphical installer will detect zvol as though it is another hard disk drive. As such, you may use this installer to select that ZVOL device and install to it. This process will copy the files and configure Ubuntu. Since these files will be later copied, it it unnecessary to create additional partitions and mount points. Ubuntu can be installed to the root of the device.

The wizard will allow you to correctly complete timezone and username information without needing to manually edit text files. The error about the failed boot loader can be safely ignored, since the boot loader will have to be manually installed.

2) The ZVOL should be mounted, so files stored on it can be copied to the new ZFS data set. The /mnt directory is likely already in use due to having previously mounted the ZFS root data set. Therefore, the zvol should be mounted elsewhere

sudo mount /dev/zd0p1 /media

3) The rsync utility can be used to copy the Ubuntu installation to the ZFS data set. The command below can be used. You may need to change the directories to correspond with the correct mount points.

sudo rsync -avPX /media/. /mnt 

4) Setup aliases so that the new Ubuntu instance will be updated by entering these commands:

sudo su
for d in proc sys dev; do mount --bind /$d /mnt/$d; done
chroot /mnt
echo "nameserver 8.8.8.8" | tee -a /etc/resolv.conf

The chroot environment now has Internet access, allowing packages to be installed directly to the newly installed instance of Ubuntu. The commands above automatically switched into the new environment.

Method 2: Command Line Install

1) When installing from the command line, a minimal base install of Ubuntu is carried out using debootstrap. Install this tool from the terminal.

sudo apt-get install -y debootstrap

Packages can then be installed as desired on top of the minimal environment, including a desktop environment. Debootstrap does not install a desktop environment and leaves configuration to the user.

2) You can use these commands to install Ubuntu into the /mnt directory.

sudo chmod 1777 /mnt/var/tmp
sudo debootstrap xenial /mnt
sudo zfs set devices=off rpool

This newly installed system is currently unconfigured, so it is advisable to configure it to ensure smooth running of the system.

3) Configuration changes need to be made to the config file in /mnt/etc/hosts. The IP address 127.0.1.1 should be bound to the local host name, or to the DNS name.

4) Configure the network interface needs to to use a valid IP address, or DHCP.

5) Setup aliases so that the new Ubuntu instance will be updated by entering these commands:

sudo su
for d in proc sys dev; do mount --bind /$d /mnt/$d; done
chroot /mnt
echo "nameserver 8.8.8.8" | tee -a /etc/resolv.conf

The chroot environment now has Internet access, allowing packages to be installed directly to the newly installed instance of Ubuntu. The commands above automatically switched into the new environment.

6) Once the files have been copied over, the chroot command is used to log in to our new Ubuntu instance. This is necessary to configure the boot loader and install ZFS tools within the new Ubuntu instance. If Ubuntu was booted in its current state, it would not boot.

7) Add Source Repositories

You need to add appropriate repositories to your /etc/sources.list file, so that software can be located and installed by the package manager. An Ubuntu configuration file generator is located here. You can copy and paste the configuration into your /etc/sources.list file.

After adding new packages, you should run apt-get update so the new packages can be used.

ln -s /proc/self/mounts /etc/mtab
apt update

8) Set System Language

You need to set the appropriate language for your system. GitHub recommends having United States English always available, even if you live outside the United States.

locale-gen en_US.UTF-8
echo 'LANG="en_US.UTF-8"' > /etc/default/locale
dpkg-reconfigure tzdata

9) Install Minimal Packages

You should install the minimal set of packages for your system

apt install --yes ubuntu-minimal
apt install -y --no-install-recommends linux-image-generic

10) Setup Groups

System groups need to be configured on the newly installed system,

addgroup --system lpadmin
addgroup --system sambashare

11) Set Root Password

A root password should be set for security purposes. It is advisable to use a password different from your username, since a compromised user password could provide an attacker with root access.

passwd

7. Install Boot Images

The ZFS and kernel boot images in the chroot environment need to be updated to support booting from ZFS. This requires the installation of the ZFS intramfs package.

apt install -y zfs-initramfs

8. Install and Reconfigure Boot Loader

The boot loader (i.e. GRUB) is required to be boot the new installation. The boot files do not reside inside the ZFS dataset, but on a small boot partition. The BIOS cannot see ZFS directly and so a separate partition is needed to kick start the boot process. The boot files in Ubuntu are located in /boot/. This directory, therefore, needs to be mapped to the correct boot partition, so that the files are physically created and updated on the correct partition.

There are two options for installing the boot loader depending on your needs:

• Legacy Booting - The old MBR setup that lacks EFI support
• UEFI - The more modern approach to booting. The system boots by looking for a small EFI partition. The MBR is not used.

Method 1: Legacy Booting

1) Legacy booting is simply a matter of installing GRUB using the following commands:

sudo apt install -y grub-pc
update-initramfs -c -k all
grub-install <main disk>

Please Note: Replace <main disk> with the path to the main disk alias e.g. /dev/sda or /dev/disk/by-id/scsi-SATA_disk1

Method 2: UEFI

UEFI booting requires a small partition formatted with the FAT32 filesystem and the appropriate boot files added to it. This can be achieved by mounting the physical boot partition to /boot, so that Ubuntu appropriately adds the files.

1) The /boot/ directory does not contain a required subdirectory, efi by default. Therefore, it is necessary to first create this directory by typing these commands:

sudo mkdir /boot/efi

2) Next, changes should be made to the /etc/fstab configuration file, so the boot partition is automatically mounted by the system at path /boot. This is the canonical path used by Ubuntu to look for boot files. It is important to do this to ensure that the correct boot files are updated. Replace <boot-partition> with the correct boot partition alias.

echo PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/<boot-partition>) /boot/efi vfat defaults 0 1 >> /etc/fstab
sudo mount /boot/efi
sudo apt install -y grub-efi-amd64

3) To install UEFI-enabled GRUB to the EFI partition, you should run the following commands:

update-initramfs -c -k all
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ubuntu --recheck --no-floppy

9. Reboot System

1) Logout of the chroot environment by typing:

exit

2) Type cd to change to the environment's home directory:

cd

3) Unmount the data pool:

umount /mnt

Note: Umount is not a spelling mistake. It is the command expected by the terminal. 4) Reboot the system by typing:

sudo reboot

10. Boot into your new system

When prompted at the menu, select Ubuntu and boot into your new system. If there are errors or boot issues, troubleshoot the problem.