Chapter 4. System setup

Table of Contents

4.1. Read-only file system
4.2. /var
4.3. /etc

4.1. Read-only file system

transactional-update is typically used on a read-only root file system, even though it also supports regular read-write systems.

4.2. /var

On a system with snapshot support /var should not be part of the root file system, otherwise doing a rollback to a previous state would also roll back the /var contents. On a read-only system this directory has to be writable in any case, variable data is stored inside.

Due to the volatile nature of /var the directory will not be mounted into the new snapshot during the transactional-update run, as this would break atomicity: The currently running system depends on the old state of the data (imagine a database migration was triggered by a package). Any modifications to /var therefore have to be in the new system, i.e. modifying the contents of /var as part of the packaging scripts is not allowed.

The only exception to this rule are directories: Those will be recreated during the first boot into the updated system by the create-dirs-from-rpmdb.service helper service. For all other cases please use one of the options described in Packaging for transactional-updates and Migration / Upgrade in the Packaging guidelines. If a package is breaking this rule by installing files into a directory which is not part of the root file system, then a warning message indicating the affected file is printed at the end of the transactional-update run.

4.3. /etc

transactional-update also supports write operations to /etc on an otherwise read-only file system. To do so an overlayfs layer is put on top of the system's /etc directory. All modified configuration files will end up in the current snapshot's overlay in /var/lib/overlay/<snapshotnum>/etc, with each snapshot having one associated overlay directory.

The mounted /etc directory will typically consist of three (or more) layers. In /etc/fstab this may look similar as the following entry:

overlay  /etc  overlay  defaults,upperdir=/sysroot/var/lib/overlay/82/etc,lowerdir=/sysroot/var/lib/overlay/81/etc:/sysroot/var/lib/overlay/76/etc:/sysroot/etc,workdir=/sysroot/var/lib/overlay/work-etc,x-systemd.requires-mounts-for=/var,x-systemd.requires-mounts-for=/var/lib/overlay,x-systemd.requires-mounts-for=/sysroot/var,x-systemd.requires-mounts-for=/sysroot/var/lib/overlay,x-initrd.mount  0  0

The important options are upperdir and lowerdir. upperdir references the write layer, i.e. the directory the system is currently writing to. If a file in /etc is opened for read access, it will first try to open the file from this directory. If the file is not present in this layer, then overlayfs will search in the lowerdirs from left to right until it finds the file, i.e. the layers are transparent. The last layer is the reference layer and always has to have a complete copy of /etc, all the layers above just contain changes to that base layer.

Example 4.1. /etc layers

upperdir  (/var/lib/overlay/x/etc)   |       | file2 | file3 | file4 |       |
lowerdir1 (/var/lib/overlay/x-1/etc) |       |       | file3 |       | file5 | file6
lowerdir2 (/etc of snapshot x-2)     | file1 | file2 | file3 |       |       | file6
	

Let's have another look from the perspective while an update is running. x is the id of the new snapshot where the update is currently being performed. x-1 is the id of the currently running snapshot, and x-2 is the id of the snapshot before the currently running one.

file1 is an old file that hasn't been changed in the last two snapshots. Therefore it is only present in the base / lowest overlay. As the file cannot be found in the first two layers, the version from the base layer will be used.

file2 has been changed during the update. The currently running system still sees the old version of the file from the base layer, the updated system will see the new version of the file.

file3 has been changed during the update, but there is also a modification in the currently running system. See the Warning below for details.

file4 was only added in the new snapshot and will only be available there. The currently running system doesn't know anything about it.

file5 was added in the current snapshot. It will also be visible in the new snapshot.

file6 was modified in the current snapshot. It will also be visible in the new snapshot. Basically the same case as file5.


To reduce the number of overlays the contents of the overlays are merged into the base layer if a new snapshot is created. Caveat: the state of /etc that is synced into the new snapshot is not the one from the currently running system, but the previous one: the files in the current system could still be changed before the reboot, and we want these changes (e.g. by the admin or a configuration management software) to be visible in the new system.

Warning

If a file has been modified both in the new snapshot and in the currently running system after the snapshot was created, then the changes done in the currently running system will be lost in the new snapshot (see the file3 case from the example above).

When the new snapshot is booted for the first time the systemd service transactional-update-etc-cleaner will check for these conflicts and print the results to the journal.

If the --continue option is used multiple times to extend a new snapshot while the system has not been rebooted, and if that snapshot is based on the currently active system, then the merge will only run for the first snapshot; the additional snapshot layers will be added to lowerdir. This is to make sure that changes to the running system will still be visible after booting into the new system.

The above /etc/fstab example is showing such a case: lowerdir contains two numbered overlay directories. The later directory with number 76 indicates the snapshot which was used as a base. As the lowerdir contains a second entry with number 81 it means that the --continue option has been used before the system was rebooted. Gaps in the numbers such as seen here may indicate that those snapshots were discarded or a rollback to snapshot 76 was performed. The lowest snapshot is always /sysroot/etc, containing the root file system's contents.

As /etc is mounted by dracut during early boot the options have to be prefixed with /sysroot. The x-systemd options are setting up the volume's systemd dependencies correctly.

Overlays no longer referenced by any snapshots will be deleted during the transactional-update cleanup-overlays run.