Extract Cluster configuration - /etc/pve

There are cases when you make changes in your configurations, only to want to partially revert it back.

Alternatively, you get hold of stale (from non-quorate node) or partially corrupt config.db and want to take out only some of the previous files. without making it your current node’s cluster filesystem.

Less often, you might want to edit the contents of the database-backed filesystem without side effects to the node or cluster, e.g. in order to implant it into a separate/cloned/new cluster.

Extraction

This is actually possible, however since the pmxcfs ¹ relies on hardcoded locations for its backend database file as well as mountpoint, you would need to use chroot. ²

mkdir -p ~/jail-pmxcfs/{dev,usr,bin,sbin,lib,lib64,etc,var/lib/pve-cluster,var/run}
for i in /dev /usr /bin /sbin /lib /lib64 /etc; do mount --bind -o ro $i /root/jail-pmxcfs/$i; done

This will create alternative root structure for your own instance of pmxcfs, the only thing left is to implant the database of interest, in this example from existing one:

sqlite3 /var/lib/pve-cluster/config.db .dump > ~/config.dump.sql
sqlite3 ~/jail-pmxcfs/var/lib/pve-cluster/config.db < ~/config.dump.sql

Now launch your own pmxcfs instance in local mode (-l) in the chroot environment:

chroot ~/jail-pmxcfs/ pmxcfs -l

You can double check your instance runs using the database file that was just provided:

lsof ~/jail-pmxcfs/var/lib/pve-cluster/config.db

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
pmxcfs  1225 root    4u   REG  252,1    77824   61 /root/jail-pmxcfs/var/lib/pve-cluster/config.db

In fact, if you have the regular pve-cluster service running, you will be able to see there’s two instances running, each over its own database, the new one in local mode (-l):

ps -C pmxcfs -f

UID          PID    PPID  C STIME TTY          TIME CMD
root         656       1  0 10:34 ?        00:00:02 /usr/bin/pmxcfs
root        1225       1  0 10:37 ?        00:00:00 pmxcfs -l

Now you can copy out your files or perform changes in ~/jail-pmxcfs/etc/pve without affecting your regular operation.

You can also make an SQL dump ³ of ~/jail-pmxcfs/var/lib/pve-cluster/config.db - your now modified database.

Cleanup

Once you are finished, you will want to get rid of the extra instance (based on the PID of the local (-l) instance obtained above):

kill $PID

And destroy the temporary chroot structure:

umount ~/jail-pmxcfs/etc/pve ~/jail-pmxcfs/* &&
rm -rf ~/jail-pmxcfs/