Notes on How to Disable the Cache File in ZoL
I noticed that even though I created our data pool with -o cachefile=none, the vdevs were still present in /etc/zfs/zpool.cache, and it was still getting imported that way. When I looked at the pool properties, cachefile was set to the default value of - instead of the local value of none. I believe this was happening because the cache file is very “sticky”: if it’s there, ZoL wants to use it—overriding the cachefile property—and I think it got there in the first place because there’s a second pool on the machine (the root pool) that was configured to use it.
This is bad for zfs-ha, so I set about trying to fix it. I decided to disable the cache file entirely, for both pools, because that seems to be an ultimate goal of the project anyway. Here are my notes on how I accomplished it; it wasn’t hard, but it feels worth documenting. Please chime in if you have answers for my open questions and/or suggestions on better ways to do any steps of the process.
The root pool is named rpool and the data pool is named tank, so adjust accordingly if you plan to follow some or all of these steps. These instructions assume that you’re on Ubuntu 18.04—although they might be portable to other ZoL targets, such as Ubuntu 16.04 and Debian 9—and have zfs-initramfs 0.7.x installed.
Set the cache file to none for each and every pool:
zpool set cachefile=none rpool
zpool set cachefile=none tank
Delete rm -f /etc/zfs/zpool.cache
Make sure zfs-import-scan.service is enabled
Make sure zfs-import-cache.service is disabled (optional; ZoL will conditionally use the correct service based on the presence or absence of the cache file)
Set the following variables in /etc/default/zfs:
ZFS_POOL_IMPORT='rpool'
N.B. This selects which pool(s) to import at boot time. It should be set to the name of your root pool!
ZPOOL_CACHE='none'
N.B. I think this can be any string that will fail a test -f, based on my reading of the ZoL source.
ZPOOL_IMPORT_OPTS='-o cachefile=none'
N.B. I think this is superfluous if you set ZPOOL_CACHE='none', but am unable to confirm. My hope is that this guarantees that a cache file will never be created at import time, although I don’t know if it has any effect on manual imports.
ZPOOL_IMPORT_PATH='/dev/disk/by-vdev:/dev/disk/by-id' (optional)
N.B. This is not strictly required for this exercise, but I prefer WWNs and would like the option of adding a vdev_id configuration in the future. This just makes sure the scan operation doesn’t import a pool using e.g. /dev/sdX, which is particularly important if you’re using dm-multipath (which we are for zfs-ha).
Update initramfs:
update-initramfs -u -k all
Reboot!
I noticed that even though I created our data pool with -o cachefile=none, the vdevs were still present in /etc/zfs/zpool.cache, and it was still getting imported that way. When I looked at the pool properties, cachefile was set to the default value of - instead of the local value of none. I believe this was happening because the cache file is very “sticky”: if it’s there, ZoL wants to use it—overriding the cachefile property—and I think it got there in the first place because there’s a second pool on the machine (the root pool) that was configured to use it.
This is bad for zfs-ha, so I set about trying to fix it. I decided to disable the cache file entirely, for both pools, because that seems to be an ultimate goal of the project anyway. Here are my notes on how I accomplished it; it wasn’t hard, but it feels worth documenting. Please chime in if you have answers for my open questions and/or suggestions on better ways to do any steps of the process.
The root pool is named rpool and the data pool is named tank, so adjust accordingly if you plan to follow some or all of these steps. These instructions assume that you’re on Ubuntu 18.04—although they might be portable to other ZoL targets, such as Ubuntu 16.04 and Debian 9—and have zfs-initramfs 0.7.x installed.
Set the cache file to none for each and every pool:
zpool set cachefile=none rpool
zpool set cachefile=none tank
Delete rm -f /etc/zfs/zpool.cache
Make sure zfs-import-scan.service is enabled
Make sure zfs-import-cache.service is disabled (optional; ZoL will conditionally use the correct service based on the presence or absence of the cache file)
Set the following variables in /etc/default/zfs:
ZFS_POOL_IMPORT='rpool'
N.B. This selects which pool(s) to import at boot time. It should be set to the name of your root pool!
ZPOOL_CACHE='none'
N.B. I think this can be any string that will fail a test -f, based on my reading of the ZoL source.
ZPOOL_IMPORT_OPTS='-o cachefile=none'
N.B. I think this is superfluous if you set ZPOOL_CACHE='none', but am unable to confirm. My hope is that this guarantees that a cache file will never be created at import time, although I don’t know if it has any effect on manual imports.
ZPOOL_IMPORT_PATH='/dev/disk/by-vdev:/dev/disk/by-id' (optional)
N.B. This is not strictly required for this exercise, but I prefer WWNs and would like the option of adding a vdev_id configuration in the future. This just makes sure the scan operation doesn’t import a pool using e.g. /dev/sdX, which is particularly important if you’re using dm-multipath (which we are for zfs-ha).
Update initramfs:
update-initramfs -u -k all
Reboot!