LVM and LVM-thin

File storages on LVM, and LVM-thin pools with a thin volume per disk: instant creation and snapshots, over-provisioning and autoextend.

VMStack uses LVM in two ways. A file storage made from disks or a volume group is an LVM logical volume with an XFS file system on it, and each server disk is a file there (see Local file storage). An LVM-thin pool has no file system: each server disk is a thin logical volume in the pool. Plain (thick) LVM, a fully allocated volume per disk, is not offered.

When to use a thin pool#

File storage on LVM LVM-thin pool
Disk qcow2 or raw file on XFS raw thin volume
New server from an image instant (reflink copy) instant (thin snapshot of the image)
Snapshots reflink copies thin snapshots
Space the file system's the pool's data and metadata
Over-provisioning sell percent above 100 sell percent above 100; the pool can fill and stop writes
Incremental backups yes, for qcow2 disks no: always full backups

A thin pool suits I/O-heavy servers: no qcow2 layer and no file system on the I/O path. It needs watching when over-provisioned.

Add an LVM-thin pool#

The node must report LVM-thin support (any current agent does); otherwise the kind is not offered.

  1. On the node's Storage tab, click Add storage.
  2. Kind: LVM-thin pool.
  3. Made from: Volume group (a share of its free space, or a size in GB, metadata included), Empty partition, or Empty disks with a layout (linear, mirror, RAID 10). Directory is not possible for a pool.
  4. Enter a Name, an optional Tag and Use for new servers.
  5. For disks and partitions, type the device names and your password in the Erase section and click Erase and add.

The node makes a thin pool thin-<key> in the group, tagged vmstack. Its metadata is 1% of the data, at least 1 GiB and at most 16 GiB; a pool needs at least 2 GiB of data. A new pool sells at 100%. On the node:

lvs -a vmstack2

shows the pool, its hidden metadata volume and LVM's spare copy of the metadata.

From the node's command line the same is:

vmstack-agent storage add --key fast --kind lvm-thin --vg vmstack2 --percent 50

Note: The first thin pool on a node raises the agent's state version: an agent release older than LVM-thin support refuses to start on that node rather than lose track of its disks. Roll back the agent only to a release that knows thin pools.

How disks live on a pool#

  • Disk 1 of a server is vm-<id>-main, an extra disk vm-<id>-<disk>, both raw. A server whose first disk is on a thin pool has raw disks.
  • For each image, the pool holds one golden volume img-<checksum>-<key>, written once from the cached image. A new server's disk is a thin snapshot of it, extended to the disk's size: created in seconds and sharing its blocks.
  • Snapshots of a server are thin snapshots of its disks. They need all of the server's disks on block storages (thin or ZFS) or all on file storages that clone files; a server with disks on both is refused.
  • Everything else of the server (its cloud-init seed, logs, journals, backup copies) stays in its home on the default storage.
  • Space freed inside the server with TRIM (fstrim) is given back to the pool.

Watch data and metadata#

The pool's row on the Storage tab shows Used as the pool's data %, and metadata N% under it: amber at 75%, red at 90%. Allocated shows the disks' sizes against what may be sold, with over-provisioned 1.2× when the disks add up to more than the pool's size. Alerts are mailed for the pool's data % (the storage thresholds) and for its metadata at 75% (warning) and 90% (critical).

When the pool is full (data or metadata at 100%), it is Degraded: servers keep running and reading, but writes fail until the pool is grown. To keep that from happening:

  • New disks, growth, copies into the pool and snapshots are refused at 100%, and already at 90% when the pool is over-provisioned (10% is kept free).
  • When the sell percent is above 100 or the data passes the warning threshold, the row shows guidance: keep free space in the volume group and let LVM grow the pool by itself, by setting in /etc/lvm/lvm.conf:
thin_pool_autoextend_threshold = 80
thin_pool_autoextend_percent = 20

The guidance shows the node's current values ("Now: threshold 100 (autoextend off), percent 20.") and how much free space the group has. VMStack never changes lvm.conf itself.

Grow, extend and remove#

  • Grow: adds a share or size of the group's free space to the pool, and grows its metadata to 1% of the new size when it is below. A full (degraded) pool may be grown: that is how it is mended.
  • Extend with a disk: a linear pool on a group VMStack made takes a whole empty disk.
  • Remove: refused while a server has a disk there or the pool holds anything but golden volumes. The golden volumes are removed; a pool VMStack made, now empty, is removed too. Also erase the volume group and its disks when nothing else is on them gives the disks back as empty. A pool made by hand (without the vmstack tag) is only forgotten.

Moves#

  • Between storages of the node: a thin disk can move to a file storage and back (see Moving disks). A move between two thin pools of the same volume group is refused (same volume group): disk names are per group.
  • To another node: once the server is stopped, each thin disk is copied into a file first, and the target imports the server onto its file storages. A thin pool is never the target of a move between nodes; move the disk onto it afterwards.

Not supported#

  • Adopting a thin pool you made by hand: only pools VMStack made are used.
  • Incremental backups of thin disks: their backups are always full (qcow2 copies), to every destination, and restore onto any storage.
  • Changing lvm.conf from the panel.
  • Shared (cluster) LVM.