A file storage keeps each disk of a server as a file (qcow2, or raw when the node's disk format is raw) on a mounted file system. Every node has one, default; you can add more on the node's Storage tab. This page also explains the Storage tab itself, which every other storage kind shares.
The default storage#
The default storage is /var/lib/vmstack/images on the node. Besides servers' disks it holds the cached images, ISOs and rescue systems, and every server's home directory (its cloud-init seed, logs and journals), whatever storage its disks are on.
On a fresh node it is a directory on the root file system. To give it its own disks instead, either:
- install with
--setup-storage(see Installer options), or - run the setup command on a node that has no servers yet:
vmstack-agent storage list
vmstack-agent storage setup
storage setup lists the empty disks (and volume groups with free space), asks which to use, the layout and the share of the space, prints the exact commands and the disks it will erase, and changes nothing until you type yes. It makes one LVM volume with an XFS file system (mkfs.xfs -m reflink=1, so copies of images are instant and snapshots work), copies the cache onto it, mounts it at the storage path and adds it to /etc/fstab by UUID with nofail. It refuses a storage path that already holds servers' disks. Everything is logged to /var/log/vmstack-storage-setup.log.
| Layout | Disks | Size | A failed disk |
|---|---|---|---|
| linear | 1 or more | the sum | loses everything |
| mirror | 2 or more | the smallest disk | the others keep a copy |
| raid10 | 4, 6, 8, … | half the total | one per mirror pair |
On a node with servers, add the disks as a new storage from the panel instead (below).
The Storage tab#
Infrastructure › Nodes, open a node, Storage tab. At the top a summary line: how many storages, the GB allocated of the GB that may be sold, and the GB used on disk; and the node's Fill order.
The Storages table has one row per storage: Name, Type (File, File (adopted), LVM-thin, ZFS, NFS (shared), Ceph RBD (shared)), Where, Size, Used, Free, Allocated, Sell %, Tag, New servers, Status and Snapshots. Row actions: Edit, Grow, Extend with a disk (linear storages only), Remove, and arrows to change the order. Opening a row lists the server disks on it, its leftover files, and its cached image copies.
Statuses:
| Status | Meaning |
|---|---|
| Ready | in use |
| Syncing 34% | a mirror or RAID 10 still building its copies; usable |
| Degraded | a disk under it is missing or failed, or a pool is full; usable, an alert is raised |
| Unavailable | not mounted as it was made, read-only, or not answering; nothing is written there and servers with a disk on it are not started |
| Being made, Removing | a storage task is under way |
Disks on this node lists every disk and partition the node has: Device, Size, Model, Serial, SSD or HDD, and its state: Empty (with Add as storage), Empty, may hold data, or In use with the reason (holds the system, a physical volume of a group, mounted, in a ZFS pool, …). Rescan reads them again. Below, every volume group with its free space and Add storage from it. When a new empty disk appears, the node page says so.
Only one storage change runs at a time on a node; creating servers does not wait for it.
Add a file storage#
- On the Storage tab, click Add storage (or Add as storage on an empty disk, or Add storage from it on a volume group).
- Kind: Files (XFS).
- Made from:
- Volume group: takes a share (%) of the group's free space, or a size in GB. No disk is touched.
- Empty partition: an empty partition the node lists. Only the partition is erased; the rest of its disk is not touched. Use it for a disk split between the system and the servers.
- Empty disks: one or more empty disks, with a Layout (linear, mirror, RAID 10) and the share of the new group.
- Directory: a directory you mounted by hand. See Adopt a mounted directory.
- Give it a Name, an optional Tag (such as
NVMe) and choose Use for new servers. - For disks and partitions, the Erase section lists what will be erased (device, size, model, serial). Type the Device names (
sdc sdd) and Your password, then click Erase and add.
The node makes a volume group (vmstack2, vmstack3, …), a logical volume storage-<key> with an XFS file system, mounts it at /var/lib/vmstack/storages/<key> and adds it to /etc/fstab by UUID with nofail. The agent's service is ordered after the mount, so servers start only once it is there. Each step is checked and taken back on failure; the task's log (Storage changes › Show log) and /var/log/vmstack-storage-setup.log show every command.
What can be erased#
Only disks and partitions the node offers as empty at that moment can be erased: no file system, partition table, RAID or LVM signature, not mounted, not swap, not used by a server, at least 10 GiB. A disk whose first or last MiB is not all zeros is flagged may hold data and needs its own tick: I know sdc may hold data and want it erased. Right before erasing, the node checks again that each disk is the same one (serial, size, model, identity) and still empty. Disks that are not empty are never offered.
Adopt a mounted directory#
Made from: Directory takes a directory you mounted yourself, such as a second disk mounted at /data, as a storage. It must be a mount point, or be under /mnt, /srv, /data or /storage, and must not be on the node's system disk. System directories, /var/lib/vmstack itself, pseudo file systems and a directory on the same file system as another storage are refused. Enter the Path and Your password to confirm. VMStack only makes vms/ and base/ in it and never erases, grows or unmounts it.
Snapshots on an adopted directory need a file system that can clone files (XFS with reflink, Btrfs); otherwise the Snapshots column says no. To share an NFS export between nodes, use NFS shared storage instead of adopting a mount.
Edit, grow and extend#
- Edit: Name, Tag (optional) ("Plans with this tag place their servers here."), Sell percent (empty: the node's) and Use for new servers. Turning it off keeps the disks there; it only stops new ones.
- Grow: adds a share or a size of the volume group's free space, online (
lvextend, thenxfs_growfs). This is how you give the default storage the spacestorage setupleft in its group. - Extend with a disk: adds an empty disk to a linear storage, with the same erase confirmation. A linear storage over several disks is lost if any one of them fails. A mirror or RAID 10 storage cannot be extended this way; add the disk as a new storage.
Shrinking is never offered.
Remove a storage#
Remove is possible only for a storage with no disk on it (delete or move the disks first) and never for default. The node unmounts it, takes it out of /etc/fstab and forgets it; its files stay on the disk. Tick Also erase it to remove its logical volume too, and the volume group and its disks when nothing else is on them, so the disks show as empty again. Erasing asks you to type the storage's name and your password. An adopted directory is only forgotten.
Placement on storages#
- A plan's Storage tag sends its servers only to storages with that tag. A plan without a tag uses any storage marked Use for new servers.
- The node's Fill order chooses among them: Most free (default: the storage with the most sellable room left) or In order (the first storage in the table's order with room).
- A server's disks from its plan go on one storage when they fit together. Extra disks added later go on disk 1's storage when it has room.
- Administrators can choose the storage when creating a server (Storage, "Automatic follows the plan's storage tag and the node's fill order.") and when adding a disk.
- Each storage's sellable size is its size × its sell percent. See Capacity and overselling.
Leftovers and orphaned disks#
Files under a storage's vms/ directory that belong to no server the node knows are listed as leftovers in the storage's row, with Delete. Infrastructure › Orphaned disks lists them across every node. A leftover is never deleted automatically, and the node refuses to delete one while it has a record or a libvirt domain of that server.
When a storage is unavailable#
A storage is unavailable when its file system is not mounted where it was made, is read-only, or does not answer (a disk that stopped responding). While it is:
- nothing is written under its path; creating a disk there, reinstalls, snapshots and backups of servers with a disk there are refused;
- a stopped server with a disk there is not started; a running one keeps running;
- a critical alert is raised, and placement skips the storage.
When it is back, the node settles interrupted changes and starts the servers it held back after a boot. See Troubleshooting nodes and storage.