Disks, CPU limits and GPUs

Extra disks and their allowance, disk bus and cache, disk I/O limits, CPU limits and weight, dedicated CPUs, and GPU passthrough for virtual machines.

A virtual machine's resources go beyond its plan's size: extra disks its owner can add, limits that keep one server from starving the others on a node, CPUs pinned to it alone, and GPUs passed through whole. Everything on this page is for KVM virtual machines; containers take their CPU, memory, swap and I/O limits from their plan (KVM and containers).

Disks#

A server's Storage tab lists its disks. Disk 1 is the system disk; it cannot be disabled or deleted, and it grows through Manage › Upgrade (Upgrades and resizing). Each row shows the size, the bus, the device name as a hint, the disk's serial, the storage it is on, its limits and its state.

Every disk has a serial, vms- and 16 hex digits, which never changes. Inside the guest the disk is /dev/disk/by-id/virtio-vms-… (on SCSI, scsi-0QEMU_QEMU_HARDDISK_vms-…): use that name in /etc/fstab, since Linux names devices in the order it finds them.

TRIM is on for every disk on every bus: space the server frees with fstrim (Linux) or Optimize Drives (Windows) is given back to the node.

The extra disks allowance#

A plan's Extra disks section sets:

Field Meaning
Included with every server (GB) Disks every new server of the plan gets, one size per disk (Add size)
Most extra disks 0 to 15
Most extra disk space (GB) The total of all extra disks, included ones too

The owner (and its reseller) may add, grow, enable, disable and delete extra disks within the allowance, the reseller's pool and the node's capacity. Administrators are held only by the node's capacity; going over the allowance shows on the tab. An administrator can give one server an allowance of its own on its Manage tab (Extra disks allowance). Lowering an allowance deletes nothing; a server over it cannot add or grow disks until it is under. Node capacity and reseller pools count every disk.

Add, grow and remove a disk#

  • Add disk: a size within the allowance, an optional label, and (administrators) the bus. The disk is empty: VMStack does not partition, format or mount it. On a running server it appears at once on VirtIO and SCSI buses; on SATA, or with no free slot, at the next restart, and the row says "Attached at next restart".
  • Resize: an extra disk only grows. On a running server it grows at once (not on SATA: "Stop the server first: a SATA disk cannot grow while running."). Then grow the partition and file system inside (growpart and resize2fs, or extend the volume in Disk Management).
  • Configure › Disable: detaches the disk and keeps its data. Unmount it inside the server first; a disk in use may be released only at the next power off.
  • Delete: only for a disabled disk, or when the server is stopped. Type the disk's name ("Disk 2") to confirm. Its data is lost, and so are its copies in the server's snapshots.

To use a new disk on Linux:

lsblk
mkfs.ext4 /dev/disk/by-id/virtio-vms-XXXXXXXXXXXXXXXX
mkdir /data
echo '/dev/disk/by-id/virtio-vms-XXXXXXXXXXXXXXXX /data ext4 defaults,nofail 0 2' >> /etc/fstab
mount -a

XXXXXXXXXXXXXXXX is the disk's serial as the Storage tab shows it. On Windows: Disk Management › Initialize disk, then New Simple Volume.

Snapshots and backups hold every disk of the server. A reinstall writes disk 1 only; extra disks are kept and stay attached.

Bus, cache and IO driver#

Administrators set these per disk with Configure; owners and resellers see them as text. They take effect at the next power off and start (Save), or now with Save and restart now.

Field Choices
Bus Node default, VirtIO Block, VirtIO SCSI, VirtIO SCSI (threaded), SATA
Cache Node default, none, writethrough, writeback, directsync, unsafe
IO driver Node default, threads, native, io_uring

VirtIO is fastest; use SATA only for a system without VirtIO drivers. native needs cache none or directsync. unsafe may lose writes if the node loses power: throwaway servers only. The node's defaults are set in its KVM settings (Node settings).

Disk I/O limits#

Four values per disk, each empty for unlimited: read IOPS, write IOPS, read MB/s and write MB/s. A plan's CPU and disk I/O section gives every disk of its servers the same values. An administrator can give one disk its own under Configure (Custom, then Apply limits now). Limits apply at once, without a restart, and a plan change reaches its servers at once.

The Usage tab's disk chart draws the read and write MB/s limits of disk 1 as dashed lines, so a customer sees the ceiling they hit.

CPU limit and weight#

Value Meaning
CPU limit (% of each vCPU) 1 to 99, empty for none. A server with 4 vCPUs limited to 50% can use at most the equivalent of 2 cores; a single-threaded load gets at most half a core.
CPU weight 1 to 10 000, empty for 100. When the node is busy, a server with weight 200 gets twice the CPU time of one with 100.

Set them on the plan (CPU and disk I/O), or per server on its Manage tab (CPU limits, Custom, Apply now). They apply at once. Customers see "Limited to 50% of each vCPU" on the Overview.

These sections are on panel plans only. A reseller's own plans take the server limits an administrator set on the reseller's account page.

Dedicated CPUs#

A plan with Dedicated CPU on gives each vCPU of its servers a host CPU of its own: no other server runs there.

  1. On each node that should take such servers, open its Settings tab and, under Dedicated CPUs, enter CPUs for dedicated servers as a CPU list, for example 8-15,24-31. The section shows the node's sockets, cores, threads and NUMA nodes. Leave at least one CPU, ideally a whole core, for the node itself and the shared servers.
  2. On the plan's Network and processor section, turn on Dedicated CPU. The CPU model then defaults to host passthrough.

Every other server runs on the remaining CPUs, and the node's vCPU ratio counts only those. The allocator never gives a CPU twice, prefers whole cores, and keeps a server on one NUMA node where it can. Nodes without enough free dedicated CPUs are passed over ("Has 3 free dedicated CPUs; this server needs 4."). An upgrade re-pins, a move pins on the target, a delete frees the CPUs.

Existing servers are pinned at their next reinstall or upgrade, or by an administrator on the server's Manage tab (Dedicated CPUs, Pin or Release). The server's Overview then says "4 dedicated vCPUs".

GPU passthrough Beta#

Beta: GPU passthrough works and is being tested on more hardware; report any problem you find.

A node's GPU can be given whole to one virtual machine. The node finds its devices itself; nobody types device XML.

Prepare the node#

The node needs the IOMMU on:

  1. Turn on VT-d (Intel) or AMD-Vi (AMD) in the server's firmware setup.

  2. Add the kernel parameters to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub: intel_iommu=on iommu=pt (Intel) or amd_iommu=on iommu=pt (AMD), then:

    update-grub
    
  3. Move the node's servers away or put it in maintenance, and reboot it.

The panel never changes a node's boot configuration or reboots it. The node page's GPUs panel says what is missing, with the parameters for the node's CPU. A node that is itself a virtual machine has no IOMMU to give.

Offer a GPU#

On the node's page (Overview), the GPUs panel lists the display controllers, accelerators and co-processors the node has, each with its driver now, its IOMMU group, and Can be passed or Cannot be passed with the reason:

  • the IOMMU is off;
  • it is the node's boot display (on servers usually the BMC's own display);
  • it shares its IOMMU group with another device (a SATA controller, a NIC);
  • vfio-pci is not installed.

The other functions of the card's slot (its HDMI audio, its USB-C controller) go with it. Press Offer to servers on a device that can be passed, and choose Give back to host when freed or Keep on vfio-pci when freed. Stop offering takes it back once no server holds it.

Give a GPU to servers#

  • By plan: on the plan's Network and processor section choose the GPU model (with how many are free) and GPUs per server (1 to 8). Each new server gets its own GPUs of that model; placement chooses only nodes with that many free, and the placement log names the others' reason.
  • One server: on its Manage tab, GPU, choose a GPU and Give. A running server gets it at its next power off and start from the panel (a reboot from inside is not enough). Take away works on a stopped server only.

At every start the node binds the GPU to vfio-pci, also after a node reboot. Inside the guest, install the vendor's driver as on bare metal. The server's Overview shows the GPU's name to everyone who sees the server; customers never see its PCI address or node.

Limits#

  • A server with a GPU cannot be moved to another node, live or stopped. Take the GPU away first.
  • A suspension powers a running GPU server off instead of pausing it, and it stays off when the suspension is lifted.
  • Memory snapshots are refused; a revert with memory boots from the disk.
  • One GPU goes to one server: no vGPU, mediated devices, SR-IOV or USB passthrough yet, and no hot-plug.
  • Clones and imported servers get no GPU.