An administrator can move a server to another node in two ways:
- Offline: the server is stopped, its disks and firmware state are copied and checked, and it is started on the new node.
- Live (no downtime): a running VM keeps running while its memory (and any disks not on shared storage) are copied; it switches over in a fraction of a second.
In both, the source copy is never changed until the target runs the server. Any failure before that puts the server back on its node as it was. Moves are for administrators only (page and API).
Requirements#
- Agent 0.31.0 or later on both nodes for offline moves. Live moves need both agents to report live migration; the move page says when one does not.
- Nodes copy directly to each other over the migration port, TCP 8444. The agent opens it only while a move needs it, and only to the other nodes of the panel (its own nftables table, kept up to date by the panel). Nodes need no SSH between them and no libvirt or QEMU port.
- If the node runs another firewall (firewalld, ufw), it must let the other nodes in on 8444. The node firewall of the panel already allows the migration port from the other nodes only.
- Node certificates issued before moves existed cannot be presented to another node; the panel renews them by itself, and such a node is not offered as a target until then.
Move a server#
- Open the server and go to the Manage tab.
- On the Move to another node row, click Move…. The move page opens. (The button is disabled, with the reason, while the server is busy, suspended, in rescue, or its agent is too old.)
- Destination: choose the Destination node. Every other node is listed with its location; a node that cannot take the server is disabled, with the first reason it cannot. When none can: "No other node can take this server:" with the reasons per node.
- Mode: Live (no downtime) or Offline (stopped during the copy). Live is chosen whenever possible; when it is not, the page says why under "It cannot move live:".
- For a live move, optionally adjust:
- Longest stop at the switch (ms): 100–10 000, default 300.
- Slow the guest down if its memory changes faster than it is copied (auto-converge): off by default.
- Give up after (minutes): worked out from the disks and memory by default. "If the copy does not finish in time it is stopped and the server keeps running here."
- Disks: the target storage for each disk ("Disk 0 (40 GB) goes on …"), with the room left. A disk on a storage both nodes share is not copied.
- Addresses (offline): Keep its addresses when the new node has an IPv4 block on the same network (same gateway and prefix), or New addresses from node … (the same count; IPv6 and NAT addresses are always new). A live move always keeps every address.
- The copy: tick Its N snapshots are deleted by the move if the server has snapshots (required). Optionally set a Copy speed limit (MB/s).
- Check the summary and What else changes, then click Move server.
What happens to the server#
| Part | On a move |
|---|---|
| Disks | Copied in their format, sparse (only the data is sent, compressed, when both agents support it). Disks on LVM-thin or ZFS are copied through a file while the server is stopped. Disks on a shared storage both nodes mount (NFS, Ceph) are not copied. |
| UEFI variable store, TPM state, first-boot seed | Copied (offline). A live move cannot carry a TPM. |
| Hardware settings, boot order, disk tuning, CPU limits, processor flags, network rules, firewall, speed limits, start on boot | Carried over. |
| Snapshots | Deleted, after the tick. They are not copied. |
| Backups | Stay, and can be restored on the new node. Scheduled backups go on. |
| ISO drives | Emptied (offline). A live move needs empty drives. |
| IPv4 | Kept (same-network rule) or new. Kept addresses are announced from the new node with gratuitous ARP. |
| IPv6, NAT | New on the new node (offline). NAT port forwards keep their server ports; a public port taken on the new node gets a new one, and the page lists each change. |
| Reverse DNS, DNS records | Updated for new addresses. |
| Server ID, MAC, host name, owner, plan, traffic history | Unchanged. |
| Passed-through GPU | A server with a GPU cannot move, live or stopped. Take the GPU away first (Manage tab, GPU), then give it one of the new node's. |
When the addresses change and the server was installed from an image, the new node writes a network seed so the guest takes the new addresses at its first start (SSH host keys kept). A server installed from an ISO must be changed by hand inside; the page says so.
While it moves#
The Manage tab's row shows the phase and a progress bar:
- offline: "Stopping the server and checking its disks on node node-1.", then "Copying to node node-2: …" with the bytes and speed, then "Running on node node-2; removing the copy on node node-1.";
- live: "Preparing node node-2: its CPU, memory and storage checked, the disks made ready.", "Copying the disks while it runs: …", "Copying memory: pass 3, 412 MB left, the guest changes 38 MB/s", then "Switching over: node node-2 confirms the server runs there."
Every other control of the server is disabled with "Server … is being moved to node …". Firewall and port forward changes are refused until the move ends.
Afterwards the row says "Moved from node node-1 to node node-2 on … UTC", with how long it took and how much was sent. The owner gets the optional server.moved mail with how long the server was stopped and what became of its addresses (turn it off in the notification settings).
Cancel a move#
Click Cancel move. It works until the copy is done (offline) or during the memory copy (live, which aborts the migration job). The move is taken back and the server stays on its node. If the copy had already finished, the move completes and the page says the cancel came too late.
Failures#
- Before the switch: the target removes what it made, and the source is released: started again if it ran (offline), or it simply keeps running (live). The row says why, for example "The move to node node-2 failed: … The server is back on node node-1."
- A step of the way back that fails leaves the move with a clean-up error and a Retry button. The server stays held so it can never run on both nodes.
- After the switch, if the source could not delete its stopped copy, the row says the old node still holds a stopped copy of the server. Click Retry clean-up. The server cannot be deleted, and no new move starts, until the clean-up is done.
What a live move needs#
A server moves live only when all of these hold; otherwise it moves offline:
- it is a running VM (containers move offline only);
- no TPM and no ISO in a drive;
- not behind NAT, and its IPv4 (and IPv6) addresses can be kept on the new node;
- the new node can run the server's CPU model (a server using
host-passthroughmoves offline only) and machine type, and has its memory free; - disks that are not on shared storage go onto a file storage on the new node.
What is verified: copied disks are mirrored by QEMU until they converge, every byte crosses the agents' TLS connection, and sizes and formats are checked. A whole-disk checksum is not possible for a disk that changes while it is copied; offline moves keep their checksum.
A guest that rewrites its memory faster than the link carries it never converges: the move gives up after its time and the server keeps running on the source. Try again with auto-converge, which throttles the guest's CPU during the copy. On a slow link, a write-heavy guest's disk writes are slowed during a live disk copy.
Moving many servers#
To empty a node before maintenance, use Empty this node on the node; it moves running servers live where they can. See Node maintenance.
The placement log#
Every time the panel picks a node for a new server it records the decision: the rule, what the server needed, every node it looked at with its verdict and reasons, and the node chosen.
- Open System › Placement log. Filter by Outcome (No room or failed, Created). Records are kept 90 days.
- Open an attempt to see "Nodes looked at, in order": each node's verdict (Chosen, Accepted: could have taken it, or Rejected) and why, with numbers, for example "2 free IPv4 addresses; this server needs 3" or the memory, vCPU and disk figures with the sell ratio.
- From a server: Overview › Details › Placement › Why this node. From a
vm.createtask's page: Placement › Why this node. From the create form, an administrator's "no room" refusal links to the attempt.
Reasons include: disabled, not enrolled, offline or not answering, wrong location, no NAT network, agent too old, no IPv6, no free IPv4, CPU model, firmware (UEFI or TPM missing), storage, memory, vCPUs, disk and server caps. The rule is the location's or plan's Placement rule (Most memory left, Most storage left, Fewest servers, Round-robin, Fill first), a node group's order, or "Chosen by an administrator".
The placement log is for administrators only; resellers and users keep the plain message "This location has no room for a server of this size right now."
API#
GET /servers/{id}/move-targets, POST /servers/{id}/move (node_id, storages, keep_addresses, limit_mb_per_second, drop_snapshots, mode, max_downtime_ms, auto_converge, timeout_minutes), GET /servers/{id}/move, POST /servers/{id}/move/cancel, POST /servers/{id}/move/retry, with the infrastructure:read and infrastructure:write scopes. Placement: GET /servers/{id}/placement and GET /placements/{id}. Webhooks: server.move_started, server.moved, server.move_failed. See the API reference.