Three actions help when a server no longer boots, lets nobody in, or has lost its network: Rescue, Reset root password and Reconfigure network. Each runs on the node as a task, behind a confirmation dialog, for whoever manages the server: an administrator, its reseller or its owner (a customer needs the matching customer ability: rescue, root password reset, network reconfigure).
Where to find them#
- The server's Settings tab, panel Recovery: Rescue (or Exit rescue), Reset root password and Reconfigure network.
- The Network tab, panel Inside the server: Reconfigure network.
- The Overview tab of a server in rescue: Exit rescue.
- Containers have no rescue system; their Reset root password is on the Manage tab and works at once, without a restart.
All three are closed while the server is suspended, while its node does not answer, and while another task of the server runs.
How reset and reconfigure work#
Both write a second first-boot seed for cloud-init with a new instance id, then restart the server, so the guest applies the settings again at start-up. To keep that from redoing a first boot:
- the SSH host keys stay (nobody sees a "host key changed" warning);
- the host name, SSH keys and the SSH password login setting stay as they are;
- no packages are installed, no commands run, nothing else is rewritten;
- the network section is the server's stored network: addresses, prefix, gateway, DNS, matched by MAC.
A success means the node gave the server the new seed and started it. The node cannot see inside the guest, so it cannot confirm the guest took it.
Note: Both work only for servers installed from an image. For a server installed from an ISO the buttons are disabled with: "This server was installed from an ISO, so the panel cannot set anything inside it at start-up. Use the console, or Rescue."
Reset the root password#
- Open the server, go to Settings, and in Recovery click Reset root password.
- Read the dialog and click Reset Root Password.
The panel generates a new 20-character password and sends only its hash. The server is shut down (powered off after 90 seconds if it does not stop) and started again. The new password replaces the stored one only when the node confirms; the Overview tab then shows it under Root password (Show is recorded in the audit log). If the reset fails, the old password stays and the page says so.
- Stopped server: it is not started. The page says "The new password is used from the server's next start."
- Windows: the button reads Reset Administrator password. With the QEMU guest agent running in Windows the password changes at once, without a restart; otherwise the server restarts and cloudbase-init sets it.
- FreeBSD: the QEMU guest agent sets it while the server runs; only when the agent does not answer is the server restarted.
If the new password does not work, the guest no longer takes settings at start-up (cloud-init was removed or disabled). Use the console or Rescue.
Reconfigure the network#
Use it after changing a server's addresses, or when someone broke the network configuration inside the server.
- Open the server and click Reconfigure network (on Network or under Settings › Recovery).
- Read the dialog and click Reconfigure Network. The server is restarted.
The stored addresses, gateway, DNS and interface configuration are written into the guest. Custom network configuration inside the server may be overwritten, and a heavily modified system may not take it.
Rescue#
Rescue starts the server from a throwaway rescue system, with the server's own disk attached unchanged as the second disk. It works for any server (image or ISO install, and a server in error).
Before you start#
The rescue system is built from the Debian 13 image. Enable it under Images and ISOs › Images. Without it, administrators read "The rescue system needs the Debian 13 image; enable it in Images." and everyone else "Rescue is not available right now. Contact support."
The node downloads the image on first use, then prepares a rescue base once per image. It gives the rescue system's root file system and partitions new UUIDs, so the rescue system can never boot from or mount the customer's disk by mistake (a server made from the same Debian image would otherwise carry the same UUIDs). If the node cannot prepare the base (a tool missing, an unknown image layout), rescue is refused with "The rescue system could not be prepared on this server's node." and is never started from an unmodified copy. Administrators see the node's reason.
Start rescue#
- Open the server, go to Settings, and in Recovery click Rescue.
- Click Start rescue.
The server is shut down (90 seconds, then powered off) and started from the rescue system, with the same network card, MAC and address. The status becomes Rescue (amber).
Work in rescue#
The Overview tab shows a Rescue panel:
- the SSH command,
ssh root@<address>. The rescue system has its own SSH host key, so your SSH client may warn that the key changed; - a Temporary root password for the rescue system (Show, recorded in the audit log), valid over SSH and on the console. The server's recorded SSH keys work too;
- where the disk is.
Inside the rescue system:
lsblk /dev/vdb # list the server's partitions
vgchange -ay # only if the server's system is on LVM
mount /dev/vdb1 /mnt # the partition number depends on the system
Extra disks follow as /dev/vdc, /dev/vdd and so on; the panel lists them. When the network allows it, the rescue system installs lvm2, xfsprogs, btrfs-progs, ntfs-3g, cryptsetup and dosfstools at its first boot (best effort).
While in rescue:
- power actions and the console act on the rescue system, which stays in rescue across a restart, an agent restart and a node reboot;
- reinstall, settings, ISO drives, boot once, password reset, reconfigure network and snapshots are refused with "This server is in rescue. Exit rescue first.";
- delete is allowed and removes the rescue disk too.
A server in rescue for more than a day is marked on its page, in the server list and, for administrators, on the dashboard. Nothing ends rescue by itself.
Exit rescue#
- Click Exit rescue (on Overview or under Settings › Recovery).
- Confirm with Exit rescue.
The rescue system is shut down and thrown away with anything saved on its own disk. The server's own definition (boot order, settings, ISO drives) is put back and the server starts from its own disk. Changes made to /dev/vdb stay.
Note: The rescue system boots with SeaBIOS even for a UEFI server; the server's own firmware, variable store and TPM state are back when rescue ends.
Audit#
vm.password_reset, vm.network_reconfigured, vm.rescue_started, vm.rescue_ended, and vm.password_revealed each time a password is shown (which: rescue for the rescue password).