VLANs

Carry tagged VLANs on a node's uplink, put on-link IPv4 blocks on them, and bridge servers onto the VLAN without touching the node's own network.

Beta. This feature works and is being tested on more setups. Report anything that does not behave as described here.

A VLAN carries an IP block on its own segment of your switch or your provider's network (a Hetzner vSwitch, an OVHcloud vRack). Servers with an address of the block are bridged onto the VLAN and reach its gateway there. VLANs are in beta: they work, and are being tested on more switch and provider setups; please report problems.

VLANs are for administrators, under Infrastructure › VLANs.

How VLANs work in VMStack#

  • A VLAN is an object of its own: a name, a tag (VLAN ID 1 to 4094), an MTU (1280 to 9000, default 1500), a note, and the nodes that carry it. Two VLANs may have the same tag (two datacentres can each have a VLAN 100); a node carries each tag once.
  • For each VLAN a node carries, the agent makes an 802.1Q interface vmtag<tag> on the node's parent interface and a bridge vmvl<tag> for the servers' cards. Both get the VLAN's MTU, no address, IPv6 off and ARP off. The node never has an address on a VLAN and drops anything from a VLAN to itself.
  • Everything is made at run time with ip link. Nothing is written into netplan, ifupdown or NetworkManager files, and the uplink's addresses, routes, link state and MTU are never changed. A reboot returns the host to its own configuration; the agent makes the VLANs again at start, before the servers start.
  • An on-link IPv4 block (gateway inside the block) can be bound to a VLAN. Its servers are bridged onto the VLAN and use the block's prefix and gateway (the router on that VLAN).
  • Tagged only: a server's card is an untagged port of one VLAN. Guests never send or receive tagged frames.

Before you start: the switch#

VMStack configures the node, never the switch. The node's switch port must carry the VLANs tagged (an 802.1Q trunk), with the node's own network untagged as it is now.

Where What to set up
Hetzner vSwitch Create the vSwitch in Robot, VLAN ID 4000 to 4091, add the node, order a subnet for the vSwitch. MTU 1400. The subnet is the IP block; its gateway is inside it.
OVHcloud vRack The VLAN ID you tag in the vRack; an IP block routed into the vRack, its gateway the block's last usable address. Untagged (VLAN 0) is not supported.
Your own switch The node's port as a trunk with the tags, and a router on each VLAN at the block's gateway.

A node whose uplink is a bridge of the host's own (br0, as some installations make) or a bond member cannot carry VLANs: name the bond, or the physical port, as its VLAN interface. VMStack's own uplink bridge vmbr2 is fine: its port is used.

Create a VLAN#

  1. Go to Infrastructure › VLANs and click New VLAN….
  2. Enter a Name (such as Backend), the VLAN ID and the MTU (1400 on a Hetzner vSwitch).
  3. Tick the Nodes that carry it.
  4. Add a Note if you want, and click Make VLAN.

The nodes get their new state within a minute. Each node's page (Overview tab, VLANs panel) shows the VLAN as up, or its problem.

The tag and the MTU change only while no server is on the VLAN. A node leaves a VLAN only once none of its IP blocks is on it. A VLAN with IP blocks on it cannot be deleted: unbind or delete them first.

Choose the VLAN interface#

By default the parent interface is the node's uplink (the default route's interface), or the port of the uplink bridge vmbr2 when there is one. To use a bond or a second port that carries the trunk:

  1. Open the node, find the VLANs panel on its Overview tab and click Change VLANs….
  2. Enter the Interface (such as bond0 or enp1s0f1); leave it empty for the uplink.
  3. Tick the VLANs it carries, and click Save.

The interface changes only while no server is on the node's VLANs.

Put an IP block on a VLAN#

  1. Add the VLAN's subnet as an IPv4 block of type Gateway inside the block, on a node that carries the VLAN (IP pools). The gateway is the router on the VLAN.
  2. On the block's page, click Edit block.
  3. Under Additional settings, choose the VLAN (only VLANs the node carries are listed; None: on the node's link unbinds it), and click Save changes.

Only a public IPv4 block on the node's link can be on a VLAN. Routed, bridged, IPv6 and NAT blocks cannot. The binding changes only while no address of the block is in use. Binding a block turns its address check off: the uplink does not see the VLAN's segment.

The block's page shows its VLAN; Infrastructure › IP pools lists it, and the VLAN's own page lists its blocks with their use.

Servers on a VLAN#

A server whose main address is in a VLAN block is a VLAN server:

  • Its card is on vmvl<tag> with the VLAN's MTU (also set inside the server by the seed: Linux, FreeBSD and Windows). The server's Overview tab shows VLAN with its MTU under Details.
  • Its additional addresses come only from blocks of the same VLAN on its node. A server on the node's link or routed never gets a VLAN address.
  • Addresses of a VLAN are given only while the node reports the VLAN up.
  • Two servers on one VLAN reach each other directly, on the same node or across nodes through the switch (cards on a VLAN are not port-isolated). Anti-spoofing still binds each card to its MAC and addresses, and is required: a VLAN server is refused on a node with anti-spoofing off.
  • A server installed from an ISO gets its main address by DHCP from the agent's responder; see Bridged networking.

Moving a VLAN server keeps its addresses only to a node that carries the same VLAN and has a block on the same network bound to it. Otherwise it gets new, plain addresses on the target and stops being a VLAN server; the move dialog says so.

Protections#

On every VLAN bridge, from and to the servers' taps: VLAN-tagged frames (802.1Q and 802.1ad) are dropped, so a guest cannot hop to another VLAN of the trunk; DHCP and DHCPv6 server answers and router advertisements from servers are dropped; ARP and neighbour discovery flood limits, the metadata block and SMTP rules apply as on the uplink bridge. The table inet vmstack_vlan drops input from VLAN bridges to the node and any routing into or out of a VLAN.

Warning: While the agent runs, DHCP requests from servers on a VLAN bridge are always dropped at the bridge. A DHCP server on the VLAN's own router no longer reaches VMStack servers: they get the agent's answer only.

Check a VLAN on the node#

The node page shows each VLAN's state. On the node itself:

ip -d link show vmtag100
bridge link | grep vmtag100
ip addr show vmvl100
nft list table inet vmstack_vlan

vmtag100 should be a VLAN with id 100 on the parent, a port of vmvl100; vmvl100 should have no address. To see a server's frames leave tagged:

tcpdump -e -ni eth0 vlan 100

Replace 100 with the tag and eth0 with the parent interface.

Refusals and problems#

The agent refuses what is not safe, makes nothing, and reports the reason on the node page and the VLAN's page:

Problem Cause and fix
"VLAN 100 is already on this host as eno1.100 (perhaps the node's own network): the agent does not touch it" The host uses that tag itself, on any interface. Use another tag, or remove the host's own interface if it is not needed.
"eno1 is a bridge (br0): VLANs need its physical port or bond" Name the physical port or bond as the node's VLAN interface.
"eno1 is a port of bond0: use bond0" Name the bond.
The VLAN's MTU is above the parent's Lower the VLAN's MTU, or raise the parent's MTU.
"the node's own network changed while VLAN … was made" The node's default routes or addresses changed while the VLAN was made, so the agent removed it again. It retries every minute.
"A node carries each tag once" Two VLANs with the same tag on one node.
vlan_unavailable on create, start or move The node does not carry the VLAN, or it has a problem.
The node refused the state (invalid_vlans) Shown as the node's VLAN problem; the reason is in the message.

The agent applies the state again every minute while a VLAN or the parent has a problem, so a tag the host frees or an interface that appears is picked up by itself.

Not supported#

  • Trunks to guests (a tagged card on a server).
  • IPv6 on VLANs; NAT, IPv6 and the per-server firewall on VLAN servers.
  • Routed, bridged, IPv6 and NAT blocks on a VLAN.
  • The address check on a VLAN.
  • Private networks carried on a VLAN.
  • Configuring the switch.