Proxmox networking - part 2

Chop up the network for security and speed. VLAN is the corporate way to segregate by duty and function. We should split our network to get rid of all the unnecessary traffic coming to the machines and also to keep up segregation of duty in the data center - home-lab.

A virtualization server allows you to run multiple machines, virtual machines (VM's/CT's), on one physical device, also known as the host node. In a commercial production system, these virtualization servers will often run similar VM's as big cluster and/or K8S clusters.
In homelab servers, there are many different VM/CT's each for different tasks. You could have containers for: your firewall, one for valuable storage, a git server, home automation, media servers of different types, development servers, others might be hosting a NGINX reverse proxy another web servers. And with many server the bigger the mess.

Why Would You Want VLAN

Why VLAN's on Proxmox?

• Should you setup your Proxmox virtualization server to use VLAN's so that you can group related VMs onto their own subnets?
• Should the guest net and road warriors be on a separate VLAN?
• Cluster management needs a dedicated low latency network but isn't hungry for speed

Real rust

For security, as well as organizational purposes, physical machines are often separated on the network from each other by VLANs.

By logically separating devices based on their functionality with VLAN's, we can make sure that personal devices aren't sitting out in the open on the same subnet exposed to our internet-facing web servers or accounting is separated from sales and services separated from both.

This is fairly easy on your physical network, the devices are physically separate from each other and so each physical port of a device can be assigned an individual VLAN.

Virtual set-up's

The virtual machines are all sitting on the same physical host, forcing each VM/CT to share the same physical connection. With a standard bridge between the individual VM/CT and the host's NIC, we will haveall VM/CT's on the same sub-net as the Proxmox host itself - bad. Additionally, we end up with each VM on the same subnet as every other VM on that host -bad.

Design the solution

Planning is at least 80% of networking and 20% is install, training of personnel, fixing errors and mistakes, but mostly monitoring.

Make Pve Virtual Bridges VLAN Aware

In Proxmox, you can make your virtual bridge VLAN-aware so you can pass multiple VLANs through to your Proxmox server using only a single physical port. The individual VMs can then be configured to use whichever VLAN you choose.

Use more than 1 NIC, add separate ports for cluster and migration if they are to be used in your lab. Enterprise gear usually have 4 ports or more.

Redundancy is needed for HA, meaning more than one NIC (not 2 port from the same card!).

auto vmbr0
iface vmbr0 inet static
        bridge-ports ens1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4092

auto vmbr0.10
iface vmbr0.10 inet static
        address 10.10.10.45/24
        gateway 10.10.10.1
#MGMT Network

Design your VLAN's and set-up the switches

VLAN1 is a special one and will not be used 4093 and 4094 like vise. Check for something special on your ISP's net, like a hidden IPTV VLAN.
All the other ones are free to us as you pleas. You could do 10, 20, 30 for subnets 10.10.10.0, 10.10.20.0, 10.10.30.0 or 100, 200, 300...
Set up your needed VLAN's on the network switches.

Subnets are easier to control if you use easy-to-understand binary values. You can choose any of the reserved private IPv4 network range in RFC 1918 for your subnets: 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16.

Subnet host count

The number of subnets available and the number of possible hosts in a network may be readily calculated. For instance, the 192.168.0.0/24 network may be subdivided into the following four /26 subnets. The highlighted two address bits become part of the network number in this process.

For the network marked in CIDR 192.168.0.128/26 we get:
Netmask 255.255.255.192 = 26 binary 1.1.1.11 000000
Wildcard in binary 0.0.0.63 0.0.0.00 111111
Addresses max (six bit wildcard ) 2⁶ = 64 and Hosts/Net 64 - 2
HostMin = (net.num + 1), HostMax = (net.num + 64 - 2) and Broadcast address is (net.num + 64 - 1)
Hosts 192.168.0.129 - 192.168.0.190 Broadcast 192.168.0.191

The network and broadcast address are excluded, except in a 31-bit subnet mask, usually point-to-point links, only two hosts may be connected and a specification of network and broadcast addresses is not needed/possible.

You can use IPv6 addresses from the RFC 4193 block fd::/8
IPv6 is not in the scope of this article.

NOTE Validity of link-local addresses is limited to a single link; e.g. to all computers connected to a switch, or to one wireless network. Hosts on different sides of a network bridge are also on the same link, whereas hosts on different sides of a network router are on different links.

Use a IP Subnet Calculator link or link

Configuring Switches

Depending on type of managed switch you need to set them up.

• Level 2 switch can handle VLAN's and QoS information
• Level 2+ can also do some routing
• Level 3 does routing and more of the OSI level 3 stuff
• Proxmox has physical NICS (for ex. igb4, eno1, ens4, enp4s0 or enp2f0s1) and you can, if needed, connect them to a Bridge, Bond or VLAN. Both Linux Native and Open vSwitch can be used. Link
• Proxmox Bridges without a NIC are connected trough the internal Linux Virtual Switch to VM/CT's on the same machine. Link
• Proxmox 8.1 introduced the SDN for local internal networks