The Power of Cloud-Init and Templates

The secret of gaining high productivity in the home and enterprise is the power of Templates built with Cloud Images and controlled with Cloud-Init.

This post will demonstrate what you can do with just Proxmox and Cloud-Init.

Why a VM

Using a VM provides better isolation and security since it runs a separate operating system and kernel, making it less vulnerable to issues in other containers. In contrast, CT (LXC) containers share the host's kernel, which can lead to potential security risks.

Automation usually needs a Template, or it's the best-practice to use one.

The benefit of Cloud Images

A Cloud Images is a ready VM disk, It is very compact 3 G for the Debian one by default. It's easy to add apps and run initial installation scripts using Cloud-Init tools.

Any way you see it, it will save you a ton of time from playing with full installations. And the footprint is minimal out of the box.

Why Cloud Images instead of ISOs

The benefit of Cloud-Init snippet files

We create one basic Template but change it by editing the Vendor/User/Network file for adding new features for our clones or VMs (cp vendor.yml vendor.yml.orig then nano vendor.yml). Another way is to introduce totally new Vendor/User/Network file/files for a VM.

The Workflow

I do this now with Debian 13, one plain and one with Docker.
Next time to renew the base Templates is after that Debian 14 is released.

Set up face – one time only

1. Download a cloud image →
2. Create a VM →
3. Import the disk →
4. Configure your cloud-init →
5. Configure your vendor file →
6. Convert it to a template

Then on demand – many times

1. Clone the template →
2. Set IP/hostname →
3. Power it on →
4. SSH in

Why a Full Clone

Select the Template and make a clone, I usually use Full Clones because the new VM does not share any storage resources with the template. And, it's possible to select a Target Storage, so one can use this to migrate a VM to a totally different storage. You can also change the disk image format if the storage driver supports several formats.

I prefer to have my Templates on the local-zfs and all VM disks on VMdata.

Why you need to use a VM

They are essential for specific use cases

• Other than Linux kernels
  • Windows: no LXC support for Windows kernels
  • No LXC support for: macOS, FreeBSD, OpenBSD or a vintage Linux Kernel...
• Hardware Pass through
  • GPUs, USB controllers, NVMe drives, capture cards
• Security Boundaries
  • High Security Systems, customer isolation, multi-tenant, web facing systems
• Kernel Modules needed
  • Use of custom drivers, the WireGuard kernel mode, ZFS on custom Linux systems
• Production HA
  • Live Migration gives clusters with zero-downtime moves

Create a Template

The process is quite simple, download a ready-made cloud-image, create a VM, add stuff by the Vendor file and transfer it into a Template. The trick is not to start the VM before making it into a Template. The easiest way is to use the CLI and the qm command. QM stands for: QEMU/KVM Virtual Machine Manager, one of the core features of Proxmox.

QM uses 4 snippet-files to further expand our templates: meta, network, user and vendor.

Download a Cloud-Image

Depending on your use-case use a raw or a qcow2 image as the base. It's up to you to choose the distro, here we use Debian.

For ZFS, LVM-Thin, or Ceph

• Use RAW. Gives maximum speed (about 10% faster than QCOW2) and low overhead. Snapshots and backups are handled by the underlying Proxmox storage pool instead of the disk file.
• Using QCOW2 on top of ZFS causes heavy write slowdowns (double copy-on-write). LVM-thin storage only supports RAW anyway.

wget https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.raw

For Directory (Ext4/XFS) or NFS

• Use QCOW2 if you want the disk image to grow dynamically as you add data rather than instantly taking up full provisioned space.
• Includes built-in features like internal snapshots and dynamic resizing, but runs slower under heavy loads.

https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2

Create the initial Template

Create a base VM

Use your preferences and configuration for this stage, see the manual. This example will create a VM: 90000 using OS type Linux 7.x with a 2.6 Kernel, using vmbr0, booting from the scsi0, having the QEMU Guest Agent activated and called Trixie

qm create 90000 --memory 1024 --net0 virtio,bridge=vmbr0 --scsihw virtio-scsi-pci --ostype l26 --boot order=scsi0 --agent enabled=1 --ide2 local-zfs:cloudinit --boot order=scsi0 -serial0 socket --ipconfig0 ip=dhcp --name Trixie

Extract and Import the disk.

Edit the command to reflect your needs

Example using a local user and storage called VMdata

qm set 90000 --scsi0 local-lvm:0,import-from=/home/nalle/debian-13-generic-amd64.raw

Example using root and local-lvm

qm set 90000 --scsi0 VMdata:0,import-from=/root/debian-13-generic-amd64.raw

This will create a VM as specified, but it only has a 3 GiB Disk. It might be OK for small servers like nameservers, but you might need a bigger disk. At first, add 1G and check how it goes, if needed gradually expand the disk.

Change BIOS

As default, it is using the default SeaBIOS and i440fx.
If you want OVMF (UEFI) and q35 you should edit the Hardware tab accordingly.

Edit the VM in the GUI

You can add this to the qm command if you like.

Set the Hardware in the GUI Hardware tab

• Add the Cloud-Init Drive
• Add a serial port (for xterm.js) – OPTIONAL

Edit the Cloud-Init file in the GUI

If you don't want to or need the extended functionality of using a User.yml file.
Add the drive and fill in your credentials, SSH-Key and IPs.
I usually set to DHCP for templates, just in case.

• User
• Password
• SSH public key (you should use one)
• IP config
  • Set an IP address or select DHCP

Set the Options in the GUI

Change as needed according to your needs

Clone the basic VM – Option

Now clone the VM Trixie as TrixieDocker

Cloud-Init configuration files

In the /var/lib/vz/snippets/ directory we can store any of the needed files. The files are called by the Cloud-Init process at startup with cicustom: [meta=<file>] [,network=<file>] [,user=<file>] [,vendor=<file>] e.g., cicustom user=/var/lib/vz/snippets/nalle.yaml

Naming the file is critical for us to remember what they are for, later when we clone the template or when we need to edit one of the files because we want new features added.
If using the same file for several Templates the situation can be messy.

Use of Cloud-Init Specific snippet files

Combined with qm command the snippet files takes Proxmox to a higher level. Cloud-Init is not specific to Proxmox, it's originally developed by Canonical. It is used for doing the initial setup of virtual machines in cloud computing

These snippets will be read when a cloning occurs, meaning you can edit the file to enhance new cloned VMs as your needs evolve.

• meta – Specify a custom file containing all metadata passed to the VM
• network – To pass a custom file containing all network data to the VM
  Only needed for complex networking setups, network configuration v2 .
• user – To pass a custom file containing all user data to the VM
  We can use the inbuilt cloud-init for all basic needs.
• vendor – To pass a custom file containing all feature requests of the VM.
  The main tool to create our Templates to be tailored by editable files.

Create a vendor.yml snippet file

A Vendor file can be called anything but has to end with .yml or .yaml.
It doesn't need to be in the local storage, but I feel comfortable having them there.
Using Cloud-Images we can tailor them using a vendor.yml file, as they pre-configure system settings, repositories, or management tools required by the hosting platform.

The vendor.yml file is performed for the first boot of the VM clone.

We use user ID 1000 here, as that is what the cloud-image delivers for the first user.

Here we will create a Template called Trixie the Vendor file will be Trixie.yml.

nano /var/lib/vz/snippets/Trixie.yml

Example for a basic Trixie VM

#cloud-config

packages:
  - qemu-guest-agent
  - sudo
  - curl
  - ufw
  - fail2ban
  - htop
  - duf
  - inxi

runcmd:
  - ufw allow ssh
  - ufw enable
  - reboot

Example with Docker for a homelab VM

#cloud-config

packages:
  - qemu-guest-agent
  - curl
  - ufw
  - fail2ban
  - htop
  - duf
  - inxi


runcmd:
  - ufw allow ssh
  - ufw enable
  - curl -sSL https://get.docker.com/ | sh
  - sudo usermod -aG docker $(id -un 1000)
  - sudo init 6

Create a user.yml snippet file

Authentication

The SSH password login will be disabled, public key only.

Timezone

PVE has no native cloud-init timezone option; set it you can set it in this file.
Checkout the list of timezones: timedatectl list-timezones

#cloud-config
timezone: Europe/Berlin

# Create a non-root admin user with sudo access and SSH key authentication
users:
  - default
  - name: admin
    gecos: Cloud Administrator
    # cloud-init silently ignores groups that do not exist on the target distro
    groups: [sudo, docker]
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
    ssh_authorized_keys:
      - ssh-ed25519 AAAAC3N... [email protected]
# To disable password authentication on SSH (key-only)
ssh_pwauth: false

package_update: true
package_upgrade: true
packages:
  - qemu-guest-agent
  - curl
  - git
  - htop
  - vim

write_files:
  - path: /etc/motd
    content: |
      Provisioned by cloud-init via a Proxmox template
    owner: root:root
    permissions: "0644"  

final_message: |
  Cloud-init provisioning is complete.
  Version: $version
  Uptime: $uptime seconds

Adding a Password for local access

Create an SHA-512 hashed password

Add a line and use the output

...
passwd: "$6$ai.sS9FSmcAGoA.H$4jYb9KRT9WQb1Uhc7B7MVPM4QoLD7gU83xCjcpdX.UYlkHkzAmP8GEBupOEfnauXJ0FWWgProyNLFH6S1PB2U1" 
lock_passwd: false

# Force password change on first login – optional, remove to keep the passwd
chpasswd:
  expire: false
...

Apply to any VM

qm set 90006 --cicustom "user=local:snippets/nalle.yml"
qm cloudinit update 90006
qm start 90006

Create a network.yml snippet file

When to use a snippet files and when not to do it

For a simple static IP assignment, the PVE native ipconfig is sufficient, and do not require a snippet file.

qm set <vmid> --ipconfig0 ip=192.0.2.40/24,gw=192.0.2.1 \
--nameserver "192.0.2.53 192.0.2.54" --searchdomain example.com

Use a network snippet file when you need settings PVE's ipconfig does not expose like multiple search domains, custom routes, bonding, VLANs, etc.

Network configuration v2 (cloud-init network-data).

Configures a static IP on the first NIC, it replaces the PVE auto-generated network-data when applied via cicustom.

version: 2
ethernets:
  eth0:
    match:
      name: "en*"
    set-name: eth0
    addresses:
      - 192.0.2.40/24
    nameservers:
      addresses:
        - 192.0.2.53
        - 192.0.2.54
        - 1.1.1.1
      search:
        - example.com
    routes:
      - to: default
        via: 192.0.2.1

Apply to a VM:

qm set <vmid> --cicustom "network=local:snippets/ci-net.yml"
qm cloudinit update <vmid>
qm start <vmid>

Import some of it to the Templates – 3 examples

qm set 90000 --cicustom "vendor=local:snippets/Trixie.yml"

qm set 90001 --cicustom "vendor=local:snippets/TrixieDocker.yml"

qm set 90001 --cicustom "vendor=local:snippets/deb13.yml,user=local:snippets/nalle.yml"

Convert to Template

Go to the VM and right click, select Convert to template from the drop-down menu

That's all folks – we now have a Debian Trixie template for all our VMs to be.

Summary of Usage

So, to sum up all above, this is how to use the power of Proxmox qm and snippet files, ready-made Cloud Image disks and the Cloud-Init to create VMs on the go.

Create the Template

1 – Download the Cloud-Image

https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.raw

2 – Create a VM

qm create 90000 --memory 1024 --net0 virtio,bridge=vmbr0 --scsihw virtio-scsi-pci --ostype l26 --boot order=scsi0 --agent enabled=1 --ide2 local-zfs:cloudinit --boot order=scsi0 -serial0 socket --ipconfig0 ip=dhcp --name Trixie

3 – Import the disk

qm set 90000 --scsi0 VMdata:0,import-from=/root/debian-13-generic-amd64.raw

A 3 GiB Disk is fine for small servers like NAS, DNS, DHCP, NTP..., but for a Docker VM you need more space on the boot drive.
For a Desktop you should use a full installation from a ISO.

4 – Configure Cloud-Init in the GUI

Go to the Cloud-Init tab and set at least the following:
User, Password, SSH public key and IP Config (ip=dhcp)

Or use a Cloud-Init user snippet file.

5 – Configure and Import your vendor.yml file

nano /var/lib/vz/snippets/TrixieDocker.yml

And import it

qm set 90001 --cicustom "vendor=local:snippets/TrixieDocker.yml"

6 – Convert to a Template

Right click and hit Convert to template

Implementation

1 – Clone the Template to a new VM

Edit the Cloud-Init, change credentials and DHCP to fixed if needed. You can also set other details of the VM and deploy it by editing the vendor.yml file.

2 – Set up the new VM

Edit the Cloud-Init and/or Vendor file as needed

3 – Start the new cloned VM

The VM will boot, stop for a vile, continue the installation and finally perform a reboot.

References

Debian ^[1] About IPs ^[2] Proxmox qm ^[3] Clones ^[4] Cloud-Init ^[5]
Cloud-init is the industry standard multi-distribution method for cross-platform cloud instance initialisation. It is supported across all major public cloud providers, provisioning systems for private cloud infrastructure, and bare-metal installations.

During boot, cloud-init identifies the cloud it is running on and initialises the system accordingly. Cloud instances will automatically be provisioned during first boot with networking, storage, SSH keys, packages and various other system aspects already configured.

Cloud-init provides the necessary glue between launching a cloud instance and connecting to it so that it works as expected.