Skip to main content

Proxmox VM/LXC Deployment

The majority of our customers run the OmniCore stack on Proxmox. A single playbook — services/proxmox.yml — provisions both QEMU VMs and LXC containers based on a site-wide deployment_type toggle.

We still continue to support VMware, HyperV and cloud (Vultr / AWS / GCP) for deployments.

See Also:

LXC vs VM​

LXC Containers:

  • Lightweight, shares host kernel
  • Fast startup, low overhead
  • Limited isolation
  • Cannot run custom kernels or kernel modules
  • Not suitable for production EPC/IMS deployments
  • Cannot run UPF (requires kernel modules/TUN devices)

VMs (KVM):

  • Full virtualization with dedicated kernel
  • Complete isolation, runs kernel modules / custom networking
  • Higher resource overhead
  • Recommended for production
  • Required for UPF deployments

Use Cases:

  • VMs: production sites, UPF, all network functions
  • LXC: lab/test environments, lightweight services (apt-cache, monitoring, dns)

One Site = One Type (recommended)​

deployment_type is read per host from the proxmoxServers entry the round-robin selects for it, so nothing forces all entries in a site to agree. As a convention, keep all proxmoxServers.*.deployment_type entries the same and pick one type per site — mixing VM and LXC across nodes makes node placement depend on host index and is hard to reason about.

The playbook only hard-fails on type collisions with existing resources: if a resource with the target hostname already exists as the wrong type (e.g. an LXC exists but the selected server is deployment_type: vm, or vice versa), it aborts with a clear error. See Type Guards.

Proxmox Setup​

1. Create API Token​

# Proxmox UI: Datacenter → Permissions → API Tokens
# Create token: root@pam!<TokenName>
# Copy the token secret (shown once)

2a. Create Cloud-Init VM Template (VM sites only)​

Run this script on the Proxmox host. It downloads Ubuntu's cloud image and creates a clean template — cloud-init credentials and SSH keys are injected at clone time by proxmox.yml (see Notes below).

#!/bin/bash
set -e

TEMPLATE_ID=9000
IMAGE_URL="https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img"
IMAGE="noble-server-cloudimg-amd64.img"

cd /var/lib/vz/template/iso
wget -N "$IMAGE_URL"

qm destroy $TEMPLATE_ID --purge 2>/dev/null || true

qm create $TEMPLATE_ID --name ubuntu-2404-template --memory 2048 --cores 2 --net0 virtio,bridge=vmbr0
qm importdisk $TEMPLATE_ID $IMAGE local-lvm
qm set $TEMPLATE_ID --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-${TEMPLATE_ID}-disk-0
qm set $TEMPLATE_ID --ide2 local-lvm:cloudinit
qm set $TEMPLATE_ID --boot c --bootdisk scsi0
qm set $TEMPLATE_ID --vga std
qm set $TEMPLATE_ID --agent enabled=1
qm template $TEMPLATE_ID

Notes:

  • The template has no hardcoded users or passwords — proxmox.yml sets ciuser, cipassword, and sshkeys at clone time.
  • Cloud-init user precedence: proxmoxTemplateUser (on the matching proxmoxServers entry) → first local_users key.
  • Cloud-init password precedence: proxmoxTemplatePassword (on the matching proxmoxServers entry) → the cloud-init user (i.e., the value chosen above).
  • SSH keys are injected from every local_users.*.public_key entry (key-based auth is the intended login path).
  • Set proxmoxTemplateUser (and optionally proxmoxTemplatePassword) on each proxmoxServers entry whenever the template was prepared with a cloud-init user that is not the first local_users entry, so the right account is selected.
  • --vga std ensures the Proxmox web console works.
  • -N flag on wget only downloads if newer than the local copy.

Alternative: Manual Template from ISO​

If cloud images aren't available or you need a custom install:

Step 1: Create VM via Web UI

  • Create New VM → VM ID 9000, Name: ubuntu-2404-template
  • OS: Upload Ubuntu Server ISO or use existing ISO
  • System: Default (SCSI Controller: VirtIO SCSI)
  • Disks: 32GB, Bus: SCSI
  • CPU: 2 cores
  • Memory: 2048 MB
  • Network: VirtIO, bridge vmbr0
  • Start VM and install Ubuntu Server

Step 2: Inside VM - Clean and prepare

# Install cloud-init
sudo apt update
sudo apt install cloud-init qemu-guest-agent -y

# Clean machine-specific data
sudo cloud-init clean
sudo rm -f /etc/machine-id /var/lib/dbus/machine-id
sudo rm -f /etc/ssh/ssh_host_*
sudo truncate -s 0 /etc/hostname
sudo truncate -s 0 /etc/hosts

# Clear bash history and shutdown
history -c
sudo poweroff

Step 3: Add Cloud-Init and Convert to Template

  • Select VM → Hardware → Add → CloudInit Drive (select storage e.g., local-lvm)
  • Hardware → Options → QEMU Guest Agent → Enable
  • Right-click VM → Convert to Template
  • Do not set cloud-init user/password on the template — proxmox.yml injects these at clone time from proxmoxTemplateUser / proxmoxTemplatePassword (with local_users fallback for the username).

2b. Download LXC OS Template (LXC sites only)​

# On the Proxmox node shell:
pveam update
pveam download local ubuntu-24.04-standard_24.04-2_amd64.tar.zst

The resulting volume path (e.g. local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst) is what you configure as proxmoxLxcOsTemplate below.

Hosts File Configuration​

VM Deployment​

all:
  vars:
    # deployment_type omitted → defaults to "vm"
    proxmoxServers:
      pve-node-01:
        proxmoxServerAddress: 192.168.1.100
        proxmoxServerPort: 8006
        proxmoxApiTokenName: ansible
        proxmoxApiTokenSecret: "your-token-secret-uuid"
        proxmoxNodeName: pve-node-01
        proxmoxTemplateName: ubuntu-2404-template
        proxmoxTemplateId: 9000
        proxmoxTemplateUser: omnitouch        # optional cloud-init username; defaults to the first local_users key
        proxmoxTemplatePassword: omnitouch    # optional cloud-init password; defaults to the cloud-init username
        storage: local-lvm  # optional
      pve-node-02:
        # ... second node config

    local_users:
      admin_user:
        name: Admin User
        public_key: "ssh-rsa AAAA..."

mme:
  hosts:
    site-mme01:
      ansible_host: 192.168.1.10
  vars:
    proxmox_interface: vmbr0
    gateway: 192.168.1.1
    mask_cidr: 24  # optional, default 24
    vlanid: 100  # optional

LXC Deployment​

all:
  vars:
    proxmox_lxc_nameserver: "1.1.1.1"   # optional, default 1.1.1.1
    proxmoxServers:
      pve-node-01:
        deployment_type: lxc
        proxmoxServerAddress: 192.168.1.100
        proxmoxServerPort: 8006
        proxmoxApiTokenName: ansible
        proxmoxApiTokenSecret: "your-token-secret-uuid"
        proxmoxNodeName: pve-node-01
        proxmoxLxcOsTemplate: "local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst"
        proxmoxLxcDefaultStorage: local-lvm  # optional, rootfs fallback
      pve-node-02:
        deployment_type: lxc
        # ... same shape

    local_users:
      admin_user:
        name: Admin User
        public_key: "ssh-rsa AAAA..."

dns:
  hosts:
    site-dns01:
      ansible_host: 192.168.1.20
  vars:
    proxmox_interface: vmbr0
    gateway: 192.168.1.1
    mask_cidr: 24                        # optional, default 24
    vlanid: 100                         # optional
    proxmoxLxcCores: 2                  # optional override
    proxmoxLxcMemoryMb: 4096            # optional override
    proxmoxLxcDiskSizeGb: 30            # optional override
    proxmoxLxcRootFsStorageName: local-lvm  # optional, overrides server proxmoxLxcDefaultStorage

Usage​

Provision (VM or LXC)​

ansible-playbook -i hosts/Customer/site.yml services/proxmox.yml

Delete​

ansible-playbook -i hosts/Customer/site.yml services/proxmox_delete.yml

The delete playbook handles both VMs and LXCs automatically, regardless of deployment_type.

Playbook Behavior​

Type Guards​

  • If a VM with the target hostname exists but deployment_type: lxc → playbook fails with clear error.
  • Inverse case (LXC exists, site configured as VM) → same.
  • UPF group on an LXC site → warn-only (not a hard fail, but UPF will not function).

New Resource​

  • Round-robins across proxmoxServers entries by host index within group.
  • Queries Proxmox API for next free VMID.
  • VM: clones from proxmoxTemplateId, sets cloud-init user/password/SSH keys (cloud-init user = proxmoxTemplateUser from the server entry if set, otherwise the first local_users key; cloud-init password = proxmoxTemplatePassword if set, otherwise the cloud-init username), resizes scsi0, starts.
  • LXC: creates from proxmoxLxcOsTemplate, rootfs on proxmoxLxcRootFsStorageName (group) → proxmoxLxcDefaultStorage (server) → storage → local-lvm (final fallback), injects all local_users.*.public_key via ssh-public-keys, starts. Root password hardcoded to 0mn1Touch@. All LXCs are unprivileged (unprivileged: 1) with nesting enabled (features: nesting=1).

Existing Resource​

Both VM and LXC update paths diff current vs desired and apply changes:

  • Network config (bridge, VLAN tag, IP for LXC)
  • Cores / memory
  • Disk size (VM scsi0 / LXC rootfs)
  • Secondary NICs from secondary_ips Reboots the resource if network or resources changed.

Tagging​

Every resource gets tagged with its Ansible group names + site_name.

Distribution Across Nodes​

Same round-robin logic for VMs and LXCs:

mme01 → pve-node-01 (0 % 3 = 0)
mme02 → pve-node-02 (1 % 3 = 1)
mme03 → pve-node-03 (2 % 3 = 2)
mme04 → pve-node-01 (3 % 3 = 0)

Per-Host Overrides​

Any of the group-level LXC knobs (proxmoxLxcCores, proxmoxLxcMemoryMb, proxmoxLxcDiskSizeGb, proxmoxLxcRootFsStorageName, proxmoxLxcOsTemplate, host_vm_network) can also be set per-host under the hostname.

dns:
  hosts:
    site-dns01:
      ansible_host: 192.168.1.20
      proxmoxLxcDiskSizeGb: 120   # per-host override
      host_vm_network: vmbr1      # per-host bridge override
  vars:
    proxmox_interface: vmbr0
    gateway: 192.168.1.1
    mask_cidr: 24  # optional, default 24

Legacy util_playbooks/proxmox_lxc.yml​

Deprecated. It uses the same modern proxmoxServers dict (with proxmoxApiTokenName/proxmoxApiTokenSecret) as the current playbook, but it only provisions LXC containers (hosts: all:!upf) and is superseded by the unified services/proxmox.yml. All new sites should use services/proxmox.yml with deployment_type: lxc.

Last updated on