pve_cpi job from bosh-proxmox-cpi/0.5.0

BOSH Cloud Provider Interface for Proxmox VE

Github source: 9b2f4f8 or master branch

Properties¶

`agent`¶

blobstore¶

Optional default blobstore for settings.json (agent.blobstore in bosh.yml)
Default
{}
mbus¶

URL the BOSH agent should bind/listen on inside the VM (the same value cloud_provider.properties.agent.mbus carries in bosh-deployment). Required for create-env: bosh-init does not pass it via the per-call env argument, only via CPI config.
Default
""

`pve`¶

agent_mode¶

Agent bootstrap mode. Valid values: cloudinit, noagent, auto. “cloudinit” — uses cloud-init configdrive (default). “noagent” — no agent bootstrap; assumes DNS/manual setup. “auto” — selects configdrive (cloudinit) for all stemcells. Equivalent to cloudinit; retained for manifest compatibility. Omitted from rendered cpi.json when default “cloudinit” is used.
Default
cloudinit
allow_disk_ops_with_snapshots¶

When true, bypasses the snapshot pre-flight guard in attach_disk, detach_disk, and resize_disk. Use only for emergency disk recovery; snapshot state will be inconsistent after the operation. Default false (guard active).
Default
false
antiaffinity_verify¶

When true, after recreating an anti-affinity rule the CPI re-lists the HA rules and asserts the target VM is present in the rule’s members. A concurrent writer that dropped the member surfaces as a retriable error rather than a silent loss of spread. When false (default), no check runs and behavior is identical to prior releases.
Default
false
api_dial_timeout_sec¶

Optional. Bounds the TCP dial step of every PVE API HTTP request. 0 (default) leaves the transport at the SDK default (no explicit dial timeout). When set, the TCP dial is cancelled if it has not completed within this many seconds. Useful on high-latency or flaky networks to surface connection failures faster. Must be >= 0.
Default
0
api_idle_conn_timeout_sec¶

Optional. How long an idle keep-alive PVE API connection remains in the transport pool before being closed, in seconds. 0 (default) selects the CPI default of 15, which retires idle connections before pveproxy’s shorter keep-alive window closes them server-side (reusing a server-closed connection races the close and surfaces as a dropped request). Set 90 for the previous SDK behavior. Shorter values free sockets sooner on clusters with infrequent CPI activity; longer values retain warmed connections across calls. Must be >= 0.
Default
0
api_max_idle_conns_per_host¶

Optional. Maximum number of idle (keep-alive) connections retained in the transport pool per PVE host. 0 (default) falls back to the SDK default (KeepAlive value). Higher values reduce connection-setup latency under burst load; lower values conserve file descriptors on constrained CPI hosts. Must be >= 0.
Default
0
api_tcp_keepalive_sec¶

Optional. TCP keep-alive probe interval for PVE API connections, in seconds. 0 (default) leaves the transport at the Go default. A positive value enables periodic TCP keep-alive probes at this interval, which helps detect silently-dropped connections on stateful firewalls between the CPI host and the PVE API endpoint. Must be >= 0.
Default
0
api_tls_handshake_timeout_sec¶

Optional. Bounds the TLS handshake step of every PVE API HTTPS request. 0 (default) leaves the transport at the SDK default (no explicit handshake timeout). When set, the handshake is cancelled if it has not completed within this many seconds. Must be >= 0.
Default
0
api_token¶

PVE API token (mutually exclusive with password). Format: !=. Must be credhub-managed in production: reference via ((pve_api_token)) against a credhub variable typed password. Empty string when password is used instead.
Default
""
balloon¶

Memory-balloon setting written to every new VM’s PVE “balloon” config key. Default “” (empty) resolves to “0”: the balloon device is disabled. BOSH sizes VMs deterministically from the manifest and the agent plans job memory against that size, so PVE’s default auto-ballooning — which reclaims guest memory beneath those assumptions — invites OOM kills that look like application failures.

Set a positive integer (MiB) to enable PVE auto-ballooning with that floor; the CPI fails fast when the value exceeds the VM’s memory. Set the sentinel “pve-default” to leave no balloon key on the VM — on clones the CPI clears the template-inherited value — restoring PVE’s own default (device enabled, balloon = memory) for clusters that deliberately overcommit memory.

Precedence (highest wins): cloud_properties.balloon (per-instance-group, resolved through the same call/disk_type/vm_type layered resolver as other create_vm knobs; the “pve-default” sentinel works there too) > this global value > built-in default “0”. cloud_properties.pve_config.balloon is rejected — this knob is the supported surface. Applies on VM creation; existing VMs keep their balloon setting until recreated.
Default
""
ca_cert¶

Optional. PEM-encoded CA certificate bundle for verifying the Proxmox VE API TLS certificate. When empty (default), the system trust pool is used unmodified — behavior is byte-identical to prior releases. When set, the PEM is parsed and the resulting cert pool replaces the system pool for PVE API HTTPS verification. Ignored when verify_ssl is false.
Default
""
clone_mode¶

Controls the clone type used when create_vm clones a stemcell template. A linked clone’s overlay volume always lands on the TEMPLATE’s own storage pool (PVE does not honor a Storage override on linked clones) — never on vm_storage; only a full clone can be placed on vm_storage. “auto” (default when unset): linked clone when the template’s storage supports it (all backends except lvm-thick) AND vm_storage is the same pool as the template’s storage (stemcell_storage); full clone otherwise, including whenever vm_storage differs from the template’s storage, so the root disk always lands where vm_storage points rather than silently landing on stemcell_storage. “linked”: force linked clone; returns an error if the template’s storage does not support linked clones (e.g. lvm-thick), or if vm_storage differs from the template’s storage (which would silently misplace the disk) — set clone_mode to auto or full, or align stemcell_storage/vm_storage, to resolve. “full”: force full clone on all backends. One of auto|linked|full.
Default
""
cluster_lock_mode¶

Selects the cross-process cluster mutex used to serialize the read-modify-write on a shared HA anti-affinity rule. Two concurrent create_vm invocations for the same instance group both read the old member set and recreate the rule (PVE rules have no partial edit); the last writer wins, silently dropping a member and losing the spread guarantee. When “pool”, the CPI acquires a sentinel resource pool (POST /pools is pmxcfs-serialized, create-or-fail) keyed on the group name around that read-modify-write. When empty or “off” (default), no lock is taken and behavior is identical to prior releases. Valid values: “” (empty, same as “off”), “off”, “pool”.
Default
""
cluster_lock_timeout_sec¶

Bounds how long the anti-affinity read-modify-write waits to acquire the cluster lock before returning a retriable error (the BOSH director then re-drives the operation). It also serves as the lock’s TTL: a holder whose recorded expiry has passed is treated as crashed and its lock is stolen. Only meaningful when cluster_lock_mode is “pool”; 0 resolves to 60 seconds.
Default
0
cpu_type¶

Emulated CPU type/model written to every new VM’s PVE “cpu” config key (e.g. “host”, “x86-64-v2-AES”, “Skylake-Server-noTSX-IBRS”). Default “” (empty) resolves to “host”: the guest sees the physical CPU’s full feature set, the best-performing choice, and safe on the homogeneous clusters typical of BOSH deployments. Set the sentinel “pve-default” to restore the legacy behavior: the CPI writes no cpu key at all and PVE falls back to kvm64.

WARNING — heterogeneous clusters: a “host”-typed guest can crash when live-migrated to a node with a different CPU generation. If your cluster mixes CPU models and relies on live migration (HA, DLB, maintenance evacuations), override this with a portable named model. “x86-64-v2-AES” (PVE’s own create-wizard default since 8.0) keeps AES-NI and live-migrates across CPU generations from roughly 2010 onward; for hardware older than that baseline, use the cluster’s lowest-common-denominator named model.

Precedence (highest wins): cloud_properties.cpu_type (per-instance-group, resolved through the same call/disk_type/vm_type layered resolver as other create_vm knobs; the “pve-default” sentinel works there too) > this global value > built-in default “host”. cloud_properties.pve_config.cpu is a separate raw escape hatch applied after VM creation in the same create_vm call — when both this and pve_config.cpu are set, pve_config.cpu wins as the final write (the CPI logs an Info line pointing at this knob when pve_config.cpu is used).

Existing VMs keep their CPU type until recreated; a changed value (or this release’s new default) applies on the next VM creation, e.g. “bosh deploy –recreate”. PVE validates the model name itself; the CPI passes the value through verbatim with no additional validation.
Default
""
create_env_deployment¶

Synthetic deployment name used for VMs created by bosh create-env. bosh-init does not pass a deployment in env, so a stable placeholder is required for the “” segment of the VM name. Defaults to “create-env” so the director comes online as “-create-env--0”.
Default
create-env
debug¶
keep_failed_vms¶

Diagnostic mode. When true, a VM that fails mid-creation is NOT destroyed. Instead it is tagged “bosh-create-failed” (plus the deployment/job derived from the BOSH env) and create_vm returns an error naming the VMID and node, leaving the VM intact for post-mortem. This deliberately breaks the normal no-orphan guarantee, so it is for debugging only. Default false (opt-in).
Default
false
destroy_unreferenced_disks¶

When true, delete_vm passes DestroyUnreferencedDisks=true to PVE’s destroy call on every non-retain delete (the synchronous path, the fast path, and the fast-path straggler sweep). PVE’s own semantics: free every volume on the VM’s storages that is (a) not referenced in the destroyed VM’s config and (b) has a VMID matching the VM being destroyed – a storage-wide scan by VMID, not scoped to this VM’s config.

Default false. On storage dedicated to a single PVE cluster this is safe and sweeps up orphaned own-VMID volumes (e.g. a disk left behind by an interrupted create) that the config-scoped guards never touch. It is UNSAFE the moment pve.vm_storage/disk_storage/iso_storage is shared with a second, independent PVE cluster (a second BOSH-Proxmox AZ pointed at the same NFS/dir export): the destroyed VM’s VMID band can overlap the other cluster’s VMID band, and this flag would then free the OTHER cluster’s live disks – from this cluster’s view they are unreferenced and VMID-matching, identical to a genuine orphan. Nothing in the delete path can tell the two cases apart.

Enable only when the configured storage pools are NOT shared with any other independent PVE cluster or non-CPI tooling allocating VMIDs in the same range. On shared storage, leave this false and rely on disjoint per-CPI VMID banding instead – orphaned own-cluster volumes then accumulate and are visible to scripts/disk-audit rather than being swept automatically.
Default
false
detached_disk_strategy¶

Lifecycle strategy for persistent disks in the detached state (between detach_disk and the next attach_disk or delete_disk). One of: “” — same as “parked” (the default). “parked” — detached disks are attached to a dedicated parker VM (bosh-parker-) in an active scsi slot (scsi0-30) with protection=1 and onboot=0. The parker VM is never started. Provides PVE-side ownership visibility and accident protection. The CPI defaults pve.parked_disk_vmid_range_start and pve.parked_disk_vmid_range_end to ⁹⁰⁰⁰⁰⁄₉₀₉₉₉ if unset, each bound independently. The parker VMID band must not overlap the VM range, the persistent-disk range, or the stemcell-template range. A deployment that asks for parking by name, or that names its own parker band, fails to load on overlap. A deployment that sets neither and whose other bands reach into 90000-90999 keeps the pre-parking behavior for new detaches on that load and warns (the band stays in force read-only, so disks parked earlier still drain), so an upgrade cannot take a working deployment down: set a free parker band to turn parking on there. Costs a few extra PVE API calls per detach and attach. “free” — opt out of parking: detached disks float as unattached volumes in their synthetic VMID container. PVE has no first-class volume object; the disk is visible only via its container VM config. Risk: administrators may mistake the container VM for an unused resource and delete it, destroying the disk with it. This was the default before parking became the default. The parker band still resolves (⁹⁰⁰⁰⁰⁄₉₀₉₉₉ unless set), so a disk parked while “parked” was in effect is recognized and unparked on its next attach_disk or delete_disk — switching to “free” only stops NEW detaches from parking. delete_vm and snapshot_disk still refuse a parker VM by tag.
Default
""
disk_cid_compression¶

Retained for compatibility; the behavior it opted into is now always on. A disk CID whose standard pvd- envelope would exceed 255 characters is emitted as pvz- so it fits the varchar(255) disk_cid column of MySQL-backed Directors (and the dynamic_disks table on all database backends). CIDs that fit 255 characters stay pvd- and byte-identical, and the CPI decodes every previously emitted format unconditionally, so no disk migration is ever needed.
Default
false
disk_delete_state_guard¶

Whether delete_disk first checks the lock state of the VM the target disk is attached to before deleting it. One of “off” or “on” (the default). When “on” (default), delete_disk scans the cluster for the VM whose config currently references the volume (the VMID baked into the volume name is only an allocation placeholder, so the attached VM is resolved by config, not by name) and, if that VM holds a destructive or in-flight config lock (backup, clone, migrate, snapshot, rollback, or create), defers the delete with a retriable error so the BOSH Director re-drives it once the operation completes — closing the window in which an imgdel could otherwise race a nightly vzdump/PBS backup, a clone, or a migration of the same disk. The guard is best-effort: a disk attached to no VM (the normal pre-delete state) is allowed straight through, and any attachment-resolution uncertainty fails open so a guard hiccup never blocks a legitimate delete — the worst case of leaving this on its default is a delayed delete during a backup window, never a hard failure. Set “off” to restore the earlier unguarded behaviour (no attachment lookup, byte-identical to releases before this default changed). Operator note: if the attached VM is left with a stuck config lock, delete_disk will keep deferring; clear it with “qm unlock ” on that VM’s node.
Default
""
disk_migration¶

Cross-node persistent-disk migration on attach. One of: “” — same as “on_attach” (the default). “on_attach” — when a stable-ID disk sits on one node and the target VM runs on another, attach_disk moves the disk instead of erroring: the disk is isolated onto a fresh single-purpose mover parker VM on its own node (a metadata-only reassignment, so sibling parked disks never travel), the never-started mover is offline-migrated to the VM’s node through the PVE migrate API (a metadata move on shared storage, a volume copy on node-local storage), the disk attaches from the mover, and the now-empty mover is destroyed through a guard that refuses to destroy a mover still referencing any volume. “off” — restore the hard cross-node errors; attach_disk names this property in the refusal. Legacy disks (created before stable disk identities) are never migrated under either setting: the migration renames the volume and a legacy CID is the volume name. Disks on an offline node cannot migrate either; PVE needs the source node online to run the migration task. Overridable per cpi-config entry as pve_disk_migration.
Default
""
disk_perf_invariant_mode¶

Controls enforcement of creation-time disk-performance invariants at attach_disk time. The structural options cache, iothread, and ssd are baked into the disk CID at create_disk time; on re-attach the CPI merges global diskperformance defaults over the recorded options. If global config has since introduced a structural option the disk did not have at creation, the disk’s runtime profile would silently diverge from its recorded one. This knob governs that case: enforce (default) - reject the attach with a non-retriable error warn - log the divergence and proceed with the merged options off - skip the check entirely Throttle options (mbps, iops_) and discard are never enforced. The check is a no-op for any disk whose CID carries no performance options, so behavior is unchanged unless disk_performance options were recorded. Options an operator updates through update_disk are recorded per disk and join the expected baseline, so an intentional update never reads as a divergence; only drift the disk never opted into trips the guard. Valid values: “” (empty, same as “enforce”), “enforce”, “warn”, “off”.
Default
""
disk_performance¶

aio¶

Default PVE AsyncIO backend applied to every disk created by this CPI. Valid values: native, io_uring, threads. Overridden per disk by cloud_properties.aio. When unset (default), the key is omitted and PVE uses its own default (io_uring on modern PVE hosts).

io_uring is PVE’s modern default and generally the best choice on a current kernel. native (Linux AIO, kernel io_submit) pairs with cache=none on block-backed storage (lvmthin, zfspool, rbd, lvm) for the lowest-overhead path to raw/thin-provisioned block devices — the classic “aio=native + cache=none” combination for high-IOPS workloads on block-native pools. threads uses a userspace thread pool and is the safest fallback for file-backed storage (dir, nfs, cifs) or older kernels where io_uring is unavailable or unstable.

Structural option: baked into the disk’s PVE volid options at create time (create_disk, create_vm’s root disk) and governed by disk_perf_invariant_mode on re-attach, same as cache/iothread/ssd — a disk whose creation-time record diverges from a later re-resolution is caught by that guard (reject in enforce mode, warn in warn mode, ignore in off mode).

cache¶

Default PVE disk cache mode applied to every disk created by this CPI. Valid values: none, writethrough, writeback, unsafe, directsync. “writeback” offers best throughput for most workloads; “none” is required for data safety on Ceph/RBD. Overridden per disk by cloud_properties.cache. When unset (default), the key is omitted and PVE uses its own default.

discard¶

Tri-state (true|false|unset, default unset = auto). Controls discard=on (TRIM/UNMAP passthrough to the underlying storage) on every disk created by this CPI. Overridden per disk by cloud_properties.discard, which accepts the same three states.

unset (auto, the default) — discard is resolved per disk at bake time (create_disk/create_vm/attach_disk) based on the disk’s actual resolved storage pool: “on” when the pool is TRIM-capable — lvmthin, zfspool, rbd (any format), or a file-backed pool (dir/nfs/cifs) with a qcow2 disk image — and omitted otherwise (thick lvm, cephfs, glusterfs, or any other backend where TRIM does not reclaim space). This matters because guest-deleted blocks are never reclaimed without discard on a thin pool, so an unmanaged thin pool grows monotonically until full. true — forces discard=on regardless of the resolved pool’s TRIM capability; PVE is left to accept or reject the value. false — always omits the key, restoring the pre-auto behavior (PVE uses its own default) on every disk, TRIM-capable or not.

An explicit true/false at any layer (cloud_properties, disk_type/ vm_type profile, or this global default) always wins over auto resolution. The resolved value is baked at create/attach time only; config changes never rewrite existing disks or VMs on their own, and a disk whose recorded creation-time value diverges from a later re-resolution follows the normal cloud_properties/pve_config precedence — discard itself is not governed by disk_perf_invariant_mode (see that property; it never was, even before auto-resolution — discard can change on a live device without a structural reconfiguration). An operator update through update_disk is recorded per disk and wins over both layers at every later attach.

iops_rd¶

Default read IOPS cap for every disk created by this CPI. Zero or unset means no cap. Must be a non-negative integer. Overridden per disk by cloud_properties.iops_rd. When unset (default), the key is omitted and PVE imposes no IOPS limit.

iops_wr¶

Default write IOPS cap for every disk created by this CPI. Zero or unset means no cap. Must be a non-negative integer. Overridden per disk by cloud_properties.iops_wr. When unset (default), the key is omitted and PVE imposes no IOPS limit.

iothread¶

When true, enables the PVE iothread option on every disk created by this CPI. One I/O thread per disk reduces contention on the QEMU main loop. Applicable to virtio-scsi-single and virtio-blk buses; ignored on ide/sata buses by PVE. Overridden per disk by cloud_properties.iothread.

Default true: a dedicated I/O thread per disk relieves QEMU main-loop contention on multi-disk BOSH VMs and matches the modern PVE creation default; leaving iothread off serializes every disk’s I/O behind the shared main loop. Set false (globally or per disk via cloud_properties.iothread) to restore the earlier off-by-default behavior. This is a create/attach-time bake only: existing VMs and previously baked disk CIDs are unaffected — the recorded structural options on an existing disk CID are never silently rewritten to the new default; a disk whose creation-time record diverges from the newly-resolved default at the next attach is instead governed by disk_perf_invariant_mode (reject in enforce mode, warn in warn mode, ignore in off mode), same as any other disk_performance configuration change. A deliberate per-disk change is still available at any time through update_disk, which records the update so it survives detach/attach cycles.

mbps_rd¶

Default read throughput cap in MB/s for every disk created by this CPI. Zero or unset means no cap. Fractional values are accepted (e.g. 100.5). Overridden per disk by cloud_properties.mbps_rd. When unset (default), the key is omitted and PVE imposes no throughput limit.

mbps_wr¶

Default write throughput cap in MB/s for every disk created by this CPI. Zero or unset means no cap. Fractional values are accepted. Overridden per disk by cloud_properties.mbps_wr. When unset (default), the key is omitted and PVE imposes no throughput limit.

ssd¶

Tri-state (true|false|unset, default unset = auto), same auto resolution as pve.disk_performance.discard — see that property for the full TRIM-capability matrix and precedence rules. When true (explicit or auto-resolved), marks the disk as SSD-backed (rotation=0 in the guest). Overridden per disk by cloud_properties.ssd, which accepts the same three states.

ssd only ever reaches a disk on the scsi bus: the pre-existing virtio-blk bus filter drops it from the VM root disk unconditionally, regardless of how the value resolved (explicit or auto). Unlike discard, ssd IS governed by disk_perf_invariant_mode on re-attach — a disk whose creation-time record diverges from a later resolution (e.g. a legacy disk created before auto-resolution existed, now auto-resolving ssd=1 on a TRIM-capable pool) is caught by that guard exactly as any other structural option divergence is.

virtio_scsi_single¶

When true, sets the SCSI controller to virtio-scsi-single mode for every VM created by this CPI, giving each disk its own dedicated virtio-scsi controller. Required to use iothread per disk on virtio-scsi. Overridden per VM by cloud_properties.virtio_scsi_single.

Default true: a dedicated controller per disk avoids serializing every persistent disk behind one shared virtio-scsi-pci controller, and matches the modern PVE creation default. Set false (globally or per VM via cloud_properties.virtio_scsi_single) to restore the earlier virtio-scsi-pci default. Create-time only: an existing VM’s controller is never changed retroactively — this only affects new VMs created after the default takes effect. The root disk itself stays on the virtio-blk bus regardless of this setting; moving the root disk onto the SCSI bus is a separate, opt-in change tracked independently of this property.

disk_storage¶

Storage pool for persistent disks

disk_types¶

Optional map of operator-named disk-type profiles. Each key is a profile name (e.g. “ssd”, “archive”) and each value is an object whose cloud_properties key holds default cloud_properties for that profile. A disk is associated with a profile by setting cloud_properties.disk_type to the profile name at deploy time. Disk-type profiles take precedence over vm_type profiles when both define the same attribute. When unset (default {}), no profiles are defined and no behavior change occurs.
Default
{}
disk_vmid_range_end¶

Inclusive upper bound of the VMID range for persistent-disk containers. When unset (0), defaults to 29999. Must be > disk_vmid_range_start.
Default
0
disk_vmid_range_start¶

Starting VMID for persistent-disk container allocation. Disk containers are allocated in [disk_vmid_range_start, disk_vmid_range_end]. When unset (0), defaults to 9000. Must not overlap the VM range or the template range.
Default
0
encrypted¶

Global opt-in for encrypted-storage disk placement. When true, create_disk and ephemeral disk creation restrict storage-tier selection to tiers that have encrypted: true set in the pve.storage_tiers map. A per-call cloud_properties.encrypted overrides this global value (per-call > global). When false or absent (default), no encrypted filter is applied and behavior is byte-identical to prior releases. The CPI cannot verify that a pool is actually encrypted — marking a tier encrypted is operator responsibility; a warning is logged whenever an encrypted tier is selected.

ensure_no_ip_conflicts¶

When true (default), create_vm scans the whole cluster’s VMs and checks that none already holds the requested static IP before provisioning. Prevents duplicate-IP collisions on static networks. Set to false only for dynamic (DHCP) networks where IP pre-assignment is not meaningful. Protective default: true. Omitting this property preserves the default.
Default
true
ephemeral_disk_min_mode¶

Action when the ephemeral_disk_min_ratio invariant is violated: “enforce” (the default) rejects create_vm with a non-retriable error naming the deficit; “warn” logs the deficit and proceeds. No effect unless ephemeral_disk_min_ratio is set. One of enforce|warn; empty resolves to enforce.
Default
""
ephemeral_disk_min_ratio¶

Minimum size floor, as a multiple of VM RAM, for a dedicated ephemeral disk created by create_vm (when ephemeral_disk_size_mb is set). The BOSH agent lays a RAM-sized swap file plus /var/vcap/data on the ephemeral disk, so an ephemeral disk smaller than ratio × RAM cannot satisfy the agent’s own layout and the job fails to come up at boot. When set, create_vm asserts ephemeral_GiB >= ratio × (RAM_MiB / 1024) and, on violation, rejects (or warns — see ephemeral_disk_min_mode). 0 (the default) disables the check entirely, leaving create_vm byte-identical to prior releases. The check is also skipped when no dedicated ephemeral disk is requested (the agent then carves ephemeral storage from the grown root disk). A conventional value is 2.
Default
0
external_command¶
allowlist¶

Allowlist of absolute executable paths permitted to run. Must be non-empty and contain external_command.command. An empty allowlist makes the hook inert (it never executes anything).
Default
[]
args¶

Arguments passed verbatim to the command as discrete argv (no shell interpretation). The CPI also injects CPI_METHOD and CPI_VMID into the scrubbed environment.
Default
[]
command¶

Absolute path of the executable to run. Must also appear in external_command.allowlist. Required when “external_command” is in pve.hooks. The command runs with no shell, a scrubbed environment, and a timeout; arguments are passed verbatim as discrete argv.
Default
""
env_passlist¶

Names of environment variables passed through from the CPI process to the child. Everything else is scrubbed; the child does not inherit the CPI environment.
Default
[]
methods¶

CPI methods that trigger the command. Empty (default) runs it on create_vm and delete_vm.
Default
[]
timeout_ms¶

Per-invocation timeout in milliseconds. Zero (default) applies a built-in 30 s default. The child process is killed when the deadline passes.
Default
0
fast_path_delete¶

When true, delete_vm and delete_disk issue the PVE destroy call and return immediately without awaiting the task’s terminal state. delete_vm additionally stamps a “bosh-deleting” tag on the VM before issuing the destroy; each subsequent fast-path delete_vm sweeps the cluster for VMs still carrying that tag and re-issues the destroy, so a stalled async destroy is reaped automatically. Disk volumes cannot carry PVE tags, so no marker is applied there. This eliminates the queue-slot hazard of a hung destroy-task poll at the cost of eventual consistency: a subsequent has_vm or has_disk call may briefly still see the resource until PVE’s async destroy completes. Default false (synchronous, fully-consistent behavior unchanged).

The fast-path destroy issues DeleteQemu with skiplock=true so a locked or still-running VM is destroyed without a separate unlock/stop step first. PVE honors skiplock=true only for the literal root@pam superuser (password auth) — not for any API token, including one owned by root@pam, and not for any other authenticated identity regardless of granted privileges or role. Enabling fast_path_delete under any other identity means the skiplock destroy call falls back to PVE’s own rejection whenever it actually hits a locked or running VM; the CPI logs a startup Warn naming the configured identity when this combination is detected, but does not block config load over it — the operator may know the identity ultimately resolves to root@pam through a proxy layer this check cannot see.

fetch_credential_defaults¶

Ordered list of URL-prefix to auth-payload mappings used by the CPI when fetching a light stemcell whose cloud_properties.image_url is set without per-stemcell image_url_auth. The entry whose url_prefix is the longest matching prefix of the request URL wins. Each entry requires two fields: url_prefix (string) and auth (object with a required type field). Supported auth.type values: basic, bearer, s3, oci, blobstore. When unset, light-stemcell fetches without per-stemcell credentials are unauthenticated.
Default
[]
Example
- auth:
    password: s3cr3t
    type: basic
    username: robot
  url_prefix: https://artifactory.corp/
- auth:
    access_key_id: AKIA...
    endpoint: https://s3.lab.local
    secret_access_key: '...'
    type: s3
  url_prefix: s3://stemcells-mirror/
health_check¶
enabled¶

When true, create_vm polls the guest QEMU agent via ping after the start task completes, waiting until the agent responds or the deadline (health_check.timeout_sec) expires. On timeout, VM status diagnostics are folded into the error before the standard rollback runs. Default false (opt-in): no polling occurs and create_vm returns as soon as PVE reports the start task complete.
Default
false
expected_agent_sha256¶

Expected SHA-256 hex digest of the BOSH agent binary (/var/vcap/bosh/bin/bosh-agent) inside the booted VM. When non-empty and health_check.enabled is true, create_vm runs sha256sum via the QEMU guest agent after the ping succeeds and fails (destroying the VM) only on a confirmed digest mismatch; any inability to verify (guest-agent error, non-zero exit, unparseable output) is fail-open. Must be 64 hex characters when set. Empty (default) disables the assertion.
Default
""
interval_sec¶

Seconds between successive agent ping attempts. Valid range 0-3600 when health_check.enabled is true. Zero (default) applies the built-in 5 s default. Set to 0 for back-to-back pings (fast test mode). Ignored when health_check.enabled is false.
Default
0
timeout_sec¶

Maximum seconds to wait for the QEMU guest agent to respond after VM start. Valid range 1-3600 when health_check.enabled is true. Zero (default) applies the built-in 300 s default. Ignored when health_check.enabled is false.
Default
0
hooks¶

Optional list of built-in dispatch middleware hooks to activate, applied in listed order. Each entry must be a known hook name; an unknown name fails CPI startup. Built-in hooks: “audit_log” (logs each CPI call’s duration and outcome, never argument content); “notes_audit” (writes the BOSH deploy identity into the VM’s PVE Notes after create_vm); “lb_register” (registers/deregisters the VM in an HAProxy backend via the Data Plane API on create_vm/delete_vm — requires the lb_register block); “external_command” (runs an allowlisted host command on selected methods — requires the external_command block). When empty (default), no middleware runs and there is zero per-call overhead. Example: [audit_log]
Default
[]
host¶

PVE host

hotplug¶

PVE hotplug flags applied to every new VM. Comma-separated list of “network,disk,cpu,memory,usb,cloudinit”; “0” disables hotplug entirely. Defaults to “network,disk,cpu,memory” so CPU + RAM can be resized live via qm set without rebooting. Per-VM override via cloud_properties.hotplug for stemcells that misbehave on memory hot-add.
Default
network,disk,cpu,memory
ip_conflict_probe¶

Selects the active IP-conflict probe mode. When empty or “off” (default), no active probe runs and behavior is identical to prior releases. When “agent”, create_vm additionally calls the QEMU guest agent on each running VM to collect dynamically assigned IP addresses and checks them against the target IPs before provisioning. This detects DHCP-assigned addresses that the static-config scan (ensure_no_ip_conflicts) cannot see. The probe is fail-open: a guest agent error is logged and that guest is skipped, never blocking provisioning. Valid values: “” (empty, same as “off”), “off”, “agent”. Only meaningful when ensure_no_ip_conflicts is true or unset.
Default
""
iso_storage¶

Storage pool (dir/nfs/cifs with iso content enabled) used to hold the per-VM ConfigDrive ISO. Block storages (lvm/lvmthin/zfspool) cannot hold ISO files. The ISO is attached as a CD-ROM on scsi30 for the VM’s whole life, not only at boot: PVE refuses to live-migrate a VM whose CD-ROM volume sits on non-shared storage, and HA recovery on another node fails at start because the ISO file does not exist there. This silently defeats placement.dlb, placement.pin_az_via_ha_rules, and placement.anti_affinity.use_ha_rules. Use a shared pool (rbd, nfs, cifs, glusterfs, cephfs) whenever any of those features is active; see pve.require_shared_iso_for_ha and pve.iso_storage_follow_vm_storage.
Default
local
iso_storage_follow_vm_storage¶

When true (the default), resolves the ConfigDrive ISO pool to pve.vm_storage instead of the pve.iso_storage default, provided vm_storage advertises PVE content type iso and is shared. Evaluated once at CPI process startup, before pve.iso_storage is otherwise applied. Because BOSH renders the “local” spec default for iso_storage whether or not the operator set it in the manifest, the CPI cannot distinguish “unset” from “explicitly local” — this flag therefore treats iso_storage resolving to the literal value “local” as the “unset” signal. An operator who deliberately sets iso_storage: local while this flag is active (its default state) gets vm_storage-following behavior instead of a literal local pool; set iso_storage to any other value to pin a literal pool this flag will never override. When vm_storage lacks iso content, is not shared, or cannot be resolved, the CPI falls back to iso_storage unchanged and logs a warning (fail-open). Set explicitly to false to disable and always use iso_storage as configured. Leave unset (~) to get the default-true behavior; explicit true/false always overrides it.

lb_register¶
allow_private_ip¶

When false (default), an endpoint resolving to a private or loopback address is rejected (SSRF guard). Set true only for a Data Plane API on a trusted private network.
Default
false
backend¶

HAProxy backend name that VMs are added to and removed from. Required when “lb_register” is listed in pve.hooks.
Default
""
ca_cert¶

Optional PEM-encoded CA certificate (or chain) that pins the HAProxy Data Plane API server certificate. When empty, the system trust store is used.
Default
""
endpoint¶

HAProxy Data Plane API base URL (e.g. https://lb.example:5555). Required when “lb_register” is listed in pve.hooks. Registration is best-effort: a Data Plane API failure is logged and never fails the CPI call.
Default
""
password¶

HAProxy Data Plane API basic-auth password.
Default
""
port¶

Server port registered for each VM in the HAProxy backend (e.g. the Cloud Foundry router port). Default 0 leaves the port unset on the server entry.
Default
0
timeout_ms¶

Per-call timeout in milliseconds for HAProxy Data Plane API requests. Zero (default) applies a built-in 10 s default.
Default
0
user¶

HAProxy Data Plane API basic-auth username.
Default
""
log_level¶

Log level
Default
info
max_inflight_per_node¶

Maximum number of concurrent mutating PVE operations (create_vm, delete_vm, create_disk, attach_disk, create_stemcell) allowed against a single node at once. Zero (default) means unlimited and is byte-identical to prior releases. Set a small positive value (for example 4) to prevent large parallel BOSH deploys from saturating a node’s fixed pvedaemon / pveproxy worker pool. A slot is held for the full lifetime of the operation, including any retry backoff (a create retrying against pushback keeps its slot through the 5-60s backoff), so size this with headroom.
Default
0
metrics¶
enabled¶

When true, the CPI appends one JSON-line sample per RPC to the file at pve.metrics.file_path. Each line carries: ts (RFC3339Nano), method, duration_ms, outcome (ok|error), and request_id. duration_ms covers handler execution; post-call work by other configured hooks is not included. Write failures are logged at Warn level and never fail the CPI call. Default false; the hook is not registered when false and adds zero dispatch-path overhead.
Default
false
file_path¶

Absolute path of the metrics file. Required when pve.metrics.enabled is true. The CPI opens, appends, and closes the file per call (atomic line append, no file descriptor held between calls). The file is created if absent. The containing directory must exist and be writable by the CPI process.
Default
""
network_bridge¶

Default network bridge. Per-NIC cloud_properties (bridge, model, firewall, vlan, mtu) and cloud_properties.network_defaults override this per network or for every NIC on one VM — see docs/networks.md for the full per-NIC cloud_properties reference.
Default
vmbr0
network_mode¶

Selects the default create_network/delete_network path. The mode governs networks whose spec does not say otherwise; an unambiguous network spec overrides it (cloud_properties naming a zone or vnet takes the SDN path under “bridge”; naming only a bridge takes the bridge path under “sdn”). This setting has no effect on create_vm’s NIC attachment: mtu=1 vnet-MTU inheritance and cloud_properties.vlan tagging are both decided by the actual SDN vnet list in every mode, not by network_mode. “bridge” — Linux bridge lifecycle via the nodes API. The default: a plain pre-existing Linux bridge (managed: false, cloud_properties.bridge) needs no SDN prerequisites and no CPI-side provisioning — the normal path for an operator-managed fabric. create_network under this mode provisions a bridge on one PVE node only (see create_network’s own single-node-scope note); multi-node bridge fabrics must be pre-provisioned outside the CPI. “sdn” — PVE SDN vnet lifecycle (cluster SDN must be enabled). CPI-created networks are SDN vnets that span every cluster node. Opt in globally (network_mode: sdn) or per network via cloud_properties naming a zone or vnet. “auto” — legacy heuristic retained for compatibility: use SDN when cloud_properties.zone or pve.sdn_zone is set; fall back to bridge otherwise. Deployments that relied on the old auto-to-bridge fallback without naming a zone can also give each bridge network an explicit cloud_properties.bridge and keep the default mode.
Default
bridge
network_resolve_retries¶

Eventual-consistency poll budget for freshly created SDN networks. SDN state propagates over inter-node SSH, so one broken node can leave a newly applied SDN vnet silently pending cluster-wide while the apply task still reports success — the data-plane realization (ifupdown2 reload, pmxcfs propagation) is asynchronous and per-node, so a create_vm that immediately follows a create_network on a different node can attach a NIC to a bridge that does not yet exist there. When > 0 (the default: 30, roughly 30 seconds at the 1-second poll cadence), create_network polls the running cluster SDN config until the new vnet converges, and create_vm confirms each SDN-managed NIC bridge is present on the target node before writing the NIC config — a not-yet-present bridge is treated as a retriable error so the BOSH Director re-drives rather than booting a NIC into a bridge that does not exist, converting the silent race into an actionable, retriable failure. Only SDN-managed vnets are gated; external or static Linux bridges (e.g. vmbr0) always pass straight through untouched, and SDN- membership lookup failures fail the gate open (never block a legitimate deploy on the guard’s own lookup blip) — so the worst case of the default is a bounded retriable delay, never a false block. Both polls are bounded by network_resolve_timeout_sec. Set explicitly to 0 to disable both gates and restore the earlier ungated behavior (create_network and create_vm byte-identical to releases before this default changed). Set to any other positive value to override the retry budget. This property has no default in this spec (see the CPI’s own accessor for why: an unset property must be distinguishable from an explicit 0 to support the disable case) — omitting it from the manifest is exactly equivalent to leaving it unset, which resolves to 30.

network_resolve_timeout_sec¶

Absolute time bound (seconds) on the SDN eventual-consistency poll described under network_resolve_retries: polling stops once this many seconds have elapsed even if the retry budget is not yet spent. Only meaningful when network_resolve_retries > 0. Zero (default) resolves to 60 seconds.
Default
0
node¶

Default PVE node

node_endpoints¶

Optional map of PVE node name to the address (host or host:port, no scheme, no path) that node’s own pveproxy is reachable at. Storage uploads targeting a mapped node (stemcell images, per-VM ConfigDrive ISOs) dial it directly instead of letting the pve.host endpoint proxy the multipart POST cross-node, a hop that sheds connections under burst load. Nodes absent from the map are discovered from /cluster/status when pve.node_endpoints_discovery allows it (default: only when verify_ssl is false); with verify_ssl true we only route nodes listed here, because the discovered address is the corosync link0 IP and stock PVE node certificates usually do not carry it as a SAN. Prefer hostnames or FQDNs the node certificate covers. When a direct dial fails TLS certificate verification, a TLS handshake failure for any other reason, or the connection itself (unresolvable, refused, unreachable, reset or stalled during the handshake), the upload falls back to the pve.host endpoint, logs a warning, and skips that node’s direct route for the rest of the process. Example: {pve02: pve02.example.com, pve03: 10.0.0.13:8006}
Default
{}
node_endpoints_discovery¶

Explicit override for whether uploadISO to a node absent from pve.node_endpoints falls back to /cluster/status name-to-IP discovery. Unset (the default) preserves the pre-existing behavior byte-for-byte: discovery runs exactly when verify_ssl is false. Set false to opt a verify_ssl:false deployment out of discovery entirely, for example a cluster running corosync on a dedicated ring, where the discovered address is not where pveproxy listens, so every discovered dial fails and falls back to the proxied path with a warning until the operator maps the affected nodes explicitly; false skips straight to the proxied path instead. Set true to force discovery on even with verify_ssl true (the operator accepts that the discovered corosync IP is unlikely to be covered by the node certificate; a mismatched dial still falls back to the proxied path rather than failing the deploy). Explicit pve.node_endpoints entries are unaffected either way; they stay opt-in by construction and need no gate.

numa¶

Enable NUMA (numa=1) on every new VM. PVE requires this at create time for memory hotplug to allocate DIMM slots; without it, memory hot-add silently no-ops. Set false to disable globally; per-VM override via cloud_properties.numa.
Default
true
operation_timeout¶
create_sec¶

Deadline in seconds for create_* methods. Zero (default) applies the built-in 1800 s. Honored only when operation_timeout.enabled is true.
Default
0
default_sec¶

Deadline in seconds for every other mutating method (reboot_vm, attach/detach/resize/snapshotdisk, set*_metadata, update_disk). Zero (default) applies the built-in 600 s. Honored only when operation_timeout.enabled is true.
Default
0
delete_sec¶

Deadline in seconds for delete_* methods. Zero (default) applies the built-in 900 s. Honored only when operation_timeout.enabled is true.
Default
0
enabled¶

Opt-in per-method deadline envelope. When false (default) no deadline wraps handler execution and behavior is identical to prior releases. When true, each CPI method runs under a context deadline sized by its class so a wedged retry/poll combination converts into a retriable timeout the Director can act on instead of holding a queue slot indefinitely.
Default
false
query_sec¶

Deadline in seconds for read-only methods (info, has_vm, has_disk, get_disks, calculate_vm_cloud_properties). Zero (default) applies the built-in 120 s. Honored only when operation_timeout.enabled is true.
Default
0
operator_id¶

Optional operator identifier appended to the User-Agent header on all PVE API requests as “pid-”. Use this to attribute CPI traffic in PVE access logs when multiple BOSH directors share a single PVE cluster. When unset (the default), the User-Agent is “BOSH-Proxmox-CPI/“ with no suffix. The value passes through unmodified; avoid whitespace and control characters, which may be interpreted differently by PVE log parsers.

otel¶
enabled¶

When true, the CPI emits OpenTelemetry traces for each RPC and the PVE API calls it makes, exported via OTLP (see pve.otel.protocol) to pve.otel.exporter_endpoint. Default false; when false no tracer provider is built, no network connection is opened, and PVE API calls bypass the tracing layer entirely.
Default
false
export_timeout_ms¶

Upper bound, in milliseconds, on how long telemetry export is allowed to block during process shutdown; each enabled signal’s final flush (traces, logs, metrics) gets its own deadline of this length. Export failures after the deadline are logged at Warn level and never fail the CPI action. A value of 0 is treated as unset and yields the 5000 default; negative values fail validation at CPI startup when tracing is enabled.
Default
5000
exporter_endpoint¶

Base URL of the OTLP collector endpoint (host:port or full URL) used for traces. Required when pve.otel.enabled is true. Also serves as the fallback endpoint for the logs and metrics signals when their own endpoint properties are left empty.
Default
""
insecure¶

When true, the OTLP exporters connect over plain HTTP/gRPC instead of TLS. Default false (TLS). Applies to every enabled signal (traces, logs, and metrics).
Default
false
logs¶
enabled¶

When true, the CPI exports its structured stderr logs as OpenTelemetry log records via OTLP to pve.otel.logs.exporter_endpoint (or, if that is empty, pve.otel.exporter_endpoint). Independent of pve.otel.enabled: logs can be exported with tracing off. Default false; when false no log exporter is built, no network connection is opened for logs, and export failures never fail a CPI call (fail-open).
Default
false
exporter_endpoint¶

OTLP collector endpoint for the logs signal. When pve.otel.logs.enabled is true and this is left empty, it defaults to pve.otel.exporter_endpoint. Rendering fails if both are empty while pve.otel.logs.enabled is true. Ignored when pve.otel.logs.enabled is false.
Default
""
metrics¶
enabled¶

When true, the CPI records a cpi.action.duration histogram (per dispatched CPI method, tagged with outcome) and exports it via OTLP to pve.otel.metrics.exporter_endpoint (or, if that is empty, pve.otel.exporter_endpoint). Independent of pve.otel.enabled: metrics can be exported with tracing off. Default false; when false no metrics provider is built, no network connection is opened for metrics, and export failures never fail a CPI call (fail-open).
Default
false
exporter_endpoint¶

OTLP collector endpoint for the metrics signal. When pve.otel.metrics.enabled is true and this is left empty, it defaults to pve.otel.exporter_endpoint. Rendering fails if both are empty while pve.otel.metrics.enabled is true. Ignored when pve.otel.metrics.enabled is false.
Default
""
protocol¶

OTLP wire protocol used by the exporter for every enabled signal (traces, logs, metrics): “http” for OTLP/HTTP protobuf (collector’s conventional port 4318) or “grpc” for OTLP/gRPC (conventional port 4317). One protocol applies uniformly across whichever signals are enabled; there is no per-signal override. Rendering fails on any other value. Only relevant when pve.otel.enabled, pve.otel.logs.enabled, or pve.otel.metrics.enabled is true.
Default
http
sample_ratio¶

Fraction of traces sampled: greater than 0.0, up to 1.0 (all). Rendering fails on values outside that range; to emit no traces, leave pve.otel.enabled false. Only relevant when pve.otel.enabled is true.
Default
1
service_name¶

Value of the service.name resource attribute attached to every emitted span, log record, and metric, identifying this CPI instance to the observability backend.
Default
bosh-proxmox-cpi
parked_anchor_strict¶

Anchor-missing invariant for the parked strategy. A disk created under “parked” carries a promise in its CID that a parker VM holds it whenever it is detached. When the cluster-wide holder scan then finds no holder at all, or a parker the scan identified vanishes before it can be read or unparked, the anchor is missing: a parker VM was deleted out-of-band. Unset or true (strict, the default): attach_disk, create_vm with disk_cids, and delete_disk refuse with an error naming the recovery instead of proceeding against a volume whose protected home silently disappeared. Set false to restore the permissive behavior (the volume is treated as free-floating), for labs that intentionally delete parker VMs. Disks without the promise in their CID (created before this release, or under the “free” strategy) are always handled permissively; their anchor was never promised.

parked_disk_vmid_range_end¶

Inclusive upper bound of the VMID range for parker VMs. Must be > parked_disk_vmid_range_start. An unset (0) bound resolves to 90999 independently of the lower bound, under every strategy. Must not overlap the VM range, the persistent-disk range, or the stemcell-template range under the parked strategy; under “free” the overlap is accepted, since nothing allocates a parker VMID there.
Default
0
parked_disk_vmid_range_start¶

Inclusive lower bound of the VMID range reserved for parker VMs (bosh-parker-). Each parker VM holds up to 31 parked disk volumes in scsi0-30 slots. The band resolves under every detached_disk_strategy, at the job level and in every cpi-config entry alike: under “parked” (the default) it is where parker VMs are allocated, and under “free” it is read-only — it lets the CPI recognize disks parked earlier and unpark them on their next attach_disk or delete_disk instead of refusing them. An unset (0) bound resolves independently of the other, so a cpi-config entry may move one bound and leave the other at 0. The resolved partner is the built-in bound when that leaves a window no wider than 90000-90999, and otherwise is derived from the bound that was set, at the same width: a start of 50000 alone gives [50000,50999], not [50000,90999]. Both bounds unset resolves to 90000-90999. Move the band only to a window whose parker VMs move with it: a parked disk on a VM outside the resolved band can no longer be unparked, and attach_disk and delete_disk refuse it until the band covers that VM again.

Must not overlap the VM range, the persistent-disk range, or the stemcell-template range under the parked strategy; under “free” nothing allocates a parker VMID, so an overlap there is accepted rather than rejected. The bands are checked against each other, never against VMIDs already allocated.
Default
0
password¶

PVE password (mutually exclusive with api_token). Must be credhub-managed in production: reference via ((pve_password)) against a credhub variable typed password. Empty string when api_token is used instead.
Default
""
placement¶
anti_affinity¶
enabled¶

When true, the node scorer spreads members of the same BOSH instance group across cluster nodes at create time (scheduler-soft anti-affinity): a node already hosting members of the VM’s instance group is penalized so the next member prefers a different node. This is advisory — under resource pressure two members may still co-locate. Default false (opt-in).
Default
false
strict¶

When true, the PVE HA negative resource-affinity rule created by anti_affinity.use_ha_rules is set to strict mode (hard spread): PVE refuses to place or failover a VM onto a node that already hosts another member of the same instance group. Strict mode can block HA failover on small clusters when no compliant node exists. Default false (advisory spreading — PVE honors the rule as a soft preference, not an absolute constraint). Has no effect unless anti_affinity.use_ha_rules is true.
Default
false
use_ha_rules¶

When true (and anti_affinity.enabled is also true), the CPI additionally registers each VM as a PVE HA resource and maintains a cluster-level negative resource-affinity rule (POST /cluster/ha/rules, type=resource-affinity, affinity=negative) keyed on the BOSH instance group, so PVE enforces spreading at the hypervisor level rather than only at create time. On delete the VM is removed from the rule and empty rules are pruned. HA-managed resources interact with the BOSH resurrector; enable only when PVE-level enforcement is desired. Has no effect unless anti_affinity.enabled is true. Default false.
Default
false
az_fallback_order¶

Ordered list of AZ names appended as fallback candidates after cloud_properties.availability_zones are exhausted. Use to express a cluster-wide “prefer AZ-a then AZ-b” policy without per-VM cloud_properties. Empty (default) means no config-level fallback chain; placement considers only AZs from cloud_properties.
Default
[]
az_map¶

Optional. Maps availability-zone names to the PVE node names that serve each AZ. When a VM is created with cloud_properties.availability_zone set to a key in this map, candidate nodes are restricted to the mapped list and the scorer picks the best-fit node within that set. If availability_zone is set in cloud_properties but the value is not present in this map, create_vm returns an error (explicit misconfiguration, not silent fall-through). When empty (default), all online cluster nodes are candidates regardless of availability_zone. Example: {us-east-1a: [pve01, pve02], us-east-1b: [pve03]}
Default
{}
az_shuffle¶

When true, randomizes the AZ order within availability_zones before scoring, breaking ordering ties with a random draw. Default false: preference order is preserved (deterministic). Set to true to distribute VMs across AZs non-deterministically.
Default
false
dlb¶
az_name¶

Sentinel availability-zone name. When a VM is created with cloud_properties.availability_zone equal to this value, the VM is treated as DLB-delegated even when dlb.enabled is false: the CPI skips its own AZ-map node restriction (any online node is a candidate) and registers the VM as a PVE HA resource with auto-rebalance so the DLB places and balances it. Default “dlb”. Set to “” to disable the sentinel trigger entirely (DLB opt-in via master flag only). When both dlb.enabled and the sentinel AZ match, the master flag takes precedence (same registration path).
Default
dlb
enabled¶

Master switch for PVE 9.2+ Dynamic Load Balancer (CRS dynamic mode) integration. When true, every VM created by this CPI is registered as a PVE HA resource with auto-rebalance=1 and state=started, delegating initial placement and continuous rebalancing to the DLB. Works alongside operator-defined availability zones: master-flag-on VMs are still scored within their AZ candidate set at create time; DLB then rebalances within allowed nodes afterward. Default false (opt-in). See also placement.dlb.az_name for per-workload sentinel-AZ opt-in.
Default
false
manage_cluster_crs¶

When true, the CPI ensures the cluster-wide CRS option is set to ha=dynamic (auto-rebalance on) via POST /cluster/options before registering a DLB VM, so CRS/DLB is active without manual pvesh set /cluster/options steps. When false (default), the CPI only reads the cluster CRS setting and logs a clear warning if a DLB workload is requested but the cluster is not in dynamic mode; the operator is responsible for setting the option via pvesh set /cluster/options --crs ha=dynamic,.... Writing the cluster CRS option affects ALL HA guests cluster-wide; default false keeps blast radius minimal.
Default
false
require_shared_storage¶

When true (default), the CPI skips DLB/HA registration for VMs whose root disk resides on local storage (dir, lvm, lvmthin, zfspool) that cannot be live-migrated. Registering a non-shared VM as an HA resource would cause PVE to attempt live-migration during rebalance and fail. Set to false to register regardless — not recommended unless all storage is shared. When the storage type cannot be determined at create time, the CPI fails open (proceeds with registration) and logs a debug entry.
Default
true
enabled¶

When true (default), create_vm runs live node-scoring to select the best-fit PVE node based on available memory, CPU headroom, storage capacity, and guest density. When false, create_vm falls back to pve.node (the existing static behavior). An explicit cloud_properties target_node always wins over scoring regardless of this setting. Protective default: true. Set to false to opt out of scoring and preserve legacy single-node behavior.
Default
true
exclude_maintenance_nodes¶

When true (default), nodes in a PVE HA maintenance or error state, or carrying any tag listed in placement.maintenance_node_tags, are excluded from placement candidates at create_vm time. Set to false only when HA status is unavailable or placing on maintenance nodes is intentional. The Go accessor defaults nil to true — leave this property unset to accept the default.
Default
true
fallback_max¶

Maximum number of alternate candidate nodes to try after a transient clone or VM start failure on the initially selected node. When 0 (default), create_vm makes a single attempt on the selected node and returns any failure immediately — behavior is unchanged from prior releases. When > 0, on a transient failure (classified by the same transient/permanent classifier used for intra-attempt retries) the CPI cleans up the partial VM (purges it, freeing the VMID) and retries on the next-ranked candidate from the same constrained placement set (same AZ restriction and disk fault-domain co-location constraints). Permanent errors (missing clone source, non-retriable PVE errors) fail immediately without consuming alternates. Valid range: 0 (off) to 5. Recommended operational value: 2.
Default
0
maintenance_node_tags¶

List of PVE node tags that mark a node as in maintenance. A node carrying any tag in this list is excluded from placement when placement.exclude_maintenance_nodes is true (or defaults true). Default [“maintenance”]. Set to [] to disable tag-based detection while still honoring HA-status-based detection. Leave unset to accept the default.
Default
[]
memory_signal¶

Selects which live memory signal the placement scorer’s Mem axis uses to rank candidate nodes: “reserved” (default) sums each resident QEMU guest’s configured memory (maxmem) on a node, regardless of run state, and scores free capacity as (node total memory − that sum) / node total memory. This tracks memory reservations rather than actual host memory in use, so a sequence of freshly-booted BOSH VMs — which touch only a fraction of their reserved RAM — still visibly lowers a node’s score on every create, causing sequential creates to fan out across equal nodes instead of stacking on whichever node’s resident-memory usage happens to lag behind. “resident” restores the legacy signal: free memory = node total memory − memory currently resident in the host (Maxmem − Mem from PVE cluster status). Any other value, or a typo, falls back to “reserved” rather than blocking the deploy.
Default
reserved
pin_az_strict¶

Controls the strictness of the rule created by pin_az_via_ha_rules. Default true: a strict rule is a hard AZ guarantee — HA will not relocate the VM off its AZ node set even if every AZ node is down (durability of locality over availability). Set false for a preferred pin that lets HA relocate off-AZ on total AZ failure. Ignored when pin_az_via_ha_rules is false.

This default is deliberate — BOSH AZ placement is a contract, not a hint — but it carries two operational hazards on an AZ mapped to a single node: (1) if that node goes down, the VM cannot restart anywhere else, since HA has no other in-AZ node to fail over to; (2) draining that node for maintenance (reboot, upgrade) has nowhere legal to place the VM, which can wedge the maintenance operation. Both hazards disappear once the AZ maps to two or more nodes, since HA then has an in-AZ fallback target. Map every strict AZ to >= 2 nodes. create_vm logs a Warn at pin time whenever a strict pin targets a single-node AZ, naming the AZ and node, so this condition is visible rather than silent.
Default
true
pin_az_via_ha_rules¶

When true and placement.az_map is set, create_vm writes a PVE HA node-affinity rule binding each VM to its AZ’s node set after scoring, so the AZ placement is durable across HA failover and DLB rebalance (scoring alone only pins the VM at creation). delete_vm removes the rule. Best-effort and non-fatal. Incompatible with the DLB sentinel AZ (rejected at config validation, since DLB intentionally un-pins guests). Default false (opt-in).
Default
false
reserve_storage_headroom¶

Enables the storage-capacity hard placement filter. When true, create_vm computes a minimum free-storage requirement for each candidate node and rejects nodes whose target storage pool does not have enough free bytes to hold the VM’s disks plus a headroom margin. The filter is fail-open: nodes whose storage facts are unavailable (ListStorage API error or no matching pool) are never rejected — they pass and may be picked by the scorer. The filter is applied only to the VM storage pool (pve.vm_storage); disks that resolve to a different pool (e.g. a separately configured ephemeral pool) are excluded from the count. A capacity shortfall is treated as a permanent, non-retriable placement failure. Default false (opt-in); placement is byte-identical to prior releases when unset or false.
Default
false
storage_headroom_mb¶

Extra storage margin in MiB added on top of the VM’s raw disk footprint when placement.reserve_storage_headroom is true. Mirrors vSphere’s DISK_HEADROOM constant: an absolute safety buffer so the placement filter accounts for filesystem overhead and thin-provisioning slack. When 0 (default), a 1 GiB floor is used. Negative values are rejected at startup. Only meaningful when placement.reserve_storage_headroom is true.
Default
0
weights¶
cpu¶

Weight applied to the free-CPU-headroom axis in the node scorer. Higher value = stronger preference for nodes with more available CPU. Must be >= 0. When unset (0), defaults to 0.5 at runtime.
Default
0
guest_count¶

Weight applied to the inverse-guest-count axis in the node scorer. Higher value = stronger preference for nodes hosting fewer guests (spreads load across the cluster). Must be >= 0. When unset (0), defaults to 0.3 at runtime.
Default
0
mem¶

Weight applied to the free-memory headroom axis in the node scorer. Higher value = stronger preference for nodes with more available RAM. Must be >= 0. When unset (0), defaults to 1.0 at runtime.
Default
0
storage¶

Weight applied to the free-storage-fraction axis in the node scorer. Higher value = stronger preference for nodes with more available storage. Must be >= 0. When unset (0), defaults to 0.5 at runtime.
Default
0
pool_reap_empty¶

When true, delete_vm deletes a CPI-managed pool (tagged with the “managed by bosh-proxmox-cpi” provenance comment) once the destroyed VM’s pool membership, captured before destroy, is reported empty by PVE. An operator-created pool without that provenance comment is never reaped, the static pve.vm_pool and pve.stemcell_template_pool are refused by name regardless of comment, and a pool that is still non-empty or already gone (a race with another creator) is tolerated silently, never failing delete_vm. Only the main delete_vm path reaps; the fast-path delete skips the reaper, so some empty pools persist until a later synchronous delete. Requires the CPI’s PVE token to hold Pool.Allocate on the pool being reaped. Default true: the per-deployment pools the pve.vm_pool_template default creates are reaped with their deployments. Set to false to keep empty pools for the operator to manage.
Default
true
port¶

PVE API port
Default
8006
realm¶

Authentication realm
Default
pam
reboot_mode¶

reboot_vm behavior. “soft” (default) performs a graceful ACPI reboot (shutdown + auto-start) and falls back to a hard reset if the guest does not shut down within reboot_timeout seconds. “hard” performs an immediate hard reset (power cycle). One of soft|hard.
Default
soft
reboot_timeout¶

Seconds the soft reboot waits for graceful ACPI shutdown before falling back to a hard reset. Range 1-3600. Ignored when reboot_mode is “hard”.
Default
60
redact_logs¶

When true, the CPI emits a debug-level trace of each call’s request arguments and result with credentials masked — the mbus URL’s embedded NATS credentials, blobstore secret_access_key/password, and any other sensitive-named key are replaced with while the structure is preserved. When false (default), no payload trace is emitted and logging is identical to prior releases. Recommended on so an operator raising the log level to triage a deploy cannot accidentally write agent credentials to syslog.
Default
false
reject_tls_downgrade_overrides¶

Hardens the per-request pve_* context override mechanism (BOSH cpi-config multi-cluster routing). When true, a request whose context carries pve_verify_ssl=false against a job-level config that itself verifies (verify_ssl=true) is rejected with a non-retriable error instead of the default warn-and-proceed behavior. A job-level config that already has verify_ssl=false is never rejected by this knob. Default false: the downgrade is still logged at Warn, never rejected — behavior is byte-identical to prior releases.
Default
false
replica_adopt_timeout_sec¶

Adopt-and-wait bound (seconds) for a racing concurrent template-replica clone. When two CPI invocations independently decide a node needs a per-node stemcell replica, both can pass the existence check while a winner is still building and then clone, producing a duplicate half-built replica template. When > 0, the replica build first probes for an in-flight winner (a VM carrying the replica tags but not yet frozen / still clone-locked) and, finding one, waits up to this many seconds for it to become a settled template and adopts it instead of building a duplicate; a winner that never settles within the bound causes the node to be skipped (best-effort, re-driven next deploy). Default 0 disables the adopt path, leaving replica-build behaviour byte-identical. A conventional value is 300. Operator note: if a winning clone crashed and left its guest stuck with a “clone”/“create” config lock, every subsequent deploy will re-wait the full timeout and skip that node (the warning names the stuck vmid and lock); clear it with “qm unlock ” on the holding node so the replica can settle or rebuild.
Default
0
require_shared_iso_for_ha¶

When true, escalates the config-drive ISO migration-safety warning to a non-retriable error that fails create_vm. The warning (and, when this is true, the error) fires whenever create_vm registers the VM under any HA-driven feature (placement.dlb, placement.pin_az_via_ha_rules, or placement.anti_affinity.use_ha_rules) while pve.iso_storage resolves to a storage pool that /storage does not report as shared. Default false: the hazard is only logged (see docs/configdrive.md), never blocks create_vm.
Default
false
require_snapshot_check_pass¶

When true, a disk operation aborts if the snapshot pre-flight check itself fails (cannot reach PVE to list snapshots). Default false: on check failure the CPI logs a warning and proceeds (fail-open).
Default
false
resize_convergence_timeout_sec¶

Bounds the resize_wait_for_convergence poll, in seconds. This is an independent budget (not the operation_timeout envelope) so the poll is bounded even when that envelope is disabled. 0 means the built-in default of 120 seconds. Negative values are rejected. Only meaningful when resize_wait_for_convergence is true.
Default
0
resize_wait_for_convergence¶

When true, resize_disk polls the VM config after the PVE resize task completes until the reported disk size reaches the requested size, before returning. On asynchronous backends (Ceph RBD, LVM-thin) the size metadata can lag the task completion, so a follow-on operation might otherwise read the old size. The poll is best-effort: if the size has not converged within resize_convergence_timeout_sec it logs a warning and returns success, never blocking the director. Default false (no extra polling).
Default
false
retry¶
disk_migrate¶
cap_ms¶

Wall-clock budget in milliseconds for awaiting the PVE migrate task the cross-node disk migration starts. Zero (default) applies the built-in 1800000 ms (30 minutes), sized for a node-local volume copy. When the budget is exhausted while the task still runs, attach_disk returns a retriable error saying the copy continues server-side; the Director’s retried attach re-enters the migration and completes once it lands.
Default
0
max_attempts¶

Maximum attempts for transient retries of the cross-node disk-migration request attach_disk issues when pve.disk_migration resolves to “on_attach”. Zero (default) applies the built-in budget of 4. The request runs while the mover’s protection flag is down, so a long budget widens that window.
Default
0
pushback¶
base_ms¶

Initial pushback backoff in milliseconds. Zero (default) applies the built-in 5000 ms (longer than the storage-lock curve). Raise to ease off a saturated cluster more aggressively.
Default
0
cap_ms¶

Maximum pushback backoff in milliseconds (the backoff grows toward this value). Zero (default) applies the built-in 60000 ms. Clamped up to base_ms if smaller.
Default
0
storage_import¶
base_ms¶

Base delay in milliseconds for the storage-import exponential backoff (base x 1.5^attempt). Zero (default) applies the built-in 2000 ms.
Default
0
cap_ms¶

Maximum delay in milliseconds for the storage-import backoff. Zero (default) applies the built-in 30000 ms. Must be >= base_ms when both set.
Default
0
jitter_pct¶

Plus/minus jitter percentage (0-100) applied to each storage-import backoff delay. Zero (default) applies the built-in 30.
Default
0
max_attempts¶

Maximum allocation attempts when PVE serialises disk/template imports under a storage lock. Zero (default) keeps the built-in per-handler budget (create_vm 10, create_disk package default). Raise this on slow Ceph where lock windows are wide. Overrides vmid_alloc_attempts for the storage-lock retry loop.
Default
0
storage_lock¶
base_ms¶

Base delay in milliseconds for the storage-lock exponential backoff (base × 1.5^attempt). Zero (default) applies the built-in 2000 ms.
Default
0
cap_ms¶

Maximum delay in milliseconds for the storage-lock backoff. Zero (default) applies the built-in 30000 ms. Must be >= base_ms when both are set.
Default
0
jitter_pct¶

Plus/minus jitter percentage (0-100) applied to each storage-lock backoff delay. Zero (default) applies the built-in 30.
Default
0
max_attempts¶

Maximum attempts for the inner PVE storage-lock retry loop (the “got timeout waiting for worker” / “storage locked” signal that fires inside a single create_disk call). Zero (default) applies the built-in budget of 10. Raise on slow Ceph or NFS where lock acquisition takes longer than the default window.
Default
0
storage_upload¶
max_attempts¶

Maximum attempts for the storage upload retry loops (stemcell image and per-VM ConfigDrive ISO uploads). Zero (default) applies the built-in budget of 30: roughly six minutes on the transient backoff curve, roughly twelve when storage-lock contention dominates. Each attempt reopens the file and re-sends the upload; the backoff curve is selected per fault from the shared transient and storage-lock curves and is not configurable per upload.
Default
0
task_poll¶
base_ms¶

PVE task poll interval in milliseconds. Zero (default) applies the built-in 2000 ms. Raise to reduce API pressure on large clusters.
Default
0
cap_ms¶

Maximum PVE task poll interval in milliseconds (the poller backs off toward this value). Zero (default) applies the built-in 10000 ms. Clamped up to base_ms if smaller.
Default
0
jitter_pct¶

Plus/minus jitter percentage (0-100) applied to each task poll interval. Zero (default) applies the built-in 10.
Default
0
transient¶
max_attempts¶

Maximum attempts for transport-layer transient retries (pvedaemon worker recycling, connection refusals, request timeouts). Zero (default) applies the built-in budget of 8. The backoff curve for this class is fixed (1s growing to 15s, plus/minus 30 percent jitter) and is not configurable.
Default
0
vmid_alloc¶
base_ms¶

Lower bound in milliseconds of the uniform VMID-conflict retry jitter. Zero (default) applies the built-in 50 ms.
Default
0
cap_ms¶

Upper bound in milliseconds of the uniform VMID-conflict retry jitter. Zero (default) applies the built-in 250 ms. Must be >= base_ms when set.
Default
0
max_attempts¶

Maximum attempts for VMID-conflict retries. Zero (default) falls back to vmid_alloc_attempts, then the per-handler default (create_vm 10, create_disk 5).
Default
0
root_disk_bus¶

Selects the PVE bus the root (system) disk is created on. “virtio” (default when empty): root disk lands on virtio0 — byte-identical to every release before this property existed. “scsi”: root disk lands on scsi0, under the same virtio-scsi controller persistent disks already use, which unlocks TRIM (discard) and ssd auto-resolution on the root disk itself (disk_performance.discard/.ssd auto-resolution is otherwise unavailable on virtio-blk).

Persistent-disk slot allocation is unaffected either way: scsi0 has always been reserved for the root disk and attach_disk has always started its free-slot search at scsi1, so there is no slot collision to manage when the root disk moves onto the scsi bus.

Clone-path requirement: create_vm’s dominant path clones a pre-built stemcell template (every “template:” stemcell CID), and a clone inherits its source template’s exact disk layout. Templates are built once by create_stemcell and reused by content-hash tag match, so flipping this setting does not retroactively rebuild existing templates. create_vm compares the resolved bus against the matched template’s actual root disk key before cloning and fails with a clear, non-retriable error naming the conflict on a mismatch, rather than silently producing a root disk on the wrong bus. After changing this value, re-run create_stemcell for affected stemcells so new templates are built on the matching bus.

One of virtio|scsi.
Default
""
sdn_auto_manage_zone¶

When true, create_network creates the SDN zone (type sdn_zone_type, name pve.sdn_zone or the turnkey default “bosh”) if it does not already exist, and delete_network deletes the zone when ALL of the following hold: (1) sdn_auto_manage_zone is true, (2) the zone name does not equal pve.sdn_zone (the operator-pinned zone is never auto-deleted), (3) the zone is not an EVPN zone (operator-owned fabric, never CPI-deleted), and (4) the zone has zero remaining vnets after the vnet delete. Set false to keep zones operator-owned; the CPI then manages only vnets and subnets.
Default
true
sdn_vni_range_end¶

End of the tag auto-allocation band. 0 uses the default band (5000-5999, or 2000-2999 for vlan/qinq zone types).
Default
0
sdn_vni_range_start¶

Start of the band from which the CPI auto-allocates vnet tags (VNIs for vxlan/evpn, 802.1Q VLAN IDs for vlan/qinq) for tagged zone types. 0 uses the default band: 5000-5999, or 2000-2999 when sdn_zone_type is vlan or qinq (the band must fit the 4094 VLAN ID cap; an explicit band beyond it fails validation). Per-network override via cloud_properties.vnet_tag.
Default
0
sdn_vxlan_peers¶

Optional list of VXLAN peer IPs used when the CPI creates a vxlan zone. Empty (the default) derives the peer list from the online cluster nodes (/cluster/status). Set it when tunnel traffic must ride a dedicated underlay network whose addresses differ from the management IPs.
Default
[]
sdn_zone¶

Default PVE SDN zone name for vnet creation. Used when cloud_properties.zone is absent. When empty with auto-manage enabled, the CPI uses the turnkey zone “bosh”; when empty with auto-manage disabled, the zone must be supplied per-call in cloud_properties.
Default
""
sdn_zone_mtu¶

Optional MTU override for CPI-created SDN zones. 0 (the default) lets PVE derive the vnet MTU from the underlay (e.g. 1450 on a 1500 underlay for vxlan). Set for unusual underlays (e.g. 8950 on a 9000-MTU jumbo underlay).
Default
0
sdn_zone_type¶

PVE zone type the CPI uses when it creates a zone (requires sdn_auto_manage_zone). One of: simple, vlan, qinq, vxlan, evpn. The default “vxlan” builds a cluster-wide L2 overlay — peers are auto-derived from the online cluster nodes (override with pve.sdn_vxlan_peers) and each vnet gets a VNI from the sdn_vni_range band. “vlan” maps each managed network onto an 802.1Q VLAN of the physical fabric: the zone is created with pve.network_bridge as its underlay (the bridge must exist on every node, VLAN-aware, with trunked switch ports), each vnet carries the VLAN ID as its tag (cloud_properties.vnet_tag, capped at 4094), and VMs join by bridge selection alone — no VLAN tag ever appears in a VM’s NIC config. “simple” remains an opt-in for single-node or deliberately node-local networks (its bridges do not span nodes). “qinq” zones must pre-exist (the CPI does not auto-create them). “evpn” is never CPI-created: pre-create the zone and its BGP controller in PVE, and the CPI manages only vnets and subnets inside it.
Default
vxlan
security_groups¶

Optional global default list of PVE firewall group names applied to every VM that does not carry a per-call or per-profile security_groups override. Each entry must be a group name that already exists in the PVE firewall configuration. When empty (default []), no global firewall groups are applied and behavior is unchanged from prior releases. As with pve.vm_firewall, the attached group rules are unenforced unless the PVE datacenter firewall master switch (Datacenter > Firewall > Options > Enable) is also on; create_vm warns once per process when it detects the switch is off. See docs/configuration.md for the anti-lockout prerequisites before enabling the master switch.
Default
[]
stemcell¶

prune_dry_run¶

When true and prune_orphans is enabled, the GC pass logs each candidate template it would delete but performs no deletions. Use to audit orphan accumulation before enabling live pruning. Has no effect when prune_orphans is false. Default false.

prune_orphans¶

When true, the delete_stemcell call performs an opt-in garbage-collection pass over bosh-stemcell-tagged templates carrying the calling Director’s “director–” tag (director identity comes from the request context the Director sends with every CPI call; no configuration needed) that no longer have a referencing linked clone. Pruning is best-effort: failures are logged and do not cause delete_stemcell to fail. A request without a director UUID in its context skips the pass with a warning. Default false.

stemcell_fetch_block_private_networks¶

Optional. When true, the https and bosh+blobstore stemcell fetchers refuse any connection whose target resolves to a private, loopback, or link-local address. The check runs on every dial, so redirect hops and DNS-rebinding flips are covered. Leave false (the default) when image_url points at a mirror on a private network — that is a legitimate deployment shape; enable when the mirror is public and a redirect chain must never re-enter the private network.
Default
false
stemcell_fetch_dial_timeout_sec¶

Optional. Bounds the TCP dial step of every stemcell-fetch HTTP request (https and bosh+blobstore sources). 0 (default) applies the built-in 30s default. Valid range when set: 1-3600 seconds. Lower values make stalled handshakes fail faster; raise only if reaching a high-latency artifact mirror.
Default
0
stemcell_fetch_idle_conn_timeout_sec¶

Optional. Bounds how long an idle keep-alive connection stays in the connection pool before being closed. 0 (default) applies the built-in 90s default. Valid range when set: 1-3600 seconds.
Default
0
stemcell_fetch_response_header_timeout_sec¶

Optional. Bounds the wait for response headers after a stemcell-fetch request is sent. Guards against slow-loris drips on the response-header phase. The request body transfer is bounded by the outer 30-minute stemcell-fetch deadline, not by this field. 0 (default) applies the built-in 120s default. Valid range when set: 1-3600 seconds.
Default
0
stemcell_fetch_tls_handshake_timeout_sec¶

Optional. Bounds the TLS handshake step of every stemcell-fetch HTTPS request. 0 (default) applies the built-in 15s default. Valid range when set: 1-3600 seconds.
Default
0
stemcell_replicate_local¶

When true, create_stemcell builds a per-node cache template (tagged “bosh-stemcell-node-”) on every candidate cluster node, enabling create_vm on nodes that cannot reach a cross-node clone source.

The gate is pve.vm_storage — where every cache template’s root disk lands — not the stemcell (qcow2) pool. Replicas are built whenever vm_storage classifies as node-local, regardless of whether the qcow2 pool is shared; a shared qcow2 pool only skips the per-node file copy, not the per-node template. This applies to every stemcell kind, including a pre-uploaded “:light:” image the operator owns.

delete_stemcell removes all replicas (best-effort; a single-node failure is logged and skipped rather than aborting).

Default false: a node-local vm_storage on a multi-node cluster is then rejected at create_stemcell time with a clear error directing the operator to use shared storage or enable this flag. Not needed when vm_storage is shared (e.g. Ceph RBD): a single cache template then serves every node via cross-node clone, even when the qcow2 staging pool (stemcell_storage) is node-local.
Default
false
stemcell_replication_concurrency¶

Maximum number of cluster nodes that receive a stemcell replica upload concurrently during create_stemcell. Only meaningful when stemcell_replicate_local is true. Zero (default) resolves to 1, which replicates one node at a time (serial, byte-identical to prior releases). Set a positive value up to 64 to replicate multiple nodes in parallel and reduce total replication time on large clusters. Per-node semantics are preserved in all modes: a per-node failure is best-effort (logged, does not abort other nodes), idempotent skip applies when a replica already exists, and the per-node in-flight gate is honored. Negative values and values > 64 are rejected at config validation.
Default
0
stemcell_staging_dir¶

Optional. When set, restricts stemcell file reads/writes to this directory using Go 1.24+ os.Root scoping. Defense-in-depth; if unset, behavior is unchanged from prior releases. Paths supplied by the BOSH director must reside under this directory when set. Must be an absolute path to an existing directory on the CPI host.
Default
""
stemcell_storage¶

Storage pool for stemcell qcow2 images. PVE only allows uploads to file-based storages (dir, nfs, cifs, glusterfs, cephfs); block-based storages (lvm, lvmthin, zfspool, rbd) cannot accept qcow2 uploads and are rejected at create_stemcell time. On a multi-node cluster this pool must be shared, unless vm_storage is shared (e.g. Ceph RBD) and stemcell_strategy is “template”: then a node-local file pool is accepted for staging, because the single cache template’s disk lives on vm_storage and clones to every node (the import-strategy fallback remains available only on the staging node). Defaults to vm_storage if unset; in that case vm_storage must satisfy the same constraints.
Default
""
stemcell_strategy¶

How create_vm materializes a VM root disk from a stemcell CID. “template” — clone the per-cluster stemcell cache template (CoW-fast; the cache is built eagerly by create_stemcell and used by every create_vm on the cluster). “import” — import the stemcell qcow2 directly into the VM root disk (full copy per VM; slower, but independent of the template cache — useful when clones must not share a base volume). Per-VM override: vm_type/resource_pool cloud_properties.stemcell_strategy.
Default
template
stemcell_template_node¶

Optional PVE node on which template VMs are created. When empty (default), falls back to pve.node. Useful when stemcell_storage is on shared storage accessible to multiple nodes but template creation should be pinned to one specific node. When using local stemcell_storage (dir, zfspool, lvmthin, or lvm), this must equal the node where stemcell_storage is mounted; pointing to a different node causes create_stemcell to fail because the uploaded qcow2 is not visible from the other node.
Default
""
stemcell_template_pool¶

PVE resource pool name assigned to newly created template VMs. The CPI creates this pool if it does not already exist, tagging it with a “managed by bosh-proxmox-cpi” provenance comment, before the first template VM is assigned to it. Default “bosh-templates” (create-if-missing); set explicitly to “” to opt out entirely — no pool assignment and no pool creation.

Must not equal pve.vm_pool (validated at config load): pools are the CPI’s ACL boundary, and sharing one pool would let a create_vm-scoped ACL also touch stemcell templates. Must not start with “bosh-lock-” (reserved for the cluster-lock sentinel pool namespace, see pve.cluster_lock_mode) and must be a flat PVE poolid (no “/”); the CPI never creates nested pools.
Default
bosh-templates
stemcell_template_vmid_range_end¶

Inclusive upper bound of the VMID range for template VM allocation. When unset (0), defaults to 30999. Must be > stemcell_template_vmid_range_start. Must not overlap the persistent disk range 9000-29999.
Default
0
stemcell_template_vmid_range_start¶

Starting VMID for template VM allocation. Template VMs are allocated in [stemcell_template_vmid_range_start, stemcell_template_vmid_range_end], a dedicated band above the persistent disk range. When unset (0), defaults to 30000. Must not overlap with vmid_range_start..vmid_range_end (VM range) or the persistent disk range 9000-29999.
Default
0
storage¶
max_utilization_mode¶

Enforcement mode when placement.storage.max_utilization_pct is exceeded. Only consulted when max_utilization_pct is > 0. One of “enforce” (default) or “warn”: - enforce: create_vm placement rejects the candidate node; create_disk and resize_disk return a RETRIABLE cloud error naming the pool, the current projected percentage, and the ceiling — capacity can be freed, so the director should re-drive rather than treat this as a permanent, non-retriable failure. - warn: the same facts are logged at Warn and the operation proceeds unblocked. snapshot_disk ignores this setting: it is always Warn-only regardless of max_utilization_mode, since snapshot growth cannot be estimated and blocking snapshots on a capacity guess would be unsound.
Default
enforce
max_utilization_pct¶

Ceiling on projected storage-pool utilization, as a percentage of pool capacity (0-100). Default 0 (disabled): behavior is byte-identical to prior releases. When set > 0, four evaluation points check the target pool’s utilization AFTER accounting for the operation’s disk footprint: - create_vm placement: a candidate node whose vm_storage pool would exceed the ceiling after adding the VM’s disk footprint is rejected from placement (enforce mode) or logged (warn mode). - create_disk: the resolved disk pool is checked before allocation. - resize_disk: the resize delta is checked before the resize call. - snapshot_disk: Warn-only regardless of storage.max_utilization_mode — snapshot growth is unbounded and cannot be estimated ahead of time, so this evaluation point only warns when the pool is already above the ceiling; it never blocks a snapshot. Computation reuses the storage status the CPI already fetches (used/total bytes from the node storage API); when storage facts cannot be determined (API error, pool not found, or pool inactive) the gate fails open (proceeds) and logs a warning, matching placement.reserve_storage_headroom’s fail-open behavior. This gate is independent of, and composes with, placement.reserve_storage_headroom: the pct ceiling is a proportional early-warning band, the headroom filter is a fixed-byte-margin floor; both can be active together.

Why this matters: copy-on-write pools (qcow2, thin-LVM, ZFS) degrade progressively as they fill — noticeably from roughly 50% utilization, badly by roughly 80%, and a pool that fills completely can take days to recover. Ceph separately enforces nearfull/backfill-full/full watermarks at 85/90/95% by default; a ceiling set below those watermarks gives an earlier, CPI-level signal before Ceph’s own thresholds engage, which matters most on small clusters where Ceph’s default watermarks leave little headroom above them — a stricter ceiling (e.g. 70-75) is sensible there. Recommended operational value: 80.
Default
0
storage_tiers¶

Optional map of operator-named storage tiers. Each key is a tier label (e.g. “fast”, “bulk”) and each value is an object with up to two fields: types (list of PVE storage types such as lvm, lvmthin, zfspool, dir, nfs, cifs, rbd, cephfs, btrfs, glusterfs, pbs) and shared (bool). At create_vm or create_disk time the CPI selects the first live cluster storage whose type and shared flag match the criteria. A tier is selected by setting cloud_properties.storage_tier to the tier name. When unset (default {}), no tier matching runs and storage is chosen by the existing vm_storage / disk_storage config fields.
Default
{}
strict_config_validation¶

When false (default) unknown configuration keys are logged as warnings and a handful of contradictory option combinations are tolerated, preserving behavior identical to prior releases. When true the CPI fails fast at start-up, rejecting unknown top-level keys and these cross-field contradictions: use_ha_rules without anti_affinity enabled; network_mode sdn without an sdn_zone or sdn_auto_manage_zone; and a DLB require_shared_storage setting while the dynamic load balancer is disabled.
Default
false
task_poll_adaptive¶

When true, PVE task polling derives its interval from the task’s reported progress (clamped 1-10 seconds) for long operations such as clone and move-disk, instead of the fixed task_poll cadence. Tasks that report no progress fall back to the fixed cadence, so short tasks are unaffected. This reduces poll-storm pressure early in a long operation and tightens polling as it nears completion. Default false (fixed-cadence polling).
Default
false
user¶

PVE username

verify_ssl¶

Verify TLS certificate
Default
true
vm_disk_format¶

Disk format: qcow2, raw, vmdk
Default
qcow2
vm_firewall¶

When true, every NIC of newly created VMs is given PVE’s per-NIC firewall flag (firewall=1). A per-NIC network cloud_property (firewall: true|false) overrides this global default for that NIC. Note: enabling the NIC flag alone does not activate packet filtering — PVE also requires the VM-level firewall to be enabled. The security_groups VM cloud_property enables both the group rules and the VM-level firewall. Default false (no behavior change). PVE also requires the cluster-wide datacenter firewall master switch (Datacenter > Firewall > Options > Enable) to be on before any of this is actually enforced; create_vm probes that switch once per process whenever this flag (or security_groups, or allowed_address_pairs) is in play and warns if it is off. Enabling the master switch activates cluster-wide host-level enforcement — see docs/configuration.md for the anti-lockout prerequisites before turning it on.
Default
false
vm_pool¶

PVE resource pool name assigned to every VM create_vm provisions, on both the import path and the clone path (both PVE’s create and clone endpoints accept “pool” directly). The CPI creates the resolved pool if it does not already exist, tagging it with a “managed by bosh-proxmox-cpi” provenance comment, before the first VM lands in it. Default “bosh” (create-if-missing); set explicitly to “” to opt this global layer out entirely — a higher-precedence layer still applies if set, and when every layer resolves empty no pool is assigned, byte-identical to every release before this property existed.

Resolution precedence (highest wins, first non-empty selected): call-level cloud_properties.pool > vm_type profile cloud_properties.pool > pve.vm_pool_template (rendered) > this global value. See pve.vm_pool_template below for the template layer’s four supported variables.

Setting this lets an operator scope the CPI’s VM.* ACL grants to /pool/ instead of cluster-wide /vms, shrinking the blast radius of a compromised CPI token on a shared cluster: PVE’s create and clone endpoints accept “VM.Allocate on /vms/{vmid}” OR “VM.Allocate on /pool/{pool}” when a pool is supplied, and existing-VM operations (config, power, disk, etc.) resolve permissions on /vms/{vmid} inclusive of any ACL granted on a pool the VM belongs to (PVE’s resource-pool permission propagation) — so a token granted only on /pool/ can fully manage every VM this CPI creates once vm_pool is set. See docs/pve-api-permissions.md for the reduced ACL table and one gap the pool scoping does not close (cloning still requires VM.Clone on the template’s own vmid/pool, which is not automatically covered).

Must not equal stemcell_template_pool, and must not start with “bosh-lock-” (reserved for the cluster-lock sentinel pool namespace, see pve.cluster_lock_mode) — both are rejected at config load. Must be a flat PVE poolid (no “/”); the CPI never creates nested pools.
Default
bosh
vm_pool_template¶

Director-level pool-name template rendered at create_vm time when neither the call-level nor the vm_type-level cloud_properties.pool is set (precedence position 3, above the pve.vm_pool global default; see pve.vm_pool for the full pipeline). Supports four variables: “{prefix}” (pve.vm_prefix), “{director}”, “{deployment}”, and “{instance_group}“; any other “{…}” token is rejected at config load. The rendered name is sanitized (repeated separators collapsed to one, leading/trailing “-” trimmed); a render that collapses to “” falls through to pve.vm_pool. Must not contain “/” (flat names only, the CPI never creates nested pools); the same PVE poolid charset rules that apply to vm_pool also apply to the rendered result, and a render that equals pve.stemcell_template_pool fails create_vm with a named error. The default “bosh-{director}-{deployment}” gives every deployment its own pool; set explicitly to “” to disable this layer entirely, so resolution falls straight from call/vm_type to pve.vm_pool (the single static pool, and the right choice on clusters whose CPI token cannot create pools).

On a create-env path (bosh create-env, no director VM yet) the {director} token is underivable and {deployment} falls back to pve.create_env_deployment, so this default renders “bosh-create-env”, shared by every create-env director on the cluster unless pve.create_env_deployment is set per bundle.
Default
bosh-{director}-{deployment}
vm_prefix¶

Optional prefix prepended to every CPI-provisioned VM’s PVE name. When set (e.g. “cpi”) VM names take the form “---”, so deployments sharing a PVE cluster can be filtered by prefix in the PVE UI. When empty the prefix is omitted and the name is “--”.
Default
""
vm_storage¶

Storage pool for VMs

vm_types¶

Optional map of operator-named VM-type profiles. Each key is a profile name (e.g. “large”, “cpu-heavy”) and each value is an object whose cloud_properties key holds default cloud_properties for that profile. A VM is associated with a profile by setting cloud_properties.vm_type to the profile name at deploy time. Attribute resolution precedence (highest to lowest): per-call cloud_properties, disk_type profile, vm_type profile, global config. When unset (default {}), no profiles are defined and no behavior change occurs.
Default
{}
vmid_range_end¶

Inclusive upper bound of the VMID range for VM allocation. VMs allocated in [vmid_range_start, vmid_range_end]. Default 8999. Must be > vmid_range_start and must not overlap the persistent disk range or the template range. With the default disk range (9000-29999) the effective maximum is 8999.
Default
8999
vmid_range_start¶

Starting VMID for VM allocation. VMs are allocated in the range [vmid_range_start, vmid_range_end]; default vmid_range_start is 100. Persistent disks use synthetic VMIDs 9000-29999 (unchanged).
Default
100

Templates¶

Templates are rendered and placed onto corresponding instances during the deployment process. This job's templates will be placed into /var/vcap/jobs/pve_cpi/ directory (learn more).

bin/cpi (from cpi.erb)
config/cpi.json (from cpi.json.erb)

Packages¶

Packages are compiled and placed onto corresponding instances during the deployment process. Packages will be placed into /var/vcap/packages/ directory.

pve_cpi

pve_cpi job from bosh-proxmox-cpi/0.5.0

Properties¶

agent¶

blobstore¶

mbus¶

pve¶

agent_mode¶

allow_disk_ops_with_snapshots¶

antiaffinity_verify¶

api_dial_timeout_sec¶

api_idle_conn_timeout_sec¶

api_max_idle_conns_per_host¶

api_tcp_keepalive_sec¶

api_tls_handshake_timeout_sec¶

api_token¶

balloon¶

ca_cert¶

clone_mode¶

cluster_lock_mode¶

cluster_lock_timeout_sec¶

cpu_type¶

create_env_deployment¶

debug¶

keep_failed_vms¶

destroy_unreferenced_disks¶

detached_disk_strategy¶

disk_cid_compression¶

disk_delete_state_guard¶

disk_migration¶

disk_perf_invariant_mode¶

disk_performance¶

aio¶

cache¶

discard¶

iops_rd¶

iops_wr¶

iothread¶

mbps_rd¶

mbps_wr¶

ssd¶

virtio_scsi_single¶

disk_storage¶

disk_types¶

disk_vmid_range_end¶

disk_vmid_range_start¶

encrypted¶

ensure_no_ip_conflicts¶

ephemeral_disk_min_mode¶

ephemeral_disk_min_ratio¶

external_command¶

allowlist¶

args¶

command¶

env_passlist¶

methods¶

timeout_ms¶

fast_path_delete¶

fetch_credential_defaults¶

health_check¶

enabled¶

expected_agent_sha256¶

interval_sec¶

timeout_sec¶

hooks¶

host¶

hotplug¶

ip_conflict_probe¶

iso_storage¶

iso_storage_follow_vm_storage¶

lb_register¶

allow_private_ip¶

backend¶

ca_cert¶

endpoint¶

password¶

port¶

timeout_ms¶

user¶

log_level¶

max_inflight_per_node¶

`agent`¶

`blobstore`¶

`mbus`¶

`pve`¶

`agent_mode`¶

`allow_disk_ops_with_snapshots`¶

`antiaffinity_verify`¶

`api_dial_timeout_sec`¶

`api_idle_conn_timeout_sec`¶

`api_max_idle_conns_per_host`¶

`api_tcp_keepalive_sec`¶

`api_tls_handshake_timeout_sec`¶

`api_token`¶

`balloon`¶

`ca_cert`¶

`clone_mode`¶

`cluster_lock_mode`¶

`cluster_lock_timeout_sec`¶

`cpu_type`¶

`create_env_deployment`¶

`debug`¶

`keep_failed_vms`¶

`destroy_unreferenced_disks`¶

`detached_disk_strategy`¶

`disk_cid_compression`¶

`disk_delete_state_guard`¶

`disk_migration`¶

`disk_perf_invariant_mode`¶

`disk_performance`¶

`aio`¶

`cache`¶

`discard`¶

`iops_rd`¶

`iops_wr`¶

`iothread`¶

`mbps_rd`¶

`mbps_wr`¶

`ssd`¶

`virtio_scsi_single`¶

`disk_storage`¶

`disk_types`¶

`disk_vmid_range_end`¶

`disk_vmid_range_start`¶

`encrypted`¶

`ensure_no_ip_conflicts`¶

`ephemeral_disk_min_mode`¶

`ephemeral_disk_min_ratio`¶

`external_command`¶

`allowlist`¶

`args`¶

`command`¶

`env_passlist`¶

`methods`¶

`timeout_ms`¶

`fast_path_delete`¶

`fetch_credential_defaults`¶

`health_check`¶

`enabled`¶

`expected_agent_sha256`¶

`interval_sec`¶

`timeout_sec`¶

`hooks`¶

`host`¶

`hotplug`¶

`ip_conflict_probe`¶

`iso_storage`¶

`iso_storage_follow_vm_storage`¶

`lb_register`¶

`allow_private_ip`¶

`backend`¶

`ca_cert`¶

`endpoint`¶

`password`¶

`port`¶

`timeout_ms`¶

`user`¶

`log_level`¶

`max_inflight_per_node`¶

`metrics`¶

`enabled`¶