Jake Tuero

Manual Steps

4 min read

Manual Steps

This is the current checklist of work that Ansible does not perform.

New or Reinstalled Proxmox Host

  • Install Proxmox and configure repositories, networking, and storage.
  • Attach the TrueNAS pve-compute storage.
  • Restore or create the Ubuntu Cloud-Init template.
  • Create or restore each VM and record its VM ID.
  • Configure DHCP reservations or Cloud-Init static addresses.
  • Enable IOMMU/VFIO on the host and attach the RTX 3090 to the GPU VM.
  • Ensure the Proxmox VM guest-agent option is enabled.

TrueNAS

  • Create the expected datasets and SMB/CIFS shares.
  • Configure users, ACLs, and share permissions.
  • Protect media, service backups, the Ansible repository, and Proxmox dumps.
  • Configure snapshots, retention, replication, and restore testing. These are not currently documented or automated.

New TrueNAS Dataset Migration

  • Take fresh snapshots and keep the old TrueNAS system unchanged until the migration is verified.
  • Prefer TrueNAS snapshot replication backed by ZFS send/receive. It preserves numeric UID/GID ownership, modes, ACLs, extended attributes, symlinks, dataset properties, and snapshots.
  • Do not migrate datasets with an ordinary root-shell cp -r; newly created objects can become owned by root, and native symlinks can be incompatible with the Varrock mfsymlinks CIFS mount.
  • Back up and restore the TrueNAS configuration where appropriate, including the local users and SMB configuration. Recheck networking before applying configuration from different hardware.
  • Verify that tuero has the same numeric UID and GID on both systems with id tuero. ZFS preserves numeric IDs, so matching names alone are insufficient.
  • Preserve encryption keys and recovery material separately before replicating encrypted datasets.
  • Recreate or verify each SMB share, share path, ACL, and intended consumer listed in TrueNAS > Current Ansible Dependencies.
  • Compare numeric ownership on both systems with ls -ln and inspect representative ACLs with getfacl.
  • Verify projects from Varrock by creating and deleting a temporary directory, modifying an imported file, and reading and deleting representative symlinks through /mnt/projects.
  • Verify the media and service-backup mounts from Ardougne and reconnect the pve-compute storage from Proxmox.
  • Confirm snapshots, retention, replication, scrub, SMART, and alert schedules on the new system before retiring the old one.

Infra VM

  • For the complete replacement-VM workflow, use Backups and Recovery > Recover a Fresh Infra VM.
  • Complete sudo tailscale up on first enrollment.
  • Approve subnet 10.0.0.0/24 and exit-node use in the Tailscale admin console.
  • Configure Tailscale split DNS for tuerolab.ca through 10.0.0.112.
  • Configure the router to distribute 10.0.0.112 as LAN DNS.
  • Keep required AdGuard settings in Ansible; manual AdGuard UI changes are not backed up or restored.

Services VM Starting Blank Only

These steps apply only when intentionally deploying Ardougne without restoring the appdata archive. Do not repeat them after Backups and Recovery > Recover a Fresh Services VM: the archive restores qBittorrent, Gluetun, ARR, Prowlarr, Bazarr, and Caddy application state.

  • Set qBittorrent’s permanent Web UI username/password.
  • Set qBittorrent’s download directory to /media/downloads.
  • Configure qBittorrent localhost authentication bypass. The current Gluetun port-forward hook uses qBittorrent’s unauthenticated local API to update its listening port.
  • Configure ARR applications, indexers, download clients, and media roots if appdata is not restored.
  • Create the Uptime Kuma administrator account, monitors, and Discord notifications if monitoring appdata is not restored.
  • Use the guarded Ansible archive restore in Backups and Recovery > Recover a Fresh Services VM when moving to a fresh services VM. It intentionally does not support in-place overwrite of live appdata.
  • Verify the Gluetun public IP, port forwarding, and kill switch.

Services VM Restored From Backup

No application Web UI reconfiguration is expected after a successful appdata restore. The archive restores:

  • qBittorrent Web UI credentials, preferences, categories, and download configuration
  • Gluetun state
  • Radarr, Sonarr, Bazarr, and Prowlarr databases and settings
  • ARR indexers, download-client configuration, media roots, and application links
  • Caddy certificate and runtime state
  • Uptime Kuma administrator account, monitors, notification integrations, status pages, and history

The generated PIA and Cloudflare environment files are recreated from Ansible Vault rather than restored from appdata. Confirm the Vault credentials remain valid, then follow Backups and Recovery > Recover a Fresh Services VM and verify the restored services.

Secrets and External Accounts

  • Maintain the encrypted Ansible Vault and its password separately from the rebuilt VMs.
  • Create or rotate the Cloudflare API token with Zone:Read and DNS:Edit for tuerolab.ca.
  • Maintain PIA, TrueNAS, AdGuard, GitHub SSH, and Jupyter credentials in Vault.
  • Ensure the Cloudflare token, Ansible Vault, and repository are recoverable after controller loss.

Before Risky Changes

  • Run and verify service-appdata-backup.service.
  • Confirm .last-success is current.
  • Create a Proxmox VM backup when the change affects the guest OS or disks.
  • Record any new service in Inventory and Services.
  • Add stateful Compose projects to the backup list.

Planned Automation

  • Proxmox VM backup schedule and retention.
  • TrueNAS snapshot retention and replication.
  • Automate initial Uptime Kuma monitor and notification provisioning, if UI-managed configuration becomes burdensome.
  • Tailscale auth-key enrollment, if unattended rebuilding becomes desirable.
  • Health checks for additional containers.

Graph View

Backlinks