Installer hangs at 'Installing conntrack…' on Ubuntu when the apt/dpkg lock is held (no lock timeout)

[saadqbal]

Symptom

A client ran the k8s installer (scripts/install.sh → install-k8s.sh) on Ubuntu and it got stuck on Installing conntrack… for a long time and never finished — just an endless spinner, no error, no progress.

Root cause

The installer's apt invocations have no DPkg::Lock::Timeout, and the spinner hides apt's "Waiting for cache lock…" output, so a held dpkg lock looks like a frozen install.

1. No lock timeout. setup_pm() builds the apt command as
   sudo apt-get install -y -q -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold
   (scripts/lib/setup-linux.sh:9). There is no -o DPkg::Lock::Timeout anywhere in the repo, so apt waits on the dpkg/apt lock for as long as the holder keeps it.
2. Something holds the lock on a fresh Ubuntu. apt-daily / apt-daily-upgrade / unattended-upgrades fire at boot and can hold /var/lib/dpkg/lock-frontend for many minutes (longer when a kernel/security batch is pending). install_system_deps() runs apt-get update (lists lock — gets through, "Updating package index…" completes) then apt-get install conntrack (needs the dpkg frontend lock → blocks) → "Installing conntrack…" hangs.
3. The spinner hides the reason. spin_cmd() (scripts/lib/common.sh:121) runs the command in the background with all output redirected to a log file and only renders a spinner — so apt's "Waiting for cache lock: … held by process N (unattended-upgr)" never reaches the screen.

Secondary (latent) issue

export DEBIAN_FRONTEND=noninteractive / NEEDRESTART_MODE=a at setup-linux.sh:260 are ineffective for the apt calls: every apt command runs under sudo, and default Ubuntu sudoers (Defaults env_reset) strips those vars before apt-get sees them. They must be passed through sudo (sudo env DEBIAN_FRONTEND=… apt-get …).

Fix

• Add -o DPkg::Lock::Timeout=600 to the apt update and install commands so apt waits a bounded time and then fails with a clear message (which spin_cmd already surfaces) instead of hanging forever.
• Pass DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a NEEDRESTART_SUSPEND=1 via sudo env so they actually apply.
• Add a visible apt_wait_for_lock step (bounded, with a "Waiting for background system updates to finish…" spinner) before the docker/system-deps installs, so a fresh-boot unattended-upgrades window is explained rather than a mystery freeze.
• Add bats regression tests asserting the apt commands carry the lock timeout and the non-interactive env.

Immediate client workaround

# wait for the background updater to release the lock, then re-run the installer
while sudo fuser /var/lib/dpkg/lock-frontend /var/lib/apt/lists/lock >/dev/null 2>&1; do
  echo "waiting for apt lock…"; sleep 5; done

Acceptance criteria

• apt install/update commands carry DPkg::Lock::Timeout (no unbounded hang).
• Non-interactive env is applied to the sudo'd apt calls.
• A held dpkg lock produces a visible message, not a silent spinner.
• bats suite green; cross-distro installer CI green.

[saadqbal]

Fixed and merged to develop:

• fix(installer): keep apt non-interactive so needrestart can't hang installs on Ubuntu 22.04+ #212 — needrestart / non-interactive apt (hidden-prompt hang)
• fix(installer): bound the apt dpkg-lock wait so a held lock can't hang the install (#210) #211 — DPkg::Lock::Timeout + visible apt_wait_for_lock (dpkg-lock hang)

Together these close both hang modes behind Installing conntrack…. Closing as merged-to-develop; will ship to prod with the next develop→main sync.