Skip to content

Add PCB-Merging and harden the web server for public deployment - #1

Merged
Roy101 merged 1 commit into
mainfrom
pcb-and-deployment-hardening
Jul 27, 2026
Merged

Roy101 merged 1 commit into
mainfrom
pcb-and-deployment-hardening

Conversation

@Roy101

@Roy101 Roy101 commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

PCB-Merging (Du et al., NeurIPS 2024):

  • pcb_merge() with intra-balancing (self-competition) x inter-balancing (cross-task consensus) scoring, top-ratio drop, score-weighted combine
  • --method pcb with --pcb-ratio/--pcb-lambda/--pcb-scope; server passthrough; frontend method + fields; preset; docs; unit tests

Security hardening (all opt-in / safe-by-default; no new runtime deps):

  • Uploads are safetensors-only by default; pickle/code files refused (_scan_unsafe_files). Every torch.load uses weights_only=True. Closes an arbitrary-code-execution path on uploaded checkpoints.
  • Archive extraction gains zip-bomb defenses (entry count, uncompressed total enforced during copy, per-entry ratio) on top of the existing Zip-Slip guard.
  • Merge worker is sandboxed: stripped environment, job-private cwd/HOME/TMPDIR, rlimits (CPU/file size), hard wall-clock timeout with process-group kill, and no network via an unprivileged namespace (unshare -rn) when available.
  • Opt-in auth (MERGESE_REQUIRE_AUTH): hashed API keys + short-lived anonymous tokens (Turnstile-gated), per-caller daily/active-job quotas, and per-job ownership. Offline admin CLI (server/manage_keys.py). Off by default, so a trusted single-tenant install is unchanged and writes no auth artifacts.

Also: point the web UI GitHub link at srlabUsask/MergeSE; add flask to the dev extra so the server tests run under CI; docs/SECURITY.md deployment runbook.

PCB-Merging (Du et al., NeurIPS 2024):
- pcb_merge() with intra-balancing (self-competition) x inter-balancing
  (cross-task consensus) scoring, top-ratio drop, score-weighted combine
- --method pcb with --pcb-ratio/--pcb-lambda/--pcb-scope; server passthrough;
  frontend method + fields; preset; docs; unit tests

Security hardening (all opt-in / safe-by-default; no new runtime deps):
- Uploads are safetensors-only by default; pickle/code files refused
  (_scan_unsafe_files). Every torch.load uses weights_only=True. Closes an
  arbitrary-code-execution path on uploaded checkpoints.
- Archive extraction gains zip-bomb defenses (entry count, uncompressed total
  enforced during copy, per-entry ratio) on top of the existing Zip-Slip guard.
- Merge worker is sandboxed: stripped environment, job-private cwd/HOME/TMPDIR,
  rlimits (CPU/file size), hard wall-clock timeout with process-group kill, and
  no network via an unprivileged namespace (unshare -rn) when available.
- Opt-in auth (MERGESE_REQUIRE_AUTH): hashed API keys + short-lived anonymous
  tokens (Turnstile-gated), per-caller daily/active-job quotas, and per-job
  ownership. Offline admin CLI (server/manage_keys.py). Off by default, so a
  trusted single-tenant install is unchanged and writes no auth artifacts.

Also: point the web UI GitHub link at srlabUsask/MergeSE; add flask to the dev
extra so the server tests run under CI; docs/SECURITY.md deployment runbook.
@Roy101
Roy101 merged commit 5631a6a into main Jul 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant