- Rust 53.5%
- Shell 35.5%
- C 6.4%
- Python 4.1%
- Makefile 0.5%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
91cc2452c4
Let an unattended, sandboxed agent submit and monitor SLURM jobs on CSCS supercomputers without ever holding the credentials to do so. MUNGE and the network exist only in a trusted broker OUTSIDE the sandbox; the agent reaches it through a file spool and never gets a route to the scheduler. Architecture (fail-closed by construction): - an in-sandbox stub shadows sbatch (and the read-only queries) and writes a request to a project-dir spool; the trusted broker — spawned outside the namespaces, keeping clean MUNGE + network — validates it and runs the real command; - the outer wrapper mints the agent-exec only from a verified sbatch<-stub bind (dev+inode witness), tears the broker down via RAII on any early return, and dies with the session (PR_SET_PDEATHSIG); - the broker spawns ONLY when SLURM is detected — no broker, no spool, no trace on a laptop. Submission surface: an allowlist, not a denylist. The broker does not forward the agent's command line; it CONSTRUCTS the sbatch invocation from an option registry — dangerous families are forced (partition, uenv/view, --export, -o/-e/-D, --wrap), benign resource options are validated against a per-option grammar and re-emitted canonically, and anything unrecognised is rejected. No raw agent token reaches slurmd's parser, so an unseen spelling or channel (getopt-glued shorts, inline at submit time; the body is never rewritten, but a dangerous or unknown compute node, and uenv is inherited from the launching session. Compute-side cage (Rust, adapted from Anthropic's Linux model): other users' homes hidden, credential files masked plus a bounded workdir scan, agent-config and auto-exec paths (.claude, .git/hooks, …) masked with a fresh tmpfs so a job cannot plant a payload — including at paths that do not exist yet — GPU device binds, network unshared. Security posture: the broker went through an adversarial multi-agent review, an independent docs-vs-code pass, and hardware bring-up. Every finding was fixed test-first and structurally rather than case by case — the submission-surface criticals are retired by the allowlist above; the spool validates ids and uses O_NOFOLLOW/O_EXCL with a query timeout and orphan GC; req.cwd is treated as hostile, and the cage policy is resolved at startup from the broker's own trusted directory rather than a path the agent controls. Verified on hardware: 32/32 on Balfrin (x86_64), driven from the trusted side with the caged job's evidence read out-of-band — network unshared, homes hidden, root read-only, credentials masked, auto-exec paths unplantable, a --wrap job coming up caged, GPUs + NVLink, CPU/NUMA pinning. 96 unit tests plus 18 deterministic policy checks that run anywhere. Also here: read-only queries brokered (squeue, sinfo, sacct, sstat, sprio, sreport, sshare); one launcher, with `husk` auto-detecting SLURM; a trusted-side self-test harness that keeps the agent off its own evidence path; seccomp-wrapper allows sched_setaffinity so HPC pinning works; the wrapper identity-maps its user namespace instead of mapping to root; husk/ renamed to seccomp-wrapper/; threat model (AV1–AV8) and design docs, NOTICE crediting Anthropic's sandbox-runtime (Apache-2.0). Known residuals: env secrets not declared in credentials.envVars reach the (network-isolated) job, since --export=ALL is what activates a uenv view; slurmd opens the job's output file outside the cage without O_NOFOLLOW, so the forced output path is not a security boundary; a carve-out under an agent-writable parent can be symlink-swapped during queue wait. Scope: single node, no MPI. Next (ROADMAP.md): recursive sandbox-broker pairs → srun/MPI → compute-node network. The trigger is concrete — a real ICON run script reaches srun inside a brokered job and cannot contact slurmctld, exactly the control-plane right the cage exists to withhold. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
| doc | ||
| project-config | ||
| scripts | ||
| seccomp-wrapper | ||
| slurm-broker | ||
| user-config | ||
| .gitignore | ||
| install-husk.sh | ||
| LICENSE | ||
| make-release.sh | ||
| NOTICE | ||
| README.md | ||
| ROADMAP.md | ||
husk — Claude Code on HPC supercomputers
Claude Code is an AI coding assistant that runs in your terminal. On a shared supercomputer login node it needs some extra care: the agent runs as your user account, so without restrictions it could read your SSH keys, reach internal cluster services, or submit SLURM jobs without asking. This repository provides husk, the tooling to run Claude safely in that environment.
Developed and tested on CSCS supercomputers (Balfrin, Santis).
Install from a release (recommended)
Install on the cluster from a published release tarball — not by cloning and
building. A release carries the prebuilt, architecture-correct seccomp-wrapper
binaries, so nothing is compiled on Balfrin or Santis. (Building from a clone is
only for development.)
Releases are published at:
A single tarball works on both Balfrin (x86_64) and Santis (aarch64) — it ships both binaries and the installer picks the right one for the machine.
-
Download the latest release tarball and its checksum file (on any machine with a browser or network — e.g. your laptop):
husk-<version>.tar.gzhusk-<version>.SHA256SUMS
-
Upload both to the cluster with
scp(Balfrin shown; repeat for Santis, or copy once if your$HOMEis shared between them):scp husk-<version>.tar.gz husk-<version>.SHA256SUMS balfrin:~/If the login node itself has outbound HTTPS, you can skip the laptop and
wgetthe two files directly from the releases page instead. -
Verify and unpack on the cluster:
sha256sum -c husk-<version>.SHA256SUMS # must print: OK tar xzf husk-<version>.tar.gz cd husk-<version> -
Install — continue with Getting started below, running
./install-husk.shfrom the unpacked release directory.
Getting started
Prerequisite: Claude Code must already be installed and authenticated.
huskwraps your existingclaudeCLI — it does not install Claude for you. See Requirements.
Run the install script once from the unpacked release directory (or a development clone):
./install-husk.sh
The script will show you exactly what it will add to
~/.claude/settings.jsonand ask for confirmation before making any changes.
SLURM: record the partition (per machine)
On a machine with SLURM, the broker forces every brokered job onto one
partition, recorded at install time. The built-in default is preemptible, so
installing bare on a site without that partition leaves the broker forcing one
that does not exist — submissions then fail at the scheduler with
invalid partition specified.
Balfrin — use preemptible. It is also the built-in default, so a bare
install works here:
./install-husk.sh --slurm-partition preemptible
Santis — has no preemptible partition, so it must be set explicitly.
debug and shared both work (debug for short test jobs, shared when nodes
are free):
./install-husk.sh --slurm-partition debug
On any other site, list what exists with sinfo -s and pick accordingly.
Prefer a low-priority or preemptible queue where the site has one, so an unattended agent's jobs can be killed and do not consume your allocation's priority. The partition is not auto-detected: which queue an unattended agent submits to is an operator decision, not something to infer from cluster state. To change it later, re-run the installer with the new value.
If ~/.local/bin is not yet on your PATH, add this to ~/.bashrc or
~/.bash_profile:
export PATH="$HOME/.local/bin:$PATH"
Then start Claude Code with husk instead of claude:
husk
The sandbox is now active for all your projects.
To remove everything later — the installed binaries and the settings blocks it added (your other settings are preserved) — run:
./install-husk.sh --uninstall
Optional — per-project restrictions: For tighter control over what Claude can do within a specific project (network access, MCP servers), copy the template into your project and commit it:
mkdir -p .claude
cp /path/to/agentskills-internal/project-config/settings.json .claude/settings.json
See Per-project setup for details.
What it restricts
Filesystem: Claude can only read and write files inside the current project
directory. All home directories — yours and other users' — are not visible to
the agent. Data on shared filesystems outside the home directories
(e.g. /scratch/) is readable but not writable.
SLURM: husk handles SLURM automatically. On a cluster it routes job
submission through a fail-closed broker — the agent can submit batch jobs and
run read-only queries (squeue/sinfo/sacct/…), but the real commands run
outside the sandbox, submitted jobs are re-sandboxed on the compute node, and the
agent never gets the credentials to submit directly. The broker is spawned only
when SLURM is detected; on a laptop there's no broker and no trace, and a
sandboxed sbatch has nothing to talk to. See
Running SLURM jobs (the broker).
Network access (per-project config): ssh, curl, wget, and similar
tools are blocked by default. This is a conservative starting point for shared
login nodes; relax it per project if your workflow needs it.
Credential files: .env files, private keys (*.pem, *.key), and
credential files are blocked from being read or modified.
MCP servers (per-project config): All MCP tool calls are blocked by default. Enable specific servers you trust in your project config.
Per-project setup
The install script applies machine-wide defaults. For project-specific restrictions, copy the template into your project:
mkdir -p .claude
cp /path/to/agentskills-internal/project-config/settings.json .claude/settings.json
Commit .claude/settings.json so the restrictions apply to everyone on the
project. See project-config/README.md for what
each entry does and which ones you may want to relax.
How it works
The sandbox is built from three layers that stack on top of each other:
Process isolation (seccomp-wrapper): A thin wrapper around the Claude process that installs a syscall deny-list — blocking low-level operations like attaching to other processes, loading kernel modules, and manipulating user IDs. This is the outermost layer and covers the entire Claude process tree.
Filesystem isolation (bubblewrap): A sandboxing tool installed system-wide on CSCS supercomputers. Claude Code uses it to run each agent subprocess in an isolated view of the filesystem, where only the project directory is present. This is what keeps your home directory invisible to the agent.
Additional filter (apply-seccomp): A binary from Anthropic that blocks
two specific escape paths: reaching host services via Unix domain sockets, and
bypassing filters through io_uring. This runs inside the filesystem sandbox.
Settings are split into two files:
~/.claude/settings.json— written by the install script; applies to all your projects; covers machine-wide security defaults..claude/settings.jsonin your project — copied fromproject-config/; covers workflow decisions that vary per project (network access, MCP servers).
Requirements
- Claude Code itself — the
claudeCLI, installed and authenticated.huskwraps your existing Claude Code install; it does not install or update Claude for you. Install it (e.g.npm install -g @anthropic-ai/claude-code, or the native installer) and sign in first; see the Claude Code docs. - x86_64 or aarch64 Linux, kernel ≥ 4.14
- bubblewrap installed system-wide (present on all CSCS supercomputers; check with your administrators on other HPC systems)
gcc,make,wget,python3(standard on HPC login nodes)
Running SLURM jobs (the broker)
On a cluster, just run husk — it detects SLURM and starts a small,
fail-closed broker that lets an unattended agent submit and monitor jobs — days
long, with no human to approve prompts — without ever handing the agent the
credentials to do so. On a machine with no SLURM, husk runs the plain sandbox,
with no broker and no trace. (There's no separate command — husk does both.)
Activate your uenv (or modules) first. The broker inherits the software
environment of the shell that launched it, and the agent cannot mount one from
inside the cage. So uenv start <image> (or load your modules) first, then husk.
How it works
Inside the sandbox, sbatch and the read-only query commands are shadowed by a
stub. The stub hands each request to the trusted broker running outside the
sandbox — the only place MUNGE and the network exist. The broker:
- validates the request as hostile input (see policy below);
- re-sandboxes the job on the compute node — the job runs inside the same kind of cage as the login session (other users' homes hidden, credential files masked, auto-exec files write-protected, network unshared), so a prompt-injected job cannot read another user's data or escape the node; and
- submits it under policy and returns the job id.
Because only the trusted broker holds MUNGE and the network path, the agent
cannot bypass it to submit directly — a sandboxed sbatch has nothing to
talk to.
What the agent can do
- Submit batch jobs —
sbatch --partition=<site> job.sh. The partition must be the site's configured one —preemptibleby default, set per machine at install with--slurm-partition(Balfrin usespreemptible; Santis has no such partition, so e.g.debug). Any other is rejected with a message telling the agent how to resubmit (so design jobs to checkpoint and tolerate preemption). Risky options (--output/--error/--chdir/--export/--wrap) are forced to safe values, and the script is snapshotted at submit time (no edit-after-validate window). - Monitor jobs (read-only) —
squeue,sinfo,sacct,sstat,sprio,sreport,sshare. The broker runs them and returns their output; they change no scheduler state. - Everything else is rejected — state-changing commands (
scancel,scontrol update, …), interactivesrun/salloc, and any unknown command.
The submitted job still runs your script with your allocation — but now inside a cage, so an agent that tampers with its own job script is contained to that job's sandbox rather than turned loose unsandboxed on the node. If you want belt-and-suspenders, still keep the job script and its imports read-only to the agent (an absolute path outside the project) and pass the agent's choices as validated data, not as a code path.
Scope in this release: single node, no MPI. Multi-process / multi-node MPI (
srun), a network allowlist for compute jobs (they currently run with the network unshared), interactivesrun/salloc, and read-onlyscontrol show/sacctmgr listare on the roadmap — seeROADMAP.mdandslurm-broker/BROKER.md.
Known limitations
- Network access: Full network isolation (restricting Claude to only reach
Anthropic's servers) is not yet implemented.
curl,wget,ssh, and friends are blocked in the project config template as a compensating control. - SSH to compute nodes: Blocked by default. Remove
Bash(ssh *)from your project's deny list if you regularly need Claude to assist on compute nodes. - SLURM: on a cluster
huskauto-brokers; the broker supports single-node batch jobs (sbatch, re-sandboxed on the compute node) and read-only queries (squeue/sinfo/sacct/…) — see Running SLURM jobs (the broker). Not yet: multi-process / multi-node MPI (srun), a network allowlist for compute jobs (they run with the network unshared), interactivesrun/salloc, and read-onlyscontrol show/sacctmgr list(seeROADMAP.md). The compute-node cage is a subset of the login cage — notably its credential auto-scan uses a built-in pattern set rather than yourRead()deny globs. - Large projects on Lustre: on very large trees (many build directories) on Lustre filesystems, the sandbox can stall for up to ~a minute while it sets up its per-command filesystem rules. This is in the bundled sandbox and not currently configurable; launch the agent from a leaner working directory (not the build-heavy project root) to avoid it.
Acknowledgements
husk builds on Anthropic's open-source
sandbox-runtime (Apache-2.0):
the login sandbox installs and runs its apply-seccomp helper, and the broker's
compute-node filesystem cage is a Rust reimplementation adapted from its Linux
model (read/write policy, credential masking, bubblewrap argument construction).
See NOTICE for details.
License
BSD 3-Clause — see LICENSE for details.