Kubernetes

boks can run a tool as an ephemeral pod in a Kubernetes cluster. This is useful for running a diagnostic or one-off command inside the cluster — against cluster-internal services, on a specific node, or in a specific namespace — without building a purpose-built image.

What it does

boks --k8s <tool> creates a pod in your current kubectl context, runs the tool, streams the output to your terminal, and deletes the pod when the tool exits. Pod names follow boks-<uuid>.

boks --k8s python --version
boks --k8s --namespace staging python check.py
boks --k8s --node worker-1 python check.py

--k8s needs to know the exact binary to run — a local boks.toml mapping, the global tool index's bin field, or --exec-as. Every boks-published tool already resolves this way, so this is transparent for the vast majority of usage. A bare, unresolved image reference (not in your config, not in the index, no --exec-as) fails fast with an actionable error instead of guessing:

Error: boks --k8s needs a known command to run 'sometool' — add it to boks.toml
(`sometool = { image = "...", command = "..." }`) or pass --exec-as <binary>.

See "Why a known command is required" below for why.

kubectl plugin

kubectl discovers plugins by looking for an executable named kubectl-<name> on your PATH. Symlink the boks binary:

ln -s ~/.local/bin/boks ~/.local/bin/kubectl-boks
kubectl boks python --version

When invoked as kubectl-boks, the --k8s flag is implied.

This requires a real kubectl binary on your PATH. The kubectl boks ... spelling only works because kubectl's own plugin discovery (search PATH for a literal kubectl-<name> executable, then exec it) is built into the real kubectl binary itself — no other program can intercept kubectl boks ... and redirect it, including a boks-provided kubectl. If you don't have (and don't want) a real kubectl installed at all, boks --k8s <tool> is the direct equivalent and needs no kubectl of any kind — boks talks to the cluster API directly.

One specific case IS handled: if kubectl on your PATH is itself a boks shim (ln -s ~/.local/bin/boks ~/.local/bin/kubectl, so bare kubectl get pods runs a containerized kubectl), kubectl boks <tool> still works — boks recognizes this one specific pattern (kubectl as argv[0], boks as the first argument) and dispatches to --k8s mode directly, without needing kubectl's own generic plugin machinery. Anything else typed as kubectl <args> still runs the containerized kubectl tool normally.

Flags

Set a default namespace in boks.toml:

[global]
k8s_namespace = "tools"

RBAC

Your kubectl context needs permission to create, get, and delete pods, read their logs, and use the exec/portforward subresources in the target namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: boks-runner
  namespace: tools
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["create", "get", "delete", "watch"]
- apiGroups: [""]
  resources: ["pods/exec", "pods/portforward"]
  verbs: ["create"]
- apiGroups: [""]
  resources: ["persistentvolumeclaims"]
  verbs: ["get"]

(The persistentvolumeclaims: get line is only needed for --pvc — boks checks the PVC exists before creating the pod, to fail fast with a clear message instead of leaving the pod stuck Pending on a FailedMount event.)

--node (node-shell, see below) needs meaningfully more: it schedules privileged pods with hostPID/hostNetwork/hostIPC and a hostPath volume mounting the node's root — the above Role is enough to create the pod, but most clusters gate those specific capabilities behind a separate admission mechanism, Pod Security Admission, not RBAC. A namespace labeled pod-security.kubernetes.io/enforce: restricted or baseline will reject a node-shell pod outright regardless of what the Role above allows; only privileged (or no label at all) permits it. This is expected, not a bug — plenty of users will only ever run plain boks --k8s <tool>, which needs none of this, and not having node-shell clearance on a given cluster is a completely normal, fine state to be in.

Security in --k8s mode

Kubernetes' securityContext gives a subset of boks's zero-trust posture a direct, Pod-level equivalent — these flags are honored:

The forced runAsUser: 1000/runAsGroup: 1000 mirrors the local podman path's own --userns=keep-id:uid=1000,gid=1000, which is applied regardless of what the image's own Dockerfile USER declares. Every boks-published image already runs as uid 1000 on its own, so this is a no-op for those — it matters for external images that default to root (the official, non-boks-published bash image, for one).

--cap net is the one exception: there is no per-pod equivalent for network isolation in Kubernetes — enforcing it needs a cluster-level NetworkPolicy, which is admin-configured infrastructure, not something a single Pod spec can request for itself. The pod always gets whatever network access the cluster's own policy allows; boks cannot restrict or grant it per-run in --k8s mode. (--port, unlike --cap net, is supported — see below; publishing a port inbound and restricting the pod's own outbound access are different mechanisms.)

A failed pod now propagates: if the tool inside the pod exits non-zero, or the container never starts at all (bad entrypoint, image pull failure, ...), boks --k8s exits with the pod's own exit code and prints the failure detail to stderr — it no longer silently reports success.

Node-shell (--node)

--node <NODE> isn't just scheduling — it means "run this tool as if you were on the node itself," the Kubernetes equivalent of sudo <tool> on that box. No separate flag or combination of flags is needed:

boks --k8s --node worker-1 vim /boks/workdir/etc/kubelet.conf   # edits the NODE's real file
boks --k8s --node worker-1 tcpdump -i eth0                  # the node's real network interface
boks --k8s --node worker-1 ps -e                            # the node's real process tree

This implies full host access on that node: --privileged, real root (runAsUser/runAsGroup: 0 — plain --privileged alone does not give you root; Kubernetes leaves the container on whatever uid the image declares, 1000 for every boks-published image, unless a uid is set explicitly), and hostPID/hostNetwork/hostIPC. Whoever can schedule a pod like this on a specific node already has the equivalent of root on it via RBAC/Pod Security Admission — a non-root container uid on top wouldn't hold back anything real, just add friction, so node-shell doesn't try.

Filesystem access: the entire node's real / is bind-mounted (via a hostPath volume) at /boks/workdir — the same mount-point convention the podman/docker path uses for the mounted cwd (see "Mounting a PVC" below for the other place it's reused), and also set as the container's workingDir, so a relative path (vim etc/kubelet.conf) resolves there too. Every node file is reachable this way — /boks/workdir/etc/kubelet.conf, /boks/workdir/var/log/..., /boks/workdir/usr/..., all of it — nothing is copied or selectively remounted.

Bare absolute paths are not the node's files. vim /etc/kubelet.conf (no /boks/workdir/ prefix) edits the container's own /etc, not the node's — this is the one real trade-off of mounting everything at a single path rather than making the node's files appear at their bare native locations. Two exceptions, bind-mounted natively regardless: /dev and /sys (kernel interfaces that plenty of tools expect at their conventional absolute paths, not under a prefix), and /proc (already correctly reflects the node's real processes via hostPID, independent of any mount).

(An earlier design instead tried natively re-binding a hand-picked list of "data" directories — /etc, /var, /root, etc. — directly onto their container-side paths, for a fully-native absolute-path feel. Simplified away in favor of the single /boks/workdir mount: it reaches strictly more — nothing is left out just because it wasn't on the list, which is exactly what closed the /usr-content gap below — at the cost of the /boks/workdir/ prefix. Before that, an even earlier attempt used a full chroot() before running the tool, which broke every dynamically-linked tool binary outright — see boks-containers/tools/k8s-init/pause.rs's module doc comment for why.)

Known gap. /run (dbus/systemd sockets, container runtime sockets, and — on most distros by default — the volatile journal) is reachable as data at /boks/workdir/run/..., including live sockets (a bind-mounted AF_UNIX socket file is the same underlying socket, not a copy) — but a tool like systemctl/journalctl that internally hardcodes /run/... wouldn't know to look under /boks/workdir instead, so live systemd/dbus introspection through such a tool still doesn't work. Editing/reading any node file or state, checking processes, and touching real devices all work.

--node and --pvc are mutually exclusive — both would mount at /boks/workdir, and node-shell already gives full access to the node's real filesystem anyway, so there's nothing a PVC mount adds on top of it.

Mounting a PVC (--pvc)

--pvc <name> mounts an existing PersistentVolumeClaim (same namespace as the pod) at /boks/workdir — the same mount-point convention the podman/docker path already uses for the mounted cwd, kept consistent rather than inventing a separate path:

boks --k8s --pvc data hx config/config.toml   # config/config.toml resolves inside the PVC

The container's workingDir is also set to /boks/workdir, so relative paths the tool is given resolve inside the PVC — kubectl exec (and the pods.exec() boks uses) inherits the container's workingDir for the exec'd process's cwd. This is a brand-new pod boks fully creates and controls, exactly like every other --k8s invocation, just with one more volume — no new mechanism, unlike node-shell.

boks checks the PVC exists before creating the pod (a plain get, not a phase check — a PVC using WaitForFirstConsumer binding legitimately shows Pending until something uses it, which is not an error) and fails fast with a clear message if it doesn't, rather than leaving the pod stuck Pending on a FailedMount event.

Permissions depend on how the PVC was provisioned. A dynamically-provisioned PVC (via a StorageClass) may come out already writable by boks's default uid 1000, depending on the provisioner — but this is not guaranteed for every storage class. A statically-provisioned PV (an admin pre-created it directly) commonly comes out root-owned instead — verified live against a plain hostPath-backed static PV, where a bare --pvc write failed with Permission denied. --cap rwimg is the fix (real root, not just bypassing the uid-1000 enforcement) — same escalation valve as any other permission mismatch, nothing PVC-specific.

One more caveat worth knowing: a ReadWriteOnce PVC already attached to a pod on a different node can leave boks's new pod stuck Pending (a volume can't attach to two nodes at once). There's no way to steer which node --pvc's pod lands on (--node is node-shell and mutually exclusive with --pvc, not a plain scheduling hint) — waiting until the other consumer releases the volume, or using a ReadWriteMany-capable PVC if the workload needs concurrent access, are the options; boks doesn't special-case this.

This is a full host-access escape hatch by design — see the RBAC section above for why some clusters won't allow it (Pod Security Admission, not just RBAC), and that not having clearance for it is a completely normal state for a --k8s user who only ever runs plain tools without --node.

Vulnerability scanning

--k8s scans images too, but the scan itself has to run in the cluster: boks never pulls the image locally in this mode (that's the whole point — no local podman/docker dependency), so there's no local tarball to hand the scanner the way the podman path does. Concretely:

1. A short-lived scan pod runs grype registry:<image> --output json --quiet directly against the image reference — grype's own registry client fetches and analyzes it, so nothing is pulled to your machine.
2. Its JSON output comes back to the boks client, which parses it and applies the exact same security_scan_deny/security_scan_prompt policy, findings table, and interactive run anyway? [y/N] prompt as local mode (see Security Configuration) — the cluster only ever runs grype and reports results; it never decides anything.
3. The scan pod is deleted immediately after (same Ctrl-C/activeDeadlineSeconds safety nets as the tool pod, see below), and only then is the real tool pod created.

boks --k8s python --version    # first run: scans, then runs
boks --k8s python --version    # second run: cached verdict, scans nothing
boks --k8s --force-scan python --version   # ignore the cache, scan again
boks --k8s --skip-scan python --version    # skip scanning entirely

The cache is weaker than local mode's. The client keeps its own verdict cache at ~/.cache/boks/scanner/k8s-scan-cache.json, keyed by the image string and a timestamp — security_scan_max_age_secs governs how long a verdict is trusted, exactly like local mode. But unlike the local cache (which is bound to the image's exact content digest, so a rebuilt :latest is detected immediately), the k8s cache has no digest to bind to — computing one without a full pull would need a registry manifest-HEAD request, not implemented yet. In practice: if a tag is rebuilt with new (or fixed) vulnerabilities, the cached verdict can be stale until it naturally expires. --force-scan is the escape hatch when you need a guaranteed-fresh answer.

If the scan pod can't even start (scanner image unpullable, a cluster hiccup), boks warns and lets the tool run anyway — the same "infrastructure failure, not a verdict" distinction local mode makes. If the scan pod runs but grype exits non-zero, or a deny-tier finding aborts the run, that's fail-closed: no tool pod is ever created.

Pod lifetime and cleanup

A pod is deleted the moment the tool exits — but if the local boks process itself is killed first (Ctrl-C, SIGKILL, a crash, a lost network connection), that cleanup call may never run. Two independent safety nets:

• Ctrl-C deletes the pod immediately. A background task races the rest of the run against SIGINT; if it fires, the pod is deleted right away and boks exits 130 (the usual 128 + signal shell convention) — instead of leaving the pod running until the deadline below.
• A 12-hour activeDeadlineSeconds backstop ([global] k8s_max_pod_seconds, configurable). Every pod is created with a hard wall-clock cap; Kubernetes itself kills it if the deadline passes, regardless of whether the local boks process is still around to ask. This protects against the cases Ctrl-C can't (the process dying outright) — it is not an idle timeout, so a long but genuinely active interactive session is capped by it too. Kubernetes only allows reducing this value after a pod is created, never extending it, so there's no way to keep pushing the deadline out for as long as the client stays active — pick a value generous enough to comfortably outlast a real session.

While a pod is being created and scheduled, boks shows a SCHEDULING spinner (or a plain scheduling pod ... line in non-interactive/CI mode) — previously this ~30-second-at-most wait was completely silent by default.

Interactive I/O

--k8s forwards your local stdin into the pod and sets TERM/COLORTERM/LANG/LC_ALL from your own environment, so interactive tools and piped input both work:

kubectl boks python                          # a real REPL session
echo hello | boks --k8s bash -c "cat -"       # pipes through and echoes back
kubectl boks htop                             # renders correctly, not garbled

For a genuinely interactive session (both your local stdin and stdout are real terminals), boks also puts your terminal into raw mode and sends it an initial size — the same reason full-screen tools render correctly locally now render correctly over --k8s too.

--port: reach a pod-side server locally

A tool that binds a port inside the pod (a dev server, a debug endpoint) is reachable at 127.0.0.1:<port> on your machine, proxied through the cluster's own portforward API — the same mechanism kubectl port-forward uses, not a change to the pod's network policy:

boks --k8s --port 8000 python -m http.server 8000
# in another terminal:
curl http://127.0.0.1:8000/

Writable HOME and /tmp

Every pod gets a writable HOME=/boks/home and a writable /tmp (both backed by emptyDir volumes, matching the same /boks/home convention the local podman path already uses) — so the experience matches running locally rather than hitting a read-only-filesystem error for things like a REPL's history file:

kubectl boks python      # writes ~/.python_history without warning
kubectl boks ipython      # has a real, writable temp directory

/boks/home starts empty on every run — it isn't yet seeded with your actual local dotfiles (that's a separate, not-yet-built feature; see the note at the end of this page).

Why a known command is required

sort, wc, and a bare python (no -i) invocation — anything that must read all of stdin before producing output — need a real, dedicated fix to work at all over Kubernetes: --k8s runs the tool via Kubernetes' exec subresource against an already-running placeholder container, rather than attach-ing to the pod's own initial process. attach connects to stdio streams that were already open at container-creation time, before any client connects, so a tool that buffers to EOF before writing anything never gets a chance — confirmed identical with real kubectl attach -i. exec spawns a genuinely fresh process, so it doesn't have this problem.

exec can't inherit an image's own ENTRYPOINT/CMD the way pod creation can — it always needs an explicit argv. That's the whole reason a known command is required (see "What it does" above): there's no attach-based fallback to quietly degrade to anymore. This is also why it's reliable: an earlier version of --k8s fell back to attach for unresolved commands, and that fallback turned out to be its own source of rare flakiness (a fast-exiting command could race the attach connection and return a clean exit with no output) — removed entirely rather than patched further.

What is not supported

The following genuinely have no effect in --k8s mode:

• No working-directory mount. Your current directory is not mounted into the pod — unlike local mode. The pod runs the tool's image as-is; any files the tool needs must already be in the image or reachable over the network.
• No environment-file building. -e @pip:requirements.txt does not build a pod from an environment image; the env-build path is local-only.
• No dotfile forwarding. Unlike local mode's dotfiles = [...], a tool's actual local dotfiles (.vimrc, .gitconfig, ...) are not copied into the pod — only a writable, empty /boks/home HOME (see above).
• No PVC mounting. There is no flag to attach a PersistentVolumeClaim.
• Hardcoded resource limits. Each pod is created with a 512 MiB memory limit and 500m CPU limit (256 MiB / 250m requests). These are not configurable.
• --cap net — see above.
• A resolvable command is required. See "Why a known command is required" above — a bare, unresolved image reference (no boks.toml entry, no index bin field, no --exec-as) fails fast with an actionable error rather than guessing at the image's ENTRYPOINT.
• Live terminal resize. boks sends the terminal size once, at exec time; resizing your terminal mid-session doesn't update the pod's view of it.

If you need any of the above, run boks locally and let the tool reach the cluster over the network instead.

Debugging a failed pod

boks prints the container runtime's own failure reason (e.g. a bad entrypoint, an OCI runtime error) to stderr when the pod's container never starts. For anything not captured there, check cluster events:

kubectl get events --sort-by='.lastTimestamp' -n <namespace>

The pod is deleted when the tool exits, so to inspect its spec, watch in a second terminal before running boks:

kubectl get pods -n <namespace> -w &
boks --k8s python --version
kubectl describe pod <pod-name> -n <namespace>