Jobs

A job creates a pod, runs argv, produces outputs and terminates. It is the primitive behind pipeline steps, codegen plugins and batch work.

Running one

RunJob is a server stream. The first event is always accepted, carrying the job id — so a caller that loses the stream (a closed tab, a restarted worker) can still find the run.

const run = await cp.jobs.run({
  environment,
  argv: ["ffmpeg", "-i", "/work/in.mp4", "-vf", "scale=320:-1", "/work/out.gif"],
  label: "thumbnail",
  resources: { cpu: "2", memory: "2Gi", ephemeral: "4Gi", pids: 256 },
  timeout: "5m",
}, {
  onAccepted: (jobId) => console.log("job", jobId),
  onState: (state) => console.log(state),
  onLog: (chunk) => process.stdout.write(chunk.text),
});

console.log(run.exitCode, run.terminationName);

run() drains the stream and hands back the outcome; the callbacks are optional. cp.jobs.stream(spec) gives you the raw event sequence instead.

argv is argv — the platform never shell-joins it. If you want a shell, ask for one explicitly: ["bash", "-lc", "..."]. Omitting argv runs the environment's default_argv, or the image entrypoint.

States are QUEUED → RUNNING → SUCCEEDED | FAILED | CANCELLED. The result carries exit_code, a status-derived termination, captured stdout, collected outputs, and usage.

Files in and out

The job's scratch is /work, and that is where both directions land.

Inputs are seeded by an init container before the workload starts. Every path is confined under /work; traversal out of it fails the pod closed.

inputs: {
  "in.mp4": bytes,                                  // inline content
  "seed.json": new URL("https://example.test/seed"), // fetched by the agent
}

Outputs are declared, and the agent collects them from /work after the workload exits, inside the pod's termination grace window:

outputs: [
  { name: "gif", path: "out.gif", maxBytes: 8_000_000, required: true },
]

Read them back off the outcome: run.outputs (each with content or uploadedTo), and run.kv for the /out/.kv.json convention.

They come back as Artifacts on the result with sha256, size and content. A missing non-required output is skipped rather than failing the run.

stdin is delivered as a file and redirected onto fd 0 by wrapping the command (exec "$@" < /work/.cp-stdin). Two consequences worth knowing: the image needs a shell, and a job with no argv — running the image entrypoint — cannot be wrapped and so gets no stdin.

For anything bigger, or anything that should outlive the run, attach a workspace with workspace_id and use /workspace instead.

Reading a job's filesystem while it runs

filesystemApi: true serves the container file API against a running job — useful for watching a build write its cache, or reading a log it has not flushed anywhere yet:

const outcome = await cp.jobs.run({
  environment,
  argv: ["./build.sh"],
  filesystemApi: true,
});

// from another task, while it runs:
await cp.jobs.fs(jobId).readText("/work/build.log");

It is opt-in because it forces the agent sidecar onto a job that may need no egress, no workspace and no outputs, and runs the workload under the supervisor.

It is also only reachable while the job is RUNNING — the pod is torn down at completion and its filesystem goes with it. This does not replace outputs: anything that must survive the run still has to be declared and collected before the pod goes away.

Environment variables

env lands on the pod spec. That makes it readable by anything that can describe pods in the runner namespace — it is not a secret channel.

Names the platform owns are rejected loudly rather than silently overridden: anything prefixed CP_, the proxy variables (HTTP_PROXY, HTTPS_PROXY, NO_PROXY and lowercase forms), JAVA_TOOL_OPTIONS, and the TLS trust-store variables (SSL_CERT_FILE, CURL_CA_BUNDLE, GIT_SSL_CAINFO, REQUESTS_CA_BUNDLE, PIP_CERT, NODE_EXTRA_CA_CERTS). Overriding those would break proxy discovery or terminating egress rules in ways that present as "the network hangs for no reason".

Adapter containers

When a product needs credentials next to untrusted code — the pattern behind upl.im's pipeline steps — run them in an adapter: a second container sharing /work and the pod's network namespace, holding the credentials the workload must never see.

adapter: {
  environment: adapterEnvironment,   // an Environment in the same namespace
  env: { UPL_INTERNAL_TOKEN: "..." },
  cpu: "200m",
  memory: "256Mi",
},
secrets: {
  "step-token": token,
},

The SDK refuses secrets without an adapter rather than dropping them: they are mounted into the adapter alone, so with no adapter there is nothing to mount them into, and a caller who believes their credential reached the job is worse off than one who got an error.

secrets are written as mode 0400 files into the adapter only — never the workload, never the pod spec, never the job record. They live in a Kubernetes Secret owned by the pod and are garbage-collected with it. The workload reaches the adapter over 127.0.0.1, which is also what keeps the pod gVisor-compatible (shared-emptyDir Unix sockets don't cross containers under runsc).

Idempotency and callbacks

idempotency_key claims a key per namespace: a repeat with the same key returns the original job rather than starting a second one.

callback gives the platform somewhere to POST when the job reaches a terminal state — useful when the caller is a workflow engine that doesn't want to hold a stream:

{
  "job_id": "...",
  "namespace": "my-app",
  "state": "JOB_STATE_SUCCEEDED",
  "exit_code": 0,
  "termination": "TERMINATION_EXITED"
}

The token is sent as Authorization: Bearer. The endpoint must be http(s), and is resolved and pinned to a validated public IP before dialling — private, loopback and link-local targets are refused, and there is no rebind window between the check and the dial.

Logs, listing and cancellation

Job.spec deliberately excludes the things that must not be replayed or read back: idempotency_key (a re-run must mint a fresh run), secrets, and the adapter's env.

Warm pools

Set pool_lane to route a job at a pre-warmed pod, cutting cold-start latency. If the lane has no idle pod — or no pool is configured — the job falls back to a cold start. See Warm pools.

Network

A job's default egress is MODE_NONE: no DNS, no route out. Pass egress (inline or by policy id) to change that.

egress: egress.allowlist("my-bucket.s3.example.com")

Or egress.presets("npm"), or a named policy's id.

The job's policy is control-plane state for exactly the length of its run. See Egress.