Deploy CosmicAC

Deploy the CosmicAC stack on your host machine with Docker Compose. For how CosmicAC connects to your cluster and runs jobs, see Deployment architecture.

Stack

The stack runs six services, plus Redis and Caddy:

cosmicac-wrk-ork
cosmicac-app-node on port 3000
cosmicac-proxy-inference-http
cosmicac-proxy-inference-hrpc
cosmicac-wrk-server-k8s-nvidia
cosmicac-ui
redis
caddy on port 5173

The CosmicAC services use private images published under ghcr.io/tetherto/, in the form ghcr.io/tetherto/<repo>:<tag>. Redis and Caddy use their official public images.

Prerequisites

The recommended host is Ubuntu 22.04 or 24.04 on x86_64. On that host, you need the following:

Docker Engine and Docker Compose v2.
Task.
The jq, node, and kubectl command-line tools.
Access to the private CosmicAC deployment repository, which holds the Compose files and deploy scripts. Request access if you need it.
GitHub Container Registry (GHCR) credentials for private images.
- Your GitHub username.
- A classic GitHub personal access token with the read:packages scope so it can pull the private ghcr.io/tetherto images. If the tetherto org enforces single sign-on, approve the token for the org. See Managing your personal access tokens.
A GPU Kubernetes cluster that already meets the Requirements. CosmicAC connects to this cluster but doesn't set it up.
A valid, readable kubeconfig file for the GPU Kubernetes cluster. Bootstrap stops immediately if the file is missing or malformed.

Steps

Verify the prerequisite tools

Enable Docker and confirm you have the tools from Prerequisites:

sudo systemctl enable --now docker
docker compose version
task --version
node --version
kubectl version --client

Set up the environment

Clone the deployment repository and change into it:

git clone <deployment-repo-url>
cd <deployment-repo>

Create the .env file from the example:

cp .env.example .env

Set the variables in .env. See Environment variables in the configuration reference.

Run the first-time bootstrap

task bootstrap deploys the whole stack with one command, using TAG from .env. For what it runs and the other deployment commands, see Task deployment commands.

If GITHUB_PAT and GITHUB_USER aren't set in .env, bootstrap prompts for them so it can log in to GHCR and pull the private images.

Set KUBECONFIG_SRC to the absolute host path of your kubeconfig in .env before you bootstrap. CosmicAC mounts this kubeconfig into the worker so it can connect to your cluster and run jobs.

A missing kubeconfig path stops bootstrap with an error. Verify the kubeconfig source:

test -s "$KUBECONFIG_SRC"
kubectl --kubeconfig "$KUBECONFIG_SRC" config current-context
kubectl --kubeconfig "$KUBECONFIG_SRC" cluster-info

Run the bootstrap:

task bootstrap

For the kubeconfig structure and how bootstrap consumes it, see Kubeconfig requirements in the configuration reference.

Verify the deployment

Run these checks:

task ps                                          # all services Up
curl -s4 -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/   # 200
TOKEN=$(curl -s4 -X POST http://127.0.0.1:5173/api/login -H 'content-type: application/json' -d "{\"email\":\"${BOOTSTRAP_ADMIN_EMAIL}\",\"password\":\"${BOOTSTRAP_ADMIN_PASSWORD}\"}" | jq -r .token)
curl -s4 "http://127.0.0.1:5173/api/auth/servers?overwrite_cache=true" -H "ttr-token: $TOKEN"
curl -s4 "http://127.0.0.1:5173/api/auth/pricing?location=IN&type=gpu&hrs=2" -H "ttr-token: $TOKEN"
curl -s4 "http://127.0.0.1:5173/api/auth/jobs?page=1&pageSize=10" -H "ttr-token: $TOKEN"

The deployment is healthy when:

task ps shows all services Up.
servers lists your GPUs.
pricing returns a total_cost instead of an error.
jobs responds.

Reference responses

Healthy results: servers → {"data":[{"location":"IN","gpu":{"available":8}}]}, pricing → {"total_cost":2}.

Next steps

With CosmicAC running, install the CLI and create your first job:

Deploy CosmicAC

On this page