run-cc-backend

cc-backend is a Go HTTP server (REST + GraphQL + embedded Svelte UI). Agents drive it through .claude/skills/run-cc-backend/driver.sh, which builds the binary, runs the server in the background in an isolated run directory, and wraps curl for line-protocol ingestion, job start, GraphQL and metric-store debugging.

All paths are relative to the repo root. The driver never touches the repo's ./var (it may hold the user's real data): binary, config, DB, job archive and checkpoints all live in $CC_RUN_DIR (default $TMPDIR/cc-backend-run).

Prerequisites

Verified on macOS (darwin/arm64) with Go 1.27.1; go.mod needs Go ≥ 1.26.3. Also needs python3, curl and bash on PATH. The frontend does not need to be built: the server, REST and GraphQL run fine without web/frontend/public/build (only the web UI pages lack their bundles).

Run (agent path)

One command, end to end (~13 s; exit code 0 means healthy):

.claude/skills/run-cc-backend/driver.sh smoke

It runs setup, start, ingests 11 one-minute samples of fs_read_bw for /home (1) and /scratch (2) on host h1, starts running job 4242, waits for it to be committed, queries jobMetrics(scopes: [node, filesystem]), asserts per-mount series 1 and 2 and a node series 3 without an id, all with no gaps, prints SMOKE OK, and stops the server. On a failed assertion it prints FAIL: ..., exits 1, and still stops the server.

Step by step (a running job on h1 with ten minutes of per-mount data):

D=.claude/skills/run-cc-backend/driver.sh
$D setup                       # build, config, archive, DB, user tester/testpw, JWT
$D start                       # background server on 127.0.0.1:18080, waits for ready

NOW=$(python3 -c 'import time; print(int(time.time()) // 60 * 60)'); START=$((NOW - 600))
python3 -c "
for ts in range($START, $NOW + 1, 60):
    for mnt, v in (('/home', 1), ('/scratch', 2)):
        print(f'fs_read_bw,cluster=devcluster,hostname=h1,type=filesystem,type-id={mnt} value={v} {ts}')
" | $D write -                 # line protocol from stdin (or: $D write FILE)

cat > "${TMPDIR:-/tmp}/job.json" <<JSON
{"cluster":"devcluster","subCluster":"a","jobId":4243,"jobState":"running","numNodes":1,
 "numHwthreads":4,"resources":[{"hostname":"h1"}],"startTime":$START,"submitTime":$START,
 "user":"tester","project":"p","walltime":86400}
JSON
$D start-job "${TMPDIR:-/tmp}/job.json"   # POST /api/jobs/start_job/
ID=$($D wait-job 4243)                    # db id once the job is committed
$D gql "{ jobMetrics(id: \"$ID\", metrics: [\"fs_read_bw\"], scopes: [node, filesystem]) { scope metric { series { id data } } } }"
$D debug h1                    # metric store levels and buffers of host h1
$D reset-store                 # stop, wipe checkpoints, start with an empty store
$D stop                        # waits for the process to exit

command	what it does
setup	rebuilds the run dir from scratch (refuses to delete a dir it did not create)
start / stop	background launch with readiness poll on /login; stop waits for the PID to exit
write FILE|-	POST /api/write?cluster=devcluster, Bearer JWT
start-job FILE	POST /api/jobs/start_job/ with a JSON payload
wait-job JOBID	polls GraphQL jobs until the job appears, prints its id
gql 'QUERY'	raw GraphQL query text, JSON-wrapped and posted to /query
debug HOST	GET /api/debug?selector=devcluster:HOST
reset-store	empty metric store, keeps DB and jobs
smoke	the full flow above, with assertions

Overrides: CC_RUN_DIR=/some/dir and CC_PORT=18081 (both verified). Log: $CC_RUN_DIR/server.log; token: $CC_RUN_DIR/token; last smoke result: $CC_RUN_DIR/result.json.

The fixture cluster devcluster is .claude/skills/run-cc-backend/cluster.json: subcluster a = host h1 (filesystems /home, /scratch, accelerators 0,1, network ib0), b = h2 (/work), c = h3 (no devices). Metrics: flops_any (hwthread), fs_read_bw (filesystem, sum), acc_util (accelerator, avg). Edit it to test other topologies; setup copies it into the archive.

Run (human path)

./startDemo.sh and ./cc-backend -server -dev run against the repo's own ./config.json and ./var, which may hold the user's real data. This skill does not exercise them; agents use the driver.

Test

go test ./...

All packages pass. The run rewrites internal/repository/testdata/job.db; restore it with git checkout -- internal/repository/testdata/job.db before committing.

Gotchas

• Device data must be tagged type=<device>,type-id=<id> (as for accelerators), e.g. fs_read_bw,cluster=devcluster,hostname=h1,type=filesystem,type-id=/home value=1 <ts>. type=node,stype=filesystem,stype-id=… lands in the host buffer itself, not on a device level, so filesystem-scope queries return nothing.
• Running jobs are invisible to GraphQL until committed. start_job writes to the job cache; the commit service moves it every cron.commit-job-worker (default 2 min). The driver config sets 5s; wait-job polls for it.
• Secrets in the caller's env override config.json. An exported JWT_PUBLIC_KEY / JWT_PRIVATE_KEY (or *_FILE) would make the minted token invalid; the driver unsets them for every invocation.
• The final checkpoint is written after the port closes. Deleting var/checkpoints as soon as the port is free resurrects old samples on the next start, and earlier timestamps are then rejected with cannot write value to buffer from past. stop and reset-store wait for the process itself.
• Samples older than a buffer's start are dropped. Ingest each series in time order, and use reset-store before backfilling a window that starts earlier.
• A running job's query window ends at "now". If a minute boundary passes after the last ingested sample, the series gains a trailing null for the open minute. That is correct, not data loss; smoke tolerates trailing nulls but not gaps.
• Fresh DB needs -migrate-db before -init-db; setup does both.

Troubleshooting

• unsupported database version 0, need 13: the DB was never migrated. Run setup again (it migrates first).
• Pipeline hangs after start (e.g. driver.sh start | tail): a background process is holding the pipe. The driver redirects the server's stdio; if you launch the binary yourself, use ( exec ./cc-backend ... ) > log 2>&1 < /dev/null &.
• macOS seq prints epoch seconds as 1.79084e+09: every line gets the same broken timestamp and only one sample lands. Generate timestamps with python3 (as smoke does).
• refusing to delete it from setup: CC_RUN_DIR points at a directory the driver did not create. Pick another path.