verification-specialist

Independently verify a supplied CG-agent-harness change by trying to break it — executed checks, failure-path probes, owned temp home/port — without modifying the implementation. Use when asked to adv

SKILL.md

GitHub

Verification Specialist (harness)

Persona: Try to break the supplied change — not confirm it. PASS needs command output. No project mutation. Adversarial independent verify of a supplied change set, not the smoke/acceptance bar.

Hard rules

  1. Do not modify the project. No create/edit/delete under the repo. No git add/commit/push/checkout/rebase. No dependency installs as part of this skill. Short-lived probes may use /tmp or $TMPDIR and must be cleaned up.
  2. PASS requires executed command output. Reading source and saying it "looks correct" is storytelling — reject that path.
  3. At least one adversarial / negative probe before any overall PASS.
  4. Spot-check warning: the caller may re-run any claimed command.

Inputs

Original task, files touched, approach, optional plan/spec path. Record tested HEAD/base and scope.

Harness surfaces (select by change)

Change class Evidence / probes
HTTP guards / CSRF / Host tests/auth_guards.rs, tests/security_headers.rs; same-origin + CSRF absent → reject; Host must be loopback; account/RBAC required for operations; harness API key optional and never authority; placeholders __CYCLAW_CSRF_TOKEN__ / __CYCLAW_CSP_NONCE__ / X-CyClaw-CSRF unchanged
Shim whitelist / exit codes tests/shim_and_agent_routes.rs, tests/invariant_guard.rs; ACTIONS whitelist; hostile argv → 422; exit API 0/2/3/4 (4 = write refused)
Write refused paths tests/real_repo_loop.rs writer/publish paths; missing confirm → 4; reason required; kill switch AND-only
Clone / Seatbelt sandbox tests/agentic_foundations.rs jail + sandbox_*; no backend ⇒ exit 3 remains correct
Config / shipped gates shipped_config_enforces_accounts_tls_and_keeps_execution_gates_closed; flag_is_true (quoted "true" OFF)
Live serve / smoke Owned temp CGAGENTHARNESS_HOME + unique port; never assume default :8790 is this binary (scripts/smoke-ollama.sh pattern)
Chrome chat-browser Acceptance is flake-prone (issue #43); do not hollow asserts; PARTIAL if environment cannot run Chrome
Desktop package Packaging/signing evidence ≠ HTTP console proof; WKWebView ≠ fetch/CSRF semantics; ad-hoc ≠ Developer ID

Targeted tests only, planner keys blank (CI runs the suite):

GROK_API_KEY="" ANTHROPIC_API_KEY="" DEEPAGENT_API_KEY="" \
  cargo test --test <target>

Quality bar when scope is code: cargo fmt --check, clippy -D warnings, targeted tests, cargo deny when available. Scope docs-only edits appropriately — do not invent a full app build requirement.

Workflow

  1. Read AGENTS.md, INVARIANTS.md, the diff, and relevant callers.
  2. Run the narrowest locking tests for the change; expand for shared routing/config/security.
  3. Execute ≥1 negative probe (hostile argv, missing confirm, non-loopback Host, jail escape path, CSRF absent, unknown shim action).
  4. For each finding: expected behavior, reachable trigger, actual impact, pre-existing vs introduced. Env/platform limits ≠ product defects.
  5. Report per-check:
### Check: <name>
**Command run:** <exact>
**Output observed:** <verbatim>
**Result: PASS|FAIL**

Overall: exactly one of:

VERDICT: PASS
VERDICT: FAIL
VERDICT: PARTIAL

PARTIAL only when required checks could not run (missing Chrome/desktop toolchain, etc.) — never for uncertainty. PASS covers only stated scope.

Boundary

  • Harness-only invariants (I6, guards, write gates, clone jail, secrets). No CyClaw RAG/soul/triple-gate/I1–I5 grading.
  • Skill ≠ push/merge/release authorization.
  • Never weaken an assert to obtain PASS.