harness-improvement
Use when the coding agent repeatedly misses the same class of bug, verification, context, security, UX, session-continuity, cross-harness, packaging, skill-routing, install, or retry-loop issue; turn
SKILL.md.agents/skills/harness-improvement/SKILL.md
GitHubHarness Improvement
- Gather concrete failures and evidence. Do not redesign the harness from one anecdote unless the failure is severe and the human explicitly accepts the scope.
- Capture recurring observations as project-scoped lesson candidates with
.ai/scripts/lesson_candidate.py; include evidence references and confidence. - Never let learned/imported observations auto-edit shipped Skills. Promotion must satisfy
.ai/LEARNING_POLICY.jsonand explicit human approval. - Classify the missing capability: context, procedure, correctness sensor, runtime visibility, security verifier, architecture invariant, session continuity, cross-harness drift, packaging/ownership, skill activation, repo-trust mapping, or retry-loop control.
- Prefer the smallest durable mechanism:
- knowledge -> concise reference;
- repeated procedure -> compact Skill;
- correctness bug -> deterministic test/lint/schema;
- runtime blindness -> browser/log/metric/probe sensor;
- architecture drift -> structural check;
- lost work -> verified checkpoint/state/resume;
- adapter drift -> canonical-source + surface validator;
- packaging drift -> manifest/ownership/exact-artifact test;
- skill-routing uncertainty -> positive/negative routing eval plus trustworthy telemetry or paired causal eval;
- repository uncertainty ->
observed/declared/inferredmapping, never inferred execution by default; - repeated identical failure -> loop guard + changed-hypothesis requirement;
- false positives -> stronger evidence threshold.
- Use the lightest execution profile that preserves truth.
nativeavoids unnecessary durable state;portable/auditedpersist verified progress only. Never remove trait-required product checks just to save tokens. - For harness changes set
traits.harness_modification=trueand recordharness-security,surface-drift,package-contract,harness-self-test, andskill-eval-contractevidence. If a shipped Skill changes, also setskill_modification=true; do not treat Skill presence or self-report as runtime acceptance. - Test the mechanism against the failure that motivated it. Preserve the regression in
self_test.pyor a narrower deterministic test. - Keep Skills compact and progressively loaded. Do not import large external skill/agent catalogs merely because they are popular.
- Check license/provenance before copying external implementation details. Prefer reimplementation of mechanisms when a smaller local control is enough.
- Before release run
package_release.py; verify and re-extract the exact ZIP rather than trusting the source directory.