Skill-Eval English Skill Quality Evaluation Assistant

Prerequisites

• This is a pure instruction skill that fits 秒哒-style usage; it does not depend on Node, Python, CLI tools, network access, or any external runtime.
• Review only local files that the user provides or that can be read from the workspace: SKILL.md, references/, scripts, and configuration files.
• The output is a heuristic, banded judgment based on rules, not a reproducible benchmark-engine score.
• If the user asks for real token usage, benchmark runs, or runtime behavior, explain that a local execution environment is required and provide a measurement plan instead of pretending to run it.
• Read the target files first; do not judge from the directory name or the user’s summary alone.

Core Capabilities / Workflow

1. Target Detection and Routing

Trigger phrases: “evaluate this skill”, “review this skill package”, “audit this skill”, “what should I fix first”.

Workflow steps:

1. Decide whether the user gave a single SKILL.md, a skill directory, a skill package directory, or only a skill name.
2. If the user only gives a skill name, search the user-specified path, the current workspace, and common skill locations first; ask one short clarification question only if the target is still ambiguous.
3. Record the evaluation scope: target path, entry file, references, scripts, configs, and whether multiple skills are bundled together.
4. Choose the correct mode: single-skill review, skill-package review, budget analysis, rewrite brief, or measurement plan.

Output focus: Say up front whether you are reviewing a single skill, a package, a budget issue, or a rewrite brief, and list the key files you read.

2. Single Skill Evaluation

Trigger phrases: “evaluate this skill”, “how good is this SKILL.md”, “does this skill look solid”.

Workflow steps:

1. Check frontmatter: whether name, description, license, packageType, and instructionOnly match the intended skill type.
2. Check the description: does it explain what the skill does, when to use it, and what user phrases should trigger it?
3. Check the body structure: prerequisites, core capabilities, execution priorities, references, communication rules, and common pitfalls.
4. Check progressive disclosure: is the main file focused on the workflow, with detailed material moved into references/?
5. Check links and relative paths: do referenced files actually exist?
6. Check whether scripts or examples fit the skill type; a pure instruction skill should not require an unavailable runtime.

Output focus: Give structure quality, trigger quality, maintainability risks, and concrete fixes.

3. Skill Package Evaluation

Trigger phrases: “review this skill package”, “audit this bundle”, “check whether these skills are organized well”.

Workflow steps:

1. Identify the package shape: root directory, skill directories, shared references/, assets, scripts, and any manifest or platform config.
2. Check role boundaries: does each skill have a clear entry, or do descriptions overlap too much?
3. Check naming consistency: directory names, frontmatter name, reference paths, and user-facing names should line up.
4. Check shared material: package-wide references should be shared, and skill-specific details should stay isolated.
5. Summarize the strongest skill and the weakest skill with evidence.

Output focus: Call out package-level risks, routing conflicts, and organization issues.

4. Token / Context Budget Analysis

Trigger phrases: “token budget”, “context cost”, “why does this feel heavy”, “is the description too long”.

Workflow steps:

1. Estimate three buckets: trigger content, invoke content, and deferred content.
2. Use references/eval-rubric.md to place each bucket into good / moderate / heavy bands.
3. Make it explicit that this is a static estimate, not measured token telemetry.
4. Identify the easiest cost reductions: shorten the description, compress SKILL.md, move long templates into references, or split a large file.

Output focus: Show the three buckets, the main cost sources, and the smallest useful fixes.

5. Score and Risk Summary

Trigger phrases: “skill score”, “why did it get this score”, “give me the evaluation report”, “what is the risk level”.

Workflow steps:

1. Combine structure, trigger quality, budget pressure, links, boundaries, and executability into an A-F band.
2. Assign riskLevel: low / medium / high.
3. Separate confirmed issues, likely risks, and items that cannot be judged yet.
4. Cite file paths and line numbers for every important conclusion.

Output focus: Start with At a Glance, then explain Why It Matters, Fix First, and Recommended Next Step.

6. Priority Fix Ordering

Trigger phrases: “what should I fix first”, “help me prioritize fixes”, “which issue matters most”.

Workflow steps:

1. Rank fixes by impact, trigger risk, user-visible failures, and cost to repair.
2. Separate must-fix items from nice-to-have items and observations.
3. Each item should include severity, evidence, reason, and the smallest repair action.
4. Do not turn style preferences into blocking defects.

Output focus: Produce a Fix First table and avoid suggesting a big rewrite when a small change is enough.

7. Rewrite Brief Generation

Trigger phrases: “improve this skill”, “rewrite this skill based on the evaluation”, “give me a rewrite brief”.

Workflow steps:

1. Turn the evaluation into a rewrite brief: objective, must-fix items, recommended items, and what should stay the same.
2. Prioritize trigger text, main workflow structure, reference splitting, broken links, and large budget items.
3. If the user wants direct file edits, follow the project’s coding workflow and user confirmation rules first.
4. After rewriting, re-run the same checks.

Output focus: Produce an actionable rewrite brief instead of vague advice.

8. Custom Scoring Dimensions

Trigger phrases: “custom scoring rules”, “add our team’s rubric”, “design a review dimension”.

Workflow steps:

1. Clarify the target: single skill, skill package, pure instruction skill, API integration skill, or MCP/CLI skill.
2. Define the smallest useful rule set: checks, metrics, severity, evidence source, and pass criteria.
3. Keep the rules stable and comparable across runs; do not let custom dimensions overwrite the core summary.
4. If the user wants visualization, define the fields and meaning only; do not fabricate measured data.

Output focus: Give a custom scoring table and an example result shape.

9. Real Measurement Guidance

Trigger phrases: “real token usage”, “benchmark”, “run the eval”, “how far off is the estimate”.

Workflow steps:

1. Explain that a pure instruction skill cannot directly perform runtime measurement.
2. Provide a local measurement flow: representative scenarios → dry run → actual run → record usage → feed results back into the report.
3. Separate static estimates, human observation, and runtime logs.
4. If the user has no local tooling or execution permissions, provide only the measurement plan and the missing data checklist.

Output focus: Give a measurement plan, not fake benchmark results.

Report Output Format

Use this default report structure:

## At a Glance
| Item | Band / Conclusion | Evidence |
|------|-------------------|----------|

## Why It Matters
- Explain how the issue affects trigger accuracy, context cost, maintainability, or user outcomes.

## Fix First
| Priority | Severity | Evidence | Smallest Fix |
|----------|----------|----------|--------------|

## Recommended Next Step
- Give one smallest actionable next step.

Execution Priorities

1. Evidence first: every important conclusion should cite the files and line numbers already read.
2. Do not guess: unread files, unrun commands, and missing logs must be marked as unknown.
3. Separate static review from real measurement; never present an estimate as a measured result.
4. Do not invent exact scores or token counts; use bands and reasons when exact counting is not reliable.
5. Lead with the conclusion and the first fix before expanding into details.
6. Suggest only the smallest change needed for the current issue.

References

For detailed scoring rules, thresholds, report templates, and rewrite-brief templates, read:

• references/eval-rubric.md

Communication Rules

• Default to English unless the user explicitly asks for another language.
• Start with overall risk and the first fix, then explain why.
• Keep jargon light for non-technical users; define terms briefly when needed.
• Stay direct and objective for skill authors; do not turn preferences into hard failures.
• End with a next step so the user knows what to do next.

Common Pitfalls

• Treating static banding as a reproducible exact score.
• Evaluating without reading the files.
• Forgetting to check trigger coverage and description length.
• Ignoring broken reference paths.
• Packing long templates into SKILL.md and making invocation heavier.
• Requiring a real runtime for a pure instruction skill without stating the boundary.