audit-activation
Audits OrchestKit sub-agent activation from real spawn telemetry — computes the generic-vs-specialist spawn split, flags dormant agents (never fired), and classifies each as fires/mis-triggered/niche. The agent-side analogue of audit-skills. Use when specialized agents feel under-used, before pruning the catalog, or after wiring new agent spawn paths.
pinned to #0e54dbcupdated 2 months ago
Ask your AI client: “install skills/audit-activation”.
Requires the metahub MCP server installed in your client. Set up MCP.
mh install skills/audit-activationmetahub onboarded this repo on the author's behalf.
If you own github.com/yonatangross/orchestkit on GitHub, claim the listing to take over publishing. Your claim preserves the existing eval history and badges; only the curator label is replaced with verified-publisher on your next publish.
Stars
201
Last commit
2 months ago
Latest release
published
- #agent-orchestration
- #agents
- #ai-agents
- #ai-development
- #anthropic
- #claude-code
- #claude-code-plugin
- #claude-plugin
- #developer-tools
- #fastapi
- #langgraph
- #llm
- #mcp
- #rag
- #react
- #security
- #testing
- #typescript
About this skill
Pulled from SKILL.md at publish time.
Reports whether OrchestKit's specialized sub-agents are actually being activated, from real spawn telemetry — not vibes. The agent-side analogue of audit-skills (which audits skill quality; this audits agent activation).
Automated checks the publisher passed at publish time — structure, docs, safety, and whether the artifact behaves as claimed.0e54dbc· 2 months ago
Behavioral checks ran but aren't published for this artifact; the static checks above ran at publish time.
Kind-specific
3 passed1 warningSkill: triggers declaredwarn
No `trigger` phrases in SKILL.md frontmatter
Add `trigger:` lines so Claude knows when to activate this skill — e.g. `when building MCP servers` or `for diagram creation`.
Skill: SKILL.md present
found at src/skills/audit-activation/SKILL.md · frontmatter source: SKILL.md
Skill: body content present
514 words · 4,168 chars · 7 sections · 2 code blocks
Skill: allowed-tools scope
no allowed-tools restriction (Claude may use anything)
Release history
1- releasecurrent0e54dbcwarn2 months ago
Contents
Reports whether OrchestKit's specialized sub-agents are actually being activated, from real spawn telemetry — not vibes. The agent-side analogue of audit-skills (which audits skill quality; this audits agent activation).
It answers: "Do my specialized agents get spawned, or does the model default to generic Explore/general-purpose?" Read-only — it never edits agents.
Why this exists: a 2026-06 audit found only ~14% of agent spawns hit the 37-agent catalog vs ~74% generic, with 17/37 agents dormant — and that agents fire ~1:1 with how often a high-traffic skill names them via
subagent_type=. Description rewrites ("use proactively") were A/B-tested and gave Δ0, so this skill measures wiring + usage, not description prose. Seedocs/feat--activation-audit/.
Quick Reference
| Category | File | Impact | When to Use |
|---|---|---|---|
| Activation Checks | ${CLAUDE_SKILL_DIR}/rules/activation-checks.md | HIGH | What to compute per agent |
| Classification | ${CLAUDE_SKILL_DIR}/rules/activation-status.md | HIGH | fires / mis-triggered / niche / dead buckets |
| Output Format | ${CLAUDE_SKILL_DIR}/references/output-format.md | MEDIUM | Report layout + the spawn-split summary |
CRITICAL: Task Management is MANDATORY (CC 2.1.16)
TaskCreate(subject="Audit activation: agent spawn analysis",
description="Computing generic-vs-specialist split + dormancy from spawn telemetry",
activeForm="Auditing agent activation")
TaskCreate(subject="Read spawn telemetry", activeForm="Reading subagent-spawns.jsonl")
TaskCreate(subject="Compute split + dormancy", activeForm="Computing split and dormant agents")
TaskCreate(subject="Classify & render", activeForm="Classifying agents and rendering report")
TaskUpdate(taskId="3", addBlockedBy=["2"])
TaskUpdate(taskId="4", addBlockedBy=["3"])
Workflow
- Inventory — Glob
src/agents/*.md(exclude README/INDEX/CONTRIBUTING) for the catalog. - Read telemetry — the FRESH stream is
.claude/logs/subagent-spawns.jsonl(writers:pretool/task/spawn-intent-logger+subagent-start/subagent-validator). The legacy~/.claude/analytics/agent-usage.jsonlis dead (orphaned in a refactor) — see edge cases. - Compute — run all checks from
Read("${CLAUDE_SKILL_DIR}/rules/activation-checks.md"): spawn split (generic / ork-catalog / other-plugin), per-agent fire counts, never-fired set, concentration (top-5 %). - Classify — apply
Read("${CLAUDE_SKILL_DIR}/rules/activation-status.md"): each dormant agent → mis-triggered (wired but no real spawn path) / niche (legit rare) / dead (no references anywhere). - Render — output using
Read("${CLAUDE_SKILL_DIR}/references/output-format.md").
Quick Start
bash "${CLAUDE_SKILL_DIR}/scripts/run-activation-audit.sh"
The script reads the spawn telemetry, joins it against src/agents/, and prints the split, top agents, never-fired list, and concentration. Then apply the classification rules to bucket the dormant agents (this step needs the model: it greps skills/agents for each dormant agent's real spawn path).
Key Decisions
| Decision | Recommendation |
|---|---|
| Telemetry source | Use .claude/logs/subagent-spawns.jsonl (fresh); ignore the dead agent-usage.jsonl |
| "never fired" caveat | Absence in the telemetry window is a strong signal, NOT proof of zero — state the window |
| Fix direction | Dormancy ⇒ wire a subagent_type= spawn from a high-traffic skill; do NOT rewrite descriptions (A/B Δ0) |
| Prune threshold | Only "dead" (no references in any skill/agent) is a prune candidate; niche stays |
Chain
After this audit, run the deeper experiment if you suspect descriptions: docs/feat--activation-audit/agent-routing-experiment.mjs (isolated A/B selection harness).
Related Skills
audit-skills— quality audit for skills (this skill's sibling, agent-side)telemetry-inspect— validates the telemetry data-plane this skill reads fromanalytics— raw usage queries across projectsdoctor— broader plugin health (manifests, hooks, memory budget)
Reviews
No reviews yet. Be the first.
Related
Test-Driven Development
Red → green → refactor discipline for any feature or bugfix
Verification Before Completion
Evidence before assertions, always
Writing Plans
Turn specs into phased implementation plans
mh install skills/audit-activation