Research Log — ai-layer-kit/CLAUDE-root.md

Research Log — ai-layer-kit/CLAUDE-root.md

Target: production-lines/digital-talent/templates/ai-layer-kit/CLAUDE-root.md Started: 2026-05-26

Iteration #0 — 2026-05-26 (baseline)

Score: 9/12 Status: baseline (no changes)

Failing criteria

  • Q2 — Section "How this talent is structured" was 13 lines (over the 12-line cap).
  • Q7 — Line 9 referenced .claude/CLAUDE-capabilities.md, a file the kit does not create. Fabricated path.
  • Q10 — Most placeholders had no inline hint (8 of 10), leaving Pablo guessing what to fill.

Iteration #1 — 2026-05-26

Score: 12/12 (was 9/12, +3) Status: Kept

Changes applied

  1. Fixed Q7 (fabricated path) — Removed the .claude/CLAUDE-capabilities.md pointer. Replaced with the truthful "Skills available: see .claude/commands/." This also tightened the Identity section by one line.
  2. Fixed Q10 (no hints) — Added inline e.g., … hints to every fill-slot placeholder: {Product Name}, {product-type}, {client-name}, {language} (both occurrences), {domain-summary}, {Domain 2}, {domain-1}, {domain-2}. Pattern-reference uses of {domain} left bare (they document the convention, not fill-slots).
  3. Fixed Q2 (over-12 section) — Compressed "How this talent is structured": dropped the redundant "Five components:" intro sentence (the table speaks for itself), shortened the closing line ("Reference copy at … — do not edit there. Live changes go in the files above"). Section now 11 lines.

Criteria results

# Question (short) Before After Delta
1 < 60 lines YES YES —
2 Section ≤ 12 lines NO YES +1
3 Identity in first 10 YES YES —
4 4 rules canonical labels YES YES —
5 "do not edit" before rules YES YES —
6 Routing explained YES YES —
7 All paths real NO YES +1
8 Kit table = 5 components YES YES —
9 Placeholders consistent YES YES —
10 Hints on placeholders NO YES +1
11 Safety ≥ 3 rules YES YES —
12 Imperatives actionable YES YES —

Observations

  • Fabricated-path bugs are easy to hide in templates. A criterion that explicitly checks "every referenced path is real" caught this in <1 min, where a vibes review would have skimmed past it. Worth porting to other template auto-researches.
  • Inline e.g., … hints are zero-cost. No new lines added, no structural change — just turn {slug} into {slug, e.g., concrete-value}. Should be the default pattern for all factory templates.
  • The "section length cap" criterion drove a real simplification — dropping the "Five components:" intro made the table more authoritative, not less.

Next direction

12/12 reached against this set. Plausible criteria expansions if we want to push further:

  • "Is the file LLM-self-bootstrap-readable?" — i.e., a talent loading only this file (no other context) can act correctly. Would test the rules+safety floor for completeness.
  • "Are placeholder hints in the talent's working language?" — French hints for French talents, English for English. Currently mixed (Architecte EA STM is French, enterprise-architect is English). Both extend the eval, not the doc.