Target Workflow: Documentation Maintainer

Source report: #4368
Run analyzed: §26998493846
Estimated cost per run: $0.17
Total tokens per run: ~664K (668,295 raw)
Cache read rate: 88.4% (590,727 tokens)
Cache write rate: 10.4% (69,233 tokens)
Net new input: 0.7% (4,364 tokens)
LLM turns: 11 (all exploratory — zero writes)
Conclusion: ❌ failure (100% failure rate — only 1 run observed)

Current Configuration

Root Cause: Pre-Computed Context Not Consumed

The pre-agent steps: block builds a context.md file with git diffs and affected docs. The agent's prompt explicitly says:

"Use the Recent Git Diffs section from that file as your sole source for recent source changes. Do not run any git commands"

Yet the agent ran:

• bash_git log --since="7 days ago" — exactly the command the pre-step already ran
• bash_wc -l docs/api-proxy... — exploring instead of acting

This means the agent either: (a) could not find or read the pre-computed context, or (b) found it empty on the first run of this new workflow. Either way, the agent spent 11 turns in exploration and never reached the edit or PR-creation phase, causing the failure.

Recommendations

1. Add context validation gate in pre-agent step

Estimated savings: 50-60% cost reduction ($0.10/run, eliminating failure waste)

The pre-agent step should validate its output and fail fast if context is empty — preventing the agent from running 11 exploratory turns for nothing.

Add to the Build documentation maintainer context step in doc-maintainer.md:

# After writing context.md — validate it has content
CONTEXT_SIZE=$(wc -c < "$CONTEXT_DIR/context.md")
if [ "$CONTEXT_SIZE" -lt 100 ]; then
  echo "::warning::Context file is empty or minimal ($CONTEXT_SIZE bytes). Skipping agent run."
  echo "skip_agent=true" >> "$GITHUB_OUTPUT"
fi

Then add if: steps.context.outputs.skip_agent != 'true' on the agent job. This prevents wasted agent turns on no-op runs.

2. Add explicit failure assertion in context step

Estimated savings: Eliminates 100% of failure-mode runs

The git log --since="7 days ago" in the pre-step may produce no output if this is a freshly checked-out repo or the workflow hasn't run before. The context.md could be skeletal.

Add a debug echo to confirm what context.md contains:

echo "Context size: $(wc -c < "$CONTEXT_DIR/context.md") bytes"
echo "Diff lines: $(wc -l < "$CONTEXT_DIR/recent-diffs.txt")"
cat "$CONTEXT_DIR/affected-docs.txt" || echo "(empty)"

This surfaces failures in the workflow logs before the expensive agent turn.

3. Reduce max-turns from 10 to 5

Estimated savings: 19% cost reduction ($0.032/run)

With working pre-computed context, the agent needs at most:

1. Read context.md (pre-computed)
2. Read 1-3 affected doc files
3. Apply edits
4. Confirm and exit

A max of 5 turns is sufficient. Fewer turns = fewer cache-read accumulations (590K cache reads across 11 turns → ~270K for 5 turns).

engine:
  id: claude
  model: claude-haiku-4-5
  max-turns: 5   # was: 10

4. Trim base context snapshot tokens (cache write reduction)

Estimated savings: ~$0.04-0.06/run (30-50% of cache write cost = $0.087)

The 69,233 cache-write tokens include AGENTS.md custom instruction (~2,500 tokens) and a large base snapshot with 100+ repo files (workflow .md files, agents, skills, etc.). The doc maintainer only needs:

• The source files it edits (docs/, *.md)
• The AGENTS.md custom instruction (required)

It does not need all .github/workflows/*.md files, skill READMEs, etc. to be pre-loaded in base context. Review .github/aw/actions-lock.json to see if the base snapshot scope can be narrowed for this workflow.

5. Investigate bash: false inconsistency

Estimated savings: 0 tokens, but improves reliability

The workflow declares bash: false yet the agent successfully ran bash commands (git log, wc -l). This inconsistency should be investigated:

• If bash is genuinely disabled, these tool calls should have failed → the pre-step context is critical
• If bash is leaking through a different mechanism, tighten tool restrictions to prevent the agent from re-running deterministic work

Cache Analysis (Anthropic-Specific)

Cache write dominates cost at 51%. Every run re-writes the full base context because there is zero cross-run cache reuse (Anthropic TTL is ~5 minutes; this workflow runs once per day). Reducing the base context size (Rec #4) is the most durable per-run cost reduction.

Cache write amortization: Turn 1 writes ~65K tokens to cache; Turns 2–11 each read that cache at 0.1× the write cost. With 11 turns: $0.087 write / 10 reads = $0.0087/turn write amortization vs $0.0059/turn read cost — marginally net positive. However, since all 11 turns were exploratory failures, every cent was wasted. Fixing the failure is the highest ROI action.

Expected Impact

Implementation Checklist

• Add context validation + skip_agent output to the Build documentation maintainer context step in .github/workflows/doc-maintainer.md
• Add debug echo to confirm context.md content before agent runs
• Reduce max-turns: 10 → max-turns: 5 in doc-maintainer.md
• Verify /tmp/gh-aw/doc-maintainer-context/context.md is non-empty on a real run (check workflow-logs for pre-step output in run §26998493846)
• Review base snapshot scope in .github/aw/actions-lock.json for this workflow — remove unused workflow files from context
• Investigate bash: false tool setting not preventing bash calls
• Recompile: gh aw compile .github/workflows/doc-maintainer.md
• Post-process: npx tsx scripts/ci/postprocess-smoke-workflows.ts
• Verify CI passes on PR
• Compare token usage on new run vs baseline (target: <300K tokens, 4-5 turns, success)

Generated by Daily Claude Token Optimization Advisor · sonnet46 1.9M · ◷