mirror of https://github.com/jparkerweb/plan2code.git synced 2026-07-21 18:33:22 -07:00

Files

T

jparkerweb 68542fd778 Release v1.16.0: install handoff skill, add publish skill, sync upstream

- Make plan2code-handoff an installed workflow prompt: move to src/,
  register in install.js SOURCE_PROMPTS, remove repo-local-only copy
- Add repo-local maintainer /plan2code-publish skill (cuts GitHub Releases;
  excluded from install.js)
- Review workflow next-step suggestion made context-aware (upstream v1.15.4)
- Doc updates: list handoff in README, QUICK-REFERENCE, architecture table;
  add "Adding a New Workflow Prompt / Skill" checklist to dev-commands
- Fix broken AGENTS.md index links: rename .agents-docs loop/metrics files
  to plan2code-* to match references
- Bump version.json to 1.16.0 to align with package.json and CHANGELOG

2026-07-20 22:31:37 -07:00

3.3 KiB

Raw Permalink Blame History

Plan2Code Metrics

Part of AGENTS.md — project guidance for AI coding agents.

A recursive self-improvement toolchain for plan2code contributors. Collects run metrics, aggregates by prompt generation, diagnoses weak steps via AI, and proposes surgical prompt edits.

Metrics Data Flow

Collect → Aggregate → Analyze → Improve → Apply

Collector reads project artifacts (specs/<feature>/) → writes RunMetrics JSON per run
Aggregator groups runs by prompt SHA fingerprint (cohorts) → aggregated.json
Analyzer invokes AI with aggregated metrics + prompt contents → diagnosis markdown
Improver invokes AI with diagnosis → validated PromptEdit[] proposals (char limit + verbatim checks)
Applier shows interactive diffs → patches src/plan2code-*.md files

Metrics Commands

cd plan2code-metrics && npm run build  # Build the CLI
plan2code-metrics                      # Run (fully interactive, no flags)

Option	Action
Collect	Read spec artifacts → run JSON
Import	Copy run JSON from another project
View	Display cohort metrics with health indicators
Analyze	AI diagnosis of weak metrics
Propose	AI improvement proposals with validation
Apply	Interactive diff review + file patching

Key Source Files

File	Purpose
`types.ts`	All interfaces (`RunMetrics`, `UserFeedback`, `CohortMetrics`, etc.) + `METRIC_TARGETS`
`collector.ts`	Reads project artifacts → run JSON (parses plan drafts, overview.md, loop logs)
`aggregator.ts`	Merges runs by prompt generation (SHA cohort) → `aggregated.json`
`analyzer.ts`	AI diagnosis via `prompts/analyze.md` template
`improver.ts`	AI proposals via `prompts/improve.md` + validation (char count, old_text match)
`applier.ts`	Interactive diff review + file patching
`cli.ts`	Menu-driven interactive CLI (100% prompts, no flags)
`invoke-llm.ts`	Unified LLM interface (Claude Code or Copilot CLI)

User Feedback

The collector parses an optional ## User Feedback table from overview.md:

## User Feedback
| Field | Value |
|-------|-------|
| Rating | 8 |
| Reason | Smooth workflow |
| Went Well | Planning was thorough |
| Went Poorly | Some tasks unclear |

Feedback is collected during finalize (Step 5) or retroactively via the CLI. Pipe characters in values are escaped as \|. The aggregator computes avg_user_rating and feedback_count per cohort.

Supported AI Agents

Claude Code (recommended): claude CLI with --inputFile for prompt delivery
GitHub Copilot CLI: copilot CLI with stdin prompt delivery

Metric Targets

Metric	Target	Direction
`avg_confidence`	≥ 90	higher is better
`avg_task_completion_rate`	≥ 0.95	higher is better
`avg_blocker_count`	≤ 1.5	lower is better
`avg_completion_marker_success_rate`	≥ 0.95	higher is better
`avg_verification_failures_found`	≤ 1.0	lower is better
`archival_success_rate`	≥ 0.99	higher is better
`avg_user_rating`	≥ 7.0	higher is better

Data stored in .plan2code-metrics/ (runs/, aggregated.json, proposals/).

3.3 KiB Raw Permalink Blame History