mirror of
https://github.com/jparkerweb/plan2code.git
synced 2026-07-21 18:33:22 -07:00
Release v1.16.0: install handoff skill, add publish skill, sync upstream
- Make plan2code-handoff an installed workflow prompt: move to src/, register in install.js SOURCE_PROMPTS, remove repo-local-only copy - Add repo-local maintainer /plan2code-publish skill (cuts GitHub Releases; excluded from install.js) - Review workflow next-step suggestion made context-aware (upstream v1.15.4) - Doc updates: list handoff in README, QUICK-REFERENCE, architecture table; add "Adding a New Workflow Prompt / Skill" checklist to dev-commands - Fix broken AGENTS.md index links: rename .agents-docs loop/metrics files to plan2code-* to match references - Bump version.json to 1.16.0 to align with package.json and CHANGELOG
This commit is contained in:
@@ -0,0 +1,82 @@
|
||||
# Plan2Code Metrics
|
||||
> Part of [AGENTS.md](../AGENTS.md) — project guidance for AI coding agents.
|
||||
|
||||
A recursive self-improvement toolchain for plan2code contributors. Collects run metrics, aggregates by prompt generation, diagnoses weak steps via AI, and proposes surgical prompt edits.
|
||||
|
||||
## Metrics Data Flow
|
||||
|
||||
```
|
||||
Collect → Aggregate → Analyze → Improve → Apply
|
||||
```
|
||||
|
||||
1. **Collector** reads project artifacts (`specs/<feature>/`) → writes `RunMetrics` JSON per run
|
||||
2. **Aggregator** groups runs by prompt SHA fingerprint (cohorts) → `aggregated.json`
|
||||
3. **Analyzer** invokes AI with aggregated metrics + prompt contents → diagnosis markdown
|
||||
4. **Improver** invokes AI with diagnosis → validated `PromptEdit[]` proposals (char limit + verbatim checks)
|
||||
5. **Applier** shows interactive diffs → patches `src/plan2code-*.md` files
|
||||
|
||||
## Metrics Commands
|
||||
|
||||
```bash
|
||||
cd plan2code-metrics && npm run build # Build the CLI
|
||||
plan2code-metrics # Run (fully interactive, no flags)
|
||||
```
|
||||
|
||||
## Metrics CLI Menu
|
||||
|
||||
| Option | Action |
|
||||
|--------|--------|
|
||||
| Collect | Read spec artifacts → run JSON |
|
||||
| Import | Copy run JSON from another project |
|
||||
| View | Display cohort metrics with health indicators |
|
||||
| Analyze | AI diagnosis of weak metrics |
|
||||
| Propose | AI improvement proposals with validation |
|
||||
| Apply | Interactive diff review + file patching |
|
||||
|
||||
## Key Source Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `types.ts` | All interfaces (`RunMetrics`, `UserFeedback`, `CohortMetrics`, etc.) + `METRIC_TARGETS` |
|
||||
| `collector.ts` | Reads project artifacts → run JSON (parses plan drafts, overview.md, loop logs) |
|
||||
| `aggregator.ts` | Merges runs by prompt generation (SHA cohort) → `aggregated.json` |
|
||||
| `analyzer.ts` | AI diagnosis via `prompts/analyze.md` template |
|
||||
| `improver.ts` | AI proposals via `prompts/improve.md` + validation (char count, old_text match) |
|
||||
| `applier.ts` | Interactive diff review + file patching |
|
||||
| `cli.ts` | Menu-driven interactive CLI (100% prompts, no flags) |
|
||||
| `invoke-llm.ts` | Unified LLM interface (Claude Code or Copilot CLI) |
|
||||
|
||||
## User Feedback
|
||||
|
||||
The collector parses an optional `## User Feedback` table from `overview.md`:
|
||||
|
||||
```markdown
|
||||
## User Feedback
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Rating | 8 |
|
||||
| Reason | Smooth workflow |
|
||||
| Went Well | Planning was thorough |
|
||||
| Went Poorly | Some tasks unclear |
|
||||
```
|
||||
|
||||
Feedback is collected during finalize (Step 5) or retroactively via the CLI. Pipe characters in values are escaped as `\|`. The aggregator computes `avg_user_rating` and `feedback_count` per cohort.
|
||||
|
||||
## Supported AI Agents
|
||||
|
||||
- **Claude Code** (recommended): `claude` CLI with `--inputFile` for prompt delivery
|
||||
- **GitHub Copilot CLI**: `copilot` CLI with stdin prompt delivery
|
||||
|
||||
## Metric Targets
|
||||
|
||||
| Metric | Target | Direction |
|
||||
|--------|--------|-----------|
|
||||
| `avg_confidence` | ≥ 90 | higher is better |
|
||||
| `avg_task_completion_rate` | ≥ 0.95 | higher is better |
|
||||
| `avg_blocker_count` | ≤ 1.5 | lower is better |
|
||||
| `avg_completion_marker_success_rate` | ≥ 0.95 | higher is better |
|
||||
| `avg_verification_failures_found` | ≤ 1.0 | lower is better |
|
||||
| `archival_success_rate` | ≥ 0.99 | higher is better |
|
||||
| `avg_user_rating` | ≥ 7.0 | higher is better |
|
||||
|
||||
Data stored in `.plan2code-metrics/` (runs/, aggregated.json, proposals/).
|
||||
Reference in New Issue
Block a user