mirror of
https://github.com/jparkerweb/plan2code.git
synced 2026-07-21 18:33:22 -07:00
plan2code-bot: replace auto-responder with LLM-as-judge evaluation
Bot now acts as an authentic QA agent instead of rubber-stamping every question and step. - intelligent-responder answers AskUserQuestion via LLM using current observations (tools used, files created, errors) instead of keyword matching; auto-responder removed. - evaluator runs after each step with step-specific criteria (prompts/evaluation-criteria.ts), produces 0-100 score plus strengths/weaknesses/critical issues. Avg <60 blocks finalization. - observation-collector captures tool_use, file writes, errors, and question reasoning; session-runner falls back to scraping tool_use blocks when canUseTool doesn't fire and dedupes both sources. - Bot writes BOT-EVALUATION.md and BOT-NOTES.md for metrics analysis. - Idea generator: 12 categories instead of CLI/web-app coin flip, stronger seed adherence, less developer-tool bias. - Init step now writes a minimal AGENTS.md stub instead of running the full init skill, avoiding hallucinated architecture before plan. - Implement step uses Read/Write/Edit/Glob/Grep directly instead of a Skill sub-session that produced no visible tool observations. - bin: add --help, accept --idea="value" form, strip surrounding quotes; evaluator maxTurns 3 -> 30; warn when parser misses SCORE.
This commit is contained in:
@@ -2,6 +2,7 @@
|
||||
|
||||
## 1.1.0
|
||||
|
||||
### Resume support
|
||||
- Add `--resume` flag to continue incomplete runs from saved state
|
||||
- Skip previously succeeded steps when resuming (init, plan, document, implement, finalize)
|
||||
- Restore idea name, description, project directory, and implement pass counter from state
|
||||
@@ -11,6 +12,31 @@
|
||||
- Add `deleteState()` and `findExistingState()` utilities to bot-state module
|
||||
- Add bot-state unit tests (saveState, loadState, deleteState, findExistingState)
|
||||
|
||||
### Idea generation improvements
|
||||
- Expand idea categories from binary CLI/web-app coin flip to 12 diverse categories (games, dashboards, browser extensions, desktop utilities, etc.)
|
||||
- Add guidance to avoid defaulting to developer-centric tools (git analyzers, code formatters)
|
||||
- Strengthen `--idea` seed clause so the LLM stays aligned with the user's theme instead of ignoring it
|
||||
- Update system prompt to encourage creative, cross-domain ideas
|
||||
|
||||
### Init step overhaul (new projects)
|
||||
- Init now creates a minimal AGENTS.md stub (name, description, status) instead of running the full `/plan2code-init` skill
|
||||
- Prevents hallucinated architecture, commands, and `.agents-docs/` files before the plan step runs
|
||||
- Init evaluation criteria updated to reward minimalism and penalize premature detail
|
||||
|
||||
### Implement step overhaul
|
||||
- Implement step now works directly with Read/Write/Edit/Glob/Grep tools instead of delegating to Skill sub-session
|
||||
- Inlined step-by-step process: find specs, pick phase, implement tasks, mark checkboxes
|
||||
- Fixes issue where Skill sub-sessions did all work invisibly, causing zero tool observations
|
||||
|
||||
### Observation tracking fix
|
||||
- Capture `tool_use` blocks from the assistant message stream in session-runner as a fallback when `canUseTool` callback doesn't fire
|
||||
- Add deduplication in ObservationCollector to prevent double-counting from both sources
|
||||
- Fixes all steps reporting 0 tools used / 0 files created in BOT-NOTES and evaluations
|
||||
|
||||
### Evaluator improvements
|
||||
- Increase evaluator `maxTurns` from 3 to 30 so it has room for tool calls before producing the scored response
|
||||
- Add warning log when evaluation parser can't find SCORE in output (was silently defaulting to 50)
|
||||
|
||||
## 1.0.0
|
||||
|
||||
- Initial release
|
||||
|
||||
Reference in New Issue
Block a user