mirror of
https://github.com/jparkerweb/plan2code.git
synced 2026-07-21 18:33:22 -07:00
plan2code-bot: replace auto-responder with LLM-as-judge evaluation
Bot now acts as an authentic QA agent instead of rubber-stamping every question and step. - intelligent-responder answers AskUserQuestion via LLM using current observations (tools used, files created, errors) instead of keyword matching; auto-responder removed. - evaluator runs after each step with step-specific criteria (prompts/evaluation-criteria.ts), produces 0-100 score plus strengths/weaknesses/critical issues. Avg <60 blocks finalization. - observation-collector captures tool_use, file writes, errors, and question reasoning; session-runner falls back to scraping tool_use blocks when canUseTool doesn't fire and dedupes both sources. - Bot writes BOT-EVALUATION.md and BOT-NOTES.md for metrics analysis. - Idea generator: 12 categories instead of CLI/web-app coin flip, stronger seed adherence, less developer-tool bias. - Init step now writes a minimal AGENTS.md stub instead of running the full init skill, avoiding hallucinated architecture before plan. - Implement step uses Read/Write/Edit/Glob/Grep directly instead of a Skill sub-session that produced no visible tool observations. - bin: add --help, accept --idea="value" form, strip surrounding quotes; evaluator maxTurns 3 -> 30; warn when parser misses SCORE.
This commit is contained in:
@@ -20,15 +20,37 @@ export function buildStepPrompt(step: StepName, config: BotConfig): string {
|
||||
function buildInitPrompt(config: BotConfig): string {
|
||||
return `${AUTONOMOUS_PREAMBLE}
|
||||
|
||||
Run the /plan2code-init skill to generate an AGENTS.md file for this project.
|
||||
This is a brand new project with no existing code. Create a minimal stub AGENTS.md file and an IDEA.md file. Do NOT run the /plan2code-init skill � there is no codebase to analyze yet.
|
||||
|
||||
The project idea is: ${config.ideaDescription}
|
||||
The project name is: ${config.ideaName}
|
||||
|
||||
When asked questions:
|
||||
- For the project description, use the idea above
|
||||
- Accept all defaults and approve all gates
|
||||
- Complete the init process fully`;
|
||||
## What to create
|
||||
|
||||
### AGENTS.md
|
||||
Create a minimal stub with ONLY the following � do NOT invent architecture, tech stack details, commands, or file structures:
|
||||
|
||||
\`\`\`markdown
|
||||
# AGENTS.md
|
||||
|
||||
This file provides guidance to AI coding agents when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
**Name:** ${config.ideaName}
|
||||
**Description:** ${config.ideaDescription}
|
||||
|
||||
## Status
|
||||
This project is in the planning phase. Architecture, commands, and detailed documentation will be added after the plan and document steps are complete.
|
||||
\`\`\`
|
||||
|
||||
### IDEA.md
|
||||
If IDEA.md does not already exist, create it with the project name and description.
|
||||
|
||||
## Rules
|
||||
- Do NOT create .agents-docs/ or any detail files � there is nothing to document yet
|
||||
- Do NOT hallucinate architecture, dependencies, file structures, or tech stack choices
|
||||
- Do NOT install dependencies or scaffold project files
|
||||
- ONLY create the two files above`;
|
||||
}
|
||||
|
||||
function buildPlanPrompt(config: BotConfig): string {
|
||||
@@ -63,14 +85,37 @@ When making decisions:
|
||||
function buildImplementPrompt(config: BotConfig): string {
|
||||
return `${AUTONOMOUS_PREAMBLE}
|
||||
|
||||
Run the /plan2code-3-implement skill to implement the next available phase.
|
||||
You are a senior software engineer implementing a project phase. Do NOT use the Skill tool � implement directly using Read, Write, Edit, Glob, and Grep tools.
|
||||
|
||||
When making decisions:
|
||||
- Pick the first available/unchecked phase
|
||||
- Write working code that fulfills the spec tasks
|
||||
- Mark tasks as complete as you finish them
|
||||
- Approve and sign off when the phase is done
|
||||
- Skip running tests if asked (or select the simplest test option)`;
|
||||
## Process
|
||||
|
||||
1. **Find the spec**: Read \`specs/*/overview.md\` to find the phase checklist
|
||||
2. **Pick the next phase**: Find the first phase marked \`[ ]\` (pending) or \`[/]\` (in-progress)
|
||||
3. **Read the phase file**: Read the corresponding \`phase-X.md\` from the same directory
|
||||
4. **Mark phase in-progress**: Update \`[ ]\` to \`[/]\` in overview.md
|
||||
5. **Implement each task sequentially**:
|
||||
- Read the task specification completely
|
||||
- Write the code using Write or Edit tools � create real files, not code blocks
|
||||
- Mark the task \`[x]\` in the phase file immediately after completing it
|
||||
6. **Complete the phase**: After all tasks, fill in the "Phase Completion Summary" in the phase file
|
||||
7. **Mark phase complete**: Update \`[/]\` to \`[x]\` in overview.md
|
||||
|
||||
## Rules
|
||||
|
||||
- Follow AGENTS.md if it exists
|
||||
- Implement specs EXACTLY � no creative additions or unsolicited improvements
|
||||
- Write task completion status (\`[x]\`) to disk immediately after each task � never batch
|
||||
- Only create files mentioned in the spec tasks
|
||||
- Use the specified file paths, function names, and structures from the spec
|
||||
- No placeholder code � fully implement every function
|
||||
- Match existing codebase conventions
|
||||
- Do NOT run git commands
|
||||
- Skip running tests unless explicitly listed as a phase task
|
||||
|
||||
## Project info
|
||||
- Project: ${config.ideaName}
|
||||
- Description: ${config.ideaDescription}
|
||||
- Project directory: ${config.projectDir}`;
|
||||
}
|
||||
|
||||
function buildFinalizePrompt(config: BotConfig): string {
|
||||
|
||||
Reference in New Issue
Block a user