diff --git a/.agents-docs/AGENTS-architecture-skill-scripts.md b/.agents-docs/AGENTS-architecture-skill-scripts.md index 6aa2585..6f5a741 100644 --- a/.agents-docs/AGENTS-architecture-skill-scripts.md +++ b/.agents-docs/AGENTS-architecture-skill-scripts.md @@ -25,7 +25,7 @@ Anything a workflow step decides with one correct answer (parsing a spec's check | `pr-prep.mjs` (own) | pr-prep | `template` (the PR template and its headings), `pr-url` (the create-a-PR link through `git-repo.mjs`) and `spec` (the branch's spec folder for the save offer). Read-only | | `coverage.mjs` (own) | test | `read --lcov` / `--go-func`: coverage output as per-file lines, hits and percent, plus the total. Read-only, runs no tool | | `dead-code.mjs` (own) | dead-code | `detect` (the dead-code tools the project already has, languages, file counts), `run --tool` (a tool's output as candidates), `narrow` (folders ranked by candidates), `keep list\|filter\|add\|remove` (the shared keep list, `.plan2code/dead-code/keep.md`; rows matched by file, name and the hash of the item's declaration line, `#n` for the n-th identical line, `file:` for a whole file; `remove` takes `--hash` or `--line` and exits 2 `ambiguous-entry` when two rows share a name), `backup` and `restore --verify` (copies outside the project and a checked put-back), `report write` (`.plan2code/dead-code/reports/DEAD-CODE-.md`; `tool` one id or a list of ids) and `gitignore-check` / `gitignore-add`. Writes only code removals the agent makes, `keep.md` and, on a yes, a report and the `.gitignore` line | -| `deps.mjs` (own) | deps | One file in fixed sections, one adapter per ecosystem behind one interface (npm, pnpm and Yarn; Cargo; pip; uv; Go; .NET): `detect` (lockfiles, members, missing tools, the run folder under the OS temp folder), `scan` (rows, held, notScanned, stillOpen; a declared runtime it cannot cut by sets `runtimeNotChecked`; a missing or failed audit sets `notAudited`), `notes` (release-note slices via `gh api` or `fetch`), `label` (the pure FR-6 to FR-9 rules: Safe, Check first, Warning), `batches` (`--picks`, `--split` narrowing), `backup`, `apply` (the package manager's own update; `failed` also on `unpicked`, `overshot` past a runtime-cut target, `undoFailed` or `needsNewerRuntime`), `restore --verify` (byte for byte; a file it cannot put back is `mismatched {path, reason}` with `verified: false`, never a crash), `handoff` (`specs/-upgrade/UPGRADE-.md`, never outside `specs/`) and `cleanup`. Never runs `--force`, an audit fix or a tool install. Writes only manifests and lockfiles (through `apply` and `restore`) and a ticked brief. Tests: `scripts/test-script-deps.mjs` against stub CLIs on a temp PATH and fixtures in `scripts/fixtures/deps/` named with each tool's version | +| `deps.mjs` (own) | deps | One file in fixed sections, one adapter per ecosystem behind one interface (npm, pnpm and Yarn; Cargo; pip; uv; Go; .NET): `detect` (lockfiles, members, missing tools, the run folder under the OS temp folder), `scan` (rows, held, notScanned, stillOpen; a declared runtime it cannot cut by sets `runtimeNotChecked`; a missing or failed audit sets `notAudited`), `notes` (release-note slices via `gh api` or `fetch`), `fit` (the newest version the project's runtime runs, per adapter; a runtime hold with one becomes a pick, `unknown` rows go to the agent), `pins` (each row's exact version written in the project's own files, outside its lockfile and manifests: `pins.json`), `label` (the pure FR-6 to FR-9 rules: Safe, Check first, Warning; `reasons` plus `reasonsPlain` in plain English; a row with a pin is never Safe; a record's `usedIn` lists the files that import the package), `checks` (the checks for the baseline, or one group's targeted checks narrowed by `usedIn` and pins), `baseline` and `compare` (read `run-check.mjs --log` logs, never run checks: tests red before or flaky never blame a group), `batches` (`--picks`, `--split` narrowing), `backup`, `apply` (the package manager's own update; `failed` also on `unpicked`, `overshot` past a runtime-cut target, `undoFailed` or `needsNewerRuntime`), `fix` (one approved replacement in one project file after `apply`: never a lockfile or manifest of the batch, never under `node_modules`, `.git` or `specs`, never binary or over 1 MB, the text found exactly once; the file is backed up into the batch's backup first, so restore puts it back), `restore --verify` (byte for byte, fixed files included and named in `fixed[]`; a file it cannot put back is `mismatched {path, reason}` with `verified: false`, never a crash), `install-tool` (the exact install line of a scanner the scan listed missing; runs nothing, the prompt runs it through `run-check.mjs` after approval), `handoff` (`specs/-upgrade/UPGRADE-.md`, never outside `specs/`) and `cleanup`. Never runs `--force`, an audit fix or a tool install. Writes manifests and lockfiles only through `apply` and `restore`, any other project file only through `fix` after the person approved that exact change, and a ticked brief. Tests: `scripts/test-script-deps.mjs` against stub CLIs on a temp PATH and fixtures in `scripts/fixtures/deps/` named with each tool's version | | `agents-md.mjs` | init, init-update | AGENTS.md structure, budget, orphans, broken links, always-inline sections. With no AGENTS.md yet (Init's run, so said once), `notices` warns when `.gitignore` keeps `specs--completed/` out of git, since archived specs are then not backed up | | `agent-files.mjs` | init, init-update | Detect and replace other agents' config files with AGENTS.md pointers | | `pathfinder.mjs` (own) | 0-pathfinder | `init` (the map in its Step 5 state from `--destination`, `--scores` and `--fog`; refuses an existing map) and `add` (the next `questions/NN-.md` plus its map row), with the references' templates kept as the fallback, then `reconcile`, `frontier`, `claim`, `resolve`, `rule-out`, `trail`, `gate` (which also prints the exact PLAN-DRAFT status line `specs.mjs` reads), `lint`, `brief-data`. Writes rows as `[Name](link): tail` with no em dash, and still reads maps written the old way | diff --git a/scripts/fixtures/deps/logs/cargo.txt b/scripts/fixtures/deps/logs/cargo.txt new file mode 100644 index 0000000..06c1097 --- /dev/null +++ b/scripts/fixtures/deps/logs/cargo.txt @@ -0,0 +1,3 @@ +running 2 tests +test math::adds ... ok +test math::divides ... FAILED diff --git a/scripts/fixtures/deps/logs/dotnet.txt b/scripts/fixtures/deps/logs/dotnet.txt new file mode 100644 index 0000000..1a1dfcc --- /dev/null +++ b/scripts/fixtures/deps/logs/dotnet.txt @@ -0,0 +1,2 @@ + Passed MyApp.Tests.Adds [1 ms] + Failed MyApp.Tests.Divides [3 ms] diff --git a/scripts/fixtures/deps/logs/go.txt b/scripts/fixtures/deps/logs/go.txt new file mode 100644 index 0000000..449db7f --- /dev/null +++ b/scripts/fixtures/deps/logs/go.txt @@ -0,0 +1,5 @@ +=== RUN TestAdds +--- PASS: TestAdds (0.00s) +=== RUN TestDivides +--- FAIL: TestDivides (0.00s) +FAIL example.com/m 0.01s diff --git a/scripts/fixtures/deps/logs/jest.txt b/scripts/fixtures/deps/logs/jest.txt new file mode 100644 index 0000000..8d4d02a --- /dev/null +++ b/scripts/fixtures/deps/logs/jest.txt @@ -0,0 +1,5 @@ +PASS src/a.test.js +FAIL src/b.test.js + math + ✓ adds (2 ms) + ✕ divides by zero (5 ms) diff --git a/scripts/fixtures/deps/logs/node-spec.txt b/scripts/fixtures/deps/logs/node-spec.txt new file mode 100644 index 0000000..3df59bd --- /dev/null +++ b/scripts/fixtures/deps/logs/node-spec.txt @@ -0,0 +1,10 @@ +✔ adds numbers (1.2ms) +✖ parses dates (3.4ms) +ℹ tests 2 +ℹ pass 1 +ℹ fail 1 + +✖ failing tests: + +test at scripts/a.mjs:3:1 +✖ parses dates (3.4ms) diff --git a/scripts/fixtures/deps/logs/pytest.txt b/scripts/fixtures/deps/logs/pytest.txt new file mode 100644 index 0000000..46f8d78 --- /dev/null +++ b/scripts/fixtures/deps/logs/pytest.txt @@ -0,0 +1,3 @@ +tests/test_a.py::test_adds PASSED +FAILED tests/test_b.py::test_divides - ZeroDivisionError: division by zero +==================== 1 failed, 1 passed in 0.12s ==================== diff --git a/scripts/fixtures/deps/logs/tap.txt b/scripts/fixtures/deps/logs/tap.txt new file mode 100644 index 0000000..9dd8876 --- /dev/null +++ b/scripts/fixtures/deps/logs/tap.txt @@ -0,0 +1,9 @@ +TAP version 13 +ok 1 - adds numbers +not ok 2 - parses dates + --- + message: 'expected 2 to equal 3' + ... +not ok 3 - writes the cache # TODO later +ok 4 - reads the cache # SKIP no disk +1..4 diff --git a/scripts/fixtures/deps/logs/unknown.txt b/scripts/fixtures/deps/logs/unknown.txt new file mode 100644 index 0000000..1c8bf27 --- /dev/null +++ b/scripts/fixtures/deps/logs/unknown.txt @@ -0,0 +1,2 @@ +Error: something broke + at main (build.js:1:1) diff --git a/scripts/fixtures/deps/logs/vitest.txt b/scripts/fixtures/deps/logs/vitest.txt new file mode 100644 index 0000000..5561d9a --- /dev/null +++ b/scripts/fixtures/deps/logs/vitest.txt @@ -0,0 +1,2 @@ + ✓ src/a.test.ts (1 test) 2ms + FAIL src/b.test.ts > math > divides by zero diff --git a/scripts/fixtures/deps/pins/project/package-lock.json b/scripts/fixtures/deps/pins/project/package-lock.json new file mode 100644 index 0000000..19a46ed --- /dev/null +++ b/scripts/fixtures/deps/pins/project/package-lock.json @@ -0,0 +1,19 @@ +{ + "name": "pins-fixture", + "version": "1.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "pins-fixture", + "version": "1.0.0", + "devDependencies": { + "playwright-core": "1.64.1" + } + }, + "node_modules/playwright-core": { + "version": "1.64.1", + "dev": true + } + } +} diff --git a/scripts/fixtures/deps/pins/project/package.json b/scripts/fixtures/deps/pins/project/package.json new file mode 100644 index 0000000..cfa3fca --- /dev/null +++ b/scripts/fixtures/deps/pins/project/package.json @@ -0,0 +1,8 @@ +{ + "name": "pins-fixture", + "version": "1.0.0", + "private": true, + "devDependencies": { + "playwright-core": "1.64.1" + } +} diff --git a/scripts/fixtures/deps/pins/project/scripts/test-browser-helper.mjs b/scripts/fixtures/deps/pins/project/scripts/test-browser-helper.mjs new file mode 100644 index 0000000..5c147e5 --- /dev/null +++ b/scripts/fixtures/deps/pins/project/scripts/test-browser-helper.mjs @@ -0,0 +1,6 @@ +// Checks the one browser build the helper downloads. +import { chromium } from "playwright-core"; + +export const BROWSER = "playwright-core@1.64.1"; +// Other versions never count: playwright-core 11.64.10 and 1.64.12 are not the locked one. +export { chromium }; diff --git a/scripts/test-console-wire-budgets.mjs b/scripts/test-console-wire-budgets.mjs index 2959849..5ff6e86 100644 --- a/scripts/test-console-wire-budgets.mjs +++ b/scripts/test-console-wire-budgets.mjs @@ -30,7 +30,7 @@ export const BUDGETS = { notes: { "changelog.md": 1400, "dead-code.md": 4900, - "deps.md": 5700, + "deps.md": 9000, // raised by the pin, newer-Node, baseline and full-suite lines, then the fix card and the install card (8765 bytes) "diagnose.md": 2840, // raised by the lessons card line's bytes only "document.md": 400, "finalize.md": 2000, diff --git a/scripts/test-script-deps.mjs b/scripts/test-script-deps.mjs index dd69b44..1a3ee9b 100644 --- a/scripts/test-script-deps.mjs +++ b/scripts/test-script-deps.mjs @@ -57,7 +57,7 @@ function copyManifest(rel, { crlf = false } = {}) { test("deps --help names every command and every exit code", () => { const r = runDeps(["--help"]); assert.equal(r.code, 0, r.stderr); - for (const command of ["detect", "scan", "notes", "label", "batches", "backup", "apply", "restore", "handoff", "cleanup"]) { + for (const command of ["detect", "scan", "notes", "fit", "pins", "label", "checks", "baseline", "compare", "batches", "backup", "apply", "restore", "handoff", "cleanup"]) { assert.match(r.stdout, new RegExp(`deps\\.mjs ${command}\\b`), command); } for (const exit of ["0 ok", "1 crash", "2 bad arguments", "3 not found", "4 invalid", "5 refused"]) assert.ok(r.stdout.includes(exit), exit); @@ -1084,8 +1084,20 @@ test("scan attaches the test map rows covering a lockfile's members; a Shared ro const { scan } = detectAndScan(root, { npm: npmRules() }, ["--no-audit"]); const byLock = Object.fromEntries(scan.lockfiles.map((l) => [l.lockfile, l.checks])); // The root lockfile resolves the root folder, so every row falls in it. - assert.deepEqual(byLock["package-lock.json"], { rows: ["App", "Web", "Build"], commands: ["npm test -w app", "x", "npm test"], discovered: [], full: true }); - assert.deepEqual(byLock["web/package-lock.json"], { rows: ["Web"], commands: ["x"], discovered: [], full: false }); + // Each command gets a stable id across the scan: the same command in two lockfiles keeps one id. + assert.deepEqual(byLock["package-lock.json"], { + rows: ["App", "Web", "Build"], + commands: ["npm test -w app", "x", "npm test"], + discovered: [], + full: true, + list: [ + { id: "c1", area: "App", run: "npm test -w app", paths: ["packages/app/**"], shared: false }, + { id: "c2", area: "Web", run: "x", paths: ["web/**"], shared: false }, + { id: "c3", area: "Build", run: "npm test", paths: ["scripts/**"], shared: true }, + ], + build: [], + }); + assert.deepEqual(byLock["web/package-lock.json"], { rows: ["Web"], commands: ["x"], discovered: [], full: false, list: [{ id: "c2", area: "Web", run: "x", paths: ["web/**"], shared: false }], build: [] }); assert.equal(scan.noChecks, false); }); @@ -2444,7 +2456,7 @@ test("handoff writes one brief per package in the 4.5 format, every section pres "", "## Why it was held back", "", - "- **chalk**: Warning (major, range ^4.1.0 → ^5.3.0, breaking notes, calls a changed API, used in 2 files)", + "- **chalk**: Warning (a big change, the allowed range widens from ^4.1.0 to ^5.3.0, the notes say something was removed or changed, your code uses something that changed, 2 files use it)", "", "## What the notes say changed", "", @@ -2484,8 +2496,8 @@ test("handoff: a pair shares one brief listing both packages; a scoped name gets assert.match(pair, /^# Upgrade react 18\.3\.1 to 19\.0\.0 and react-dom 18\.3\.1 to 19\.0\.0\n/); for (const section of ["Why it was held back", "What the notes say changed", "Where the code calls it", "Checks that cover it", "How to apply"]) assert.match(pair, new RegExp(`^## ${section}$`, "m"), section); assert.match(pair, /^\*\*Status:\*\* Ready$/m); - assert.match(pair, /- \*\*react\*\*: Warning \(major, .*used in 9 files/, "a major with unreadable notes in 9 files"); - assert.match(pair, /- \*\*react-dom\*\*: Check first \(major/); + assert.match(pair, /- \*\*react\*\*: Warning \(a big change, .*9 files use it/, "a big change with unreadable notes in 9 files"); + assert.match(pair, /- \*\*react-dom\*\*: Check first \(a big change/); assert.match(pair, /### react\n\nThe notes could not be read\./); assert.match(pair, /### react-dom\n\nThe notes were not read\./); assert.match(pair, /npm update react react-dom --package-lock-only/, "one command moves both"); @@ -2535,6 +2547,475 @@ test("handoff refuses an id not in labels.json (exit 3) and writes nothing; upgr assert.equal(deps.upgradeFolder(".."), "package-upgrade"); }); +/* ------------------------------------------------------------ pins */ + +const PINS = fx("pins", "project"); + +/** The pins fixture project copied into a temp folder, with extra files on top. */ +function pinsProject(extra = {}) { + const dir = tmp("deps-pins"); + fs.cpSync(PINS, dir, { recursive: true }); + for (const [rel, content] of Object.entries(extra)) write(dir, rel, content); + return dir; +} + +const PW = { id: "package-lock.json#playwright-core", package: "playwright-core", current: "1.64.1", target: "1.66.0" }; + +test("findPins finds a whole-token exact version beside the package name, never a stray number", () => { + const root = pinsProject(); + const pins = deps.findPins(root, [PW], { skip: ["package.json", "package-lock.json"] }); + assert.deepEqual(pins, [{ id: PW.id, file: "scripts/test-browser-helper.mjs", line: 4, text: 'export const BROWSER = "playwright-core@1.64.1";' }], "the 11.64.10 and 1.64.12 line is no pin"); +}); + +test("findPins skips lockfiles, files a lockfile entry owns, binary files and files over 1 MB", () => { + const big = `playwright-core 1.64.1\n${"x".repeat(deps.PIN_FILE_CAP)}`; + const root = pinsProject({ "web/package-lock.json": '{"playwright-core": "1.64.1"}\n', "notes/big.txt": big, "docs/readme.md": "Uses playwright-core 1.64.1 today.\n" }); + fs.writeFileSync(path.join(root, "assets.bin"), Buffer.concat([Buffer.from("playwright-core 1.64.1\n"), Buffer.from([0, 1, 2])])); + const files = deps.findPins(root, [PW], { skip: ["package.json", "package-lock.json"] }).map((p) => p.file); + assert.deepEqual(files.sort(), ["docs/readme.md", "scripts/test-browser-helper.mjs"]); + const owned = deps.findPins(root, [PW], { skip: ["package-lock.json"] }).map((p) => p.file); + assert.ok(owned.includes("package.json"), "a manifest not on the skip list is searched"); + assert.deepEqual(deps.findPins(root, [PW], { files: ["docs/readme.md"] }).map((p) => p.file), ["docs/readme.md"], "files limits the search"); + assert.deepEqual(deps.findPins(root, []), [], "no rows, no search"); +}); + +test("label: a pin outside the row's own files moves a Safe row to Check first, naming the file", () => { + const before = labeled(); + assert.equal(before.r1.label, "safe"); + const pins = { pins: [{ id: "r1", file: "scripts/test-browser-helper.mjs", line: 4, text: "x" }, { id: "r1", file: "package-lock.json", line: 9, text: "x" }] }; + const after = Object.fromEntries(deps.label(SCAN, RECORDS, pins).map((l) => [l.id, l])); + assert.equal(after.r1.label, "check-first"); + assert.ok(after.r1.reasons.includes("exact version used in scripts/test-browser-helper.mjs"), after.r1.reasons.join("; ")); + assert.match(after.r1.reasonsPlain.join(" "), /scripts\/test-browser-helper\.mjs names this exact version \(1\.2\.3\)/); + const own = Object.fromEntries(deps.label(SCAN, RECORDS, [{ id: "r1", file: "package-lock.json" }]).map((l) => [l.id, l])); + assert.equal(own.r1.label, "safe", "a pin in its own lockfile moves with the update"); + assert.deepEqual(deps.label(SCAN, RECORDS, null), deps.label(SCAN, RECORDS), "no pins.json changes nothing"); + const warning = Object.fromEntries(deps.label(SCAN, RECORDS, [{ id: "r5", file: "a.js" }]).map((l) => [l.id, l])); + assert.equal(warning.r5.label, "warning", "the Warning rules stay as they are"); +}); + +test("pins --run writes pins.json, label reads it, and the row with a pin in a test file leaves Safe", () => { + const root = pinsProject(); + const lock = { lockfile: "package-lock.json", manifest: "package.json", adapter: "npm", ecosystem: "npm", manager: "npm", members: ["."], checks: { rows: ["Root"], discovered: [] }, rows: [{ ...PW, jump: "minor", inRange: false, range: { from: "1.64.1", to: "1.66.0" }, pinned: true, security: null, declaredIn: [{ file: "package.json", range: "1.64.1" }] }] }; + const dir = runFolder({ "scan.json": { root, noChecks: false, lockfiles: [lock] } }); + const r = runDeps(["pins", "--run", dir]); + assert.equal(r.code, 0, r.stderr); + assert.equal(r.json.count, 1); + assert.deepEqual(r.json.pins.map((p) => [p.file, p.line]), [["scripts/test-browser-helper.mjs", 4]]); + assert.deepEqual(JSON.parse(fs.readFileSync(path.join(dir, "pins.json"), "utf8")), { pins: r.json.pins }); + const records = path.join(dir, "records-in.json"); + fs.writeFileSync(records, JSON.stringify([{ id: PW.id, notes: "read", breaking: [], usedIn: ["scripts/test-browser-helper.mjs"] }])); + const unpinned = deps.label({ noChecks: false, lockfiles: [lock] }, JSON.parse(fs.readFileSync(records, "utf8"))); + assert.equal(unpinned[0].label, "safe", "without the pin the row is Safe"); + const l = runDeps(["label", "--run", dir, "--records", records]); + assert.equal(l.code, 0, l.stderr); + assert.equal(l.json.labels[0].label, "check-first"); + assert.ok(l.json.labels[0].reasons.includes("exact version used in scripts/test-browser-helper.mjs")); +}); + +test("pins: a run folder that is not there and a run with no scan.json exit 3", () => { + const gone = runDeps(["pins", "--run", path.join(os.tmpdir(), "plan2code-deps-19990101-deadbeef")]); + assert.equal(gone.code, 3); + assert.equal(gone.json.error, "run-missing"); + const empty = runDeps(["pins", "--run", runFolder()]); + assert.equal(empty.code, 3); + assert.equal(empty.json.next, "Run scan first."); +}); + +/* ------------------------------------------------------------ checks: failing tests */ + +const LOGS = fx("logs"); +const logText = (name) => fs.readFileSync(path.join(LOGS, name), "utf8"); + +test("failedTests reads each runner's failing tests; TODO and SKIP lines are no failures", () => { + const cases = { + "tap.txt": ["parses dates"], + "node-spec.txt": ["parses dates"], + "jest.txt": ["divides by zero"], + "vitest.txt": ["src/b.test.ts > math > divides by zero"], + "pytest.txt": ["tests/test_b.py::test_divides"], + "cargo.txt": ["math::divides"], + "go.txt": ["TestDivides"], + "dotnet.txt": ["MyApp.Tests.Divides"], + }; + for (const [file, names] of Object.entries(cases)) assert.deepEqual(deps.failedTests(logText(file)), { names, known: true }, file); + assert.deepEqual(deps.failedTests(logText("unknown.txt")), { names: [], known: false }, "an unknown format falls back to the command"); + assert.deepEqual(deps.failedTests("ok 1 - adds\nok 2 - parses\n"), { names: [], known: true }, "a known format with nothing failing"); + assert.deepEqual(deps.failedTests("\x1b[31m--- FAIL: TestColor (0.00s)\x1b[0m\n"), { names: ["TestColor"], known: true }, "color codes are dropped"); +}); + +test("judgeCheck and compareCheck: red before, flaky and new failures by name; a whole command when the log is unknown", () => { + const log = (names, outcome = names.length ? "failed" : "passed", known = true) => ({ names, known, outcome }); + const base = (first, retry) => { + const j = deps.judgeCheck(first, retry); + return { failing: j.names, flaky: j.flaky, known: j.known, commandFailed: j.commandFailed, ...(j.timeout ? { timeout: true } : {}) }; + }; + const red = base(log(["a"]), log(["a"])); + assert.deepEqual(red.failing, ["a"]); + const wobbly = base(log(["b"]), log([])); + assert.deepEqual([wobbly.failing, wobbly.flaky], [[], ["b"]]); + assert.deepEqual(deps.compareCheck(red, deps.judgeCheck(log(["a"]), log(["a"]))), { newFailures: [], stillFailing: ["a"], flaky: [], blame: false, known: true }); + assert.equal(deps.compareCheck(wobbly, deps.judgeCheck(log(["b"]), log(["b"]))).blame, false, "flaky before never blames"); + const broke = deps.compareCheck(red, deps.judgeCheck(log(["a", "c"]), log(["a", "c"]))); + assert.deepEqual([broke.newFailures, broke.blame], [["c"], true]); + assert.equal(deps.compareCheck(red, deps.judgeCheck(log(["c"]), log([]))).blame, false, "a test that fails once and passes on the retry is flaky now"); + const unknownPass = base(log([], "passed", false)); + assert.equal(deps.compareCheck(unknownPass, deps.judgeCheck(log([], "failed", false))).blame, true, "a command that passed before and fails now"); + const unknownFail = base(log([], "failed", false)); + assert.equal(deps.compareCheck(unknownFail, deps.judgeCheck(log([], "failed", false))).blame, false, "a command already failing as a whole"); + assert.equal(deps.compareCheck(red, deps.judgeCheck(log([], "failed", false))).blame, true, "a run that names no test now, after named failures before, is blamed"); +}); + +/* ------------------------------------------------------------ checks: commands */ + +const CHECKS_LOCK = "package-lock.json"; +const TOOLS_LOCK = "tools/package-lock.json"; +/** A run folder: the root version list with test map rows (one Shared) and a build check, and tools/ with only a confirmed command. */ +function checksRun({ records = null, pins = null, list = null } = {}) { + const lock = { + lockfile: CHECKS_LOCK, + manifest: "package.json", + adapter: "npm", + ecosystem: "npm", + manager: "npm", + members: ["."], + install: "npm ci", + checks: { + rows: ["App", "Web", "Everything"], + commands: ["npm test -w app", "npm test -w web", "npm test"], + discovered: [], + full: true, + list: list || [ + { id: "c1", area: "App", run: "npm test -w app", paths: ["packages/app/**"], shared: false }, + { id: "c2", area: "Web", run: "npm test -w web", paths: ["web/**"], shared: false }, + { id: "c3", area: "Everything", run: "npm test", paths: ["scripts/**"], shared: true }, + ], + build: [{ command: "npm run build", cwd: "." }], + }, + rows: ["a", "b", "c", "d"].map((p) => ({ id: `${CHECKS_LOCK}#${p}`, package: p, current: "1.0.0", target: "1.0.1" })), + held: [], + }; + const tools = { + lockfile: TOOLS_LOCK, + manifest: "tools/package.json", + adapter: "npm", + ecosystem: "npm", + manager: "npm", + members: ["tools"], + install: "npm ci", + checks: { rows: [], commands: [], discovered: [{ command: "npm run lint", cwd: "tools" }], full: false, list: [], build: [] }, + rows: [{ id: `${TOOLS_LOCK}#e`, package: "e", current: "1.0.0", target: "1.0.1" }], + held: [], + }; + const batches = [...["a", "b", "c", "d"].map((p, i) => ({ id: `b${i + 1}`, lockfile: CHECKS_LOCK, picks: [`${CHECKS_LOCK}#${p}`] })), { id: "b5", lockfile: TOOLS_LOCK, picks: [`${TOOLS_LOCK}#e`] }]; + const files = { + "scan.json": { root: tmp("deps-checks"), noChecks: false, lockfiles: [lock, tools] }, + "batches.json": { batches }, + }; + if (records) files["records.json"] = records; + if (pins) files["pins.json"] = pins; + return runFolder(files); +} + +test("checks lists every check with an id for the baseline, the Shared rows too; full is the full suite", () => { + const r = runDeps(["checks", "--run", checksRun()]); + assert.equal(r.code, 0, r.stderr); + assert.deepEqual( + r.json.checks.map((c) => [c.id, c.command, c.cwd, c.kind, c.lockfile, Boolean(c.shared)]), + [ + ["install", "npm ci", ".", "install", CHECKS_LOCK, false], + ["c1", "npm test -w app", ".", "map", CHECKS_LOCK, false], + ["c2", "npm test -w web", ".", "map", CHECKS_LOCK, false], + ["c3", "npm test", ".", "map", CHECKS_LOCK, true], + ["d1", "npm run build", ".", "build", CHECKS_LOCK, false], + ["install-2", "npm ci", "tools", "install", TOOLS_LOCK, false], + ["d2", "npm run lint", "tools", "discovered", TOOLS_LOCK, false], + ], + ); + assert.deepEqual(r.json.full, [{ id: "c3", command: "npm test", cwd: ".", area: "Everything" }]); +}); + +test("checks --batch narrows by usedIn and pins and adds the build; a Shared row runs only when it alone covers a file", () => { + const dir = checksRun({ + records: [ + { id: `${CHECKS_LOCK}#a`, usedIn: ["packages/app/src/x.js"] }, + { id: `${CHECKS_LOCK}#c`, usedIn: ["tools2/y.js"] }, + { id: `${CHECKS_LOCK}#d`, usedIn: ["scripts\\helper.mjs", "packages/app/z.js"] }, + ], + pins: { pins: [{ id: `${CHECKS_LOCK}#b`, file: "web/test/pin.test.js", line: 3, text: "b 1.0.0" }] }, + }); + const a = runDeps(["checks", "--run", dir, "--batch", "b1"]); + assert.equal(a.code, 0, a.stderr); + assert.equal(a.json.narrowed, true); + assert.deepEqual(a.json.commands.map((c) => c.id), ["c1", "d1"], "the App row and the build"); + assert.deepEqual(a.json.install, { id: "install", command: "npm ci", cwd: "." }); + assert.deepEqual(a.json.full.map((c) => c.id), ["c3"], "the Shared row is left for the end"); + assert.equal(a.json.message, null); + const b = runDeps(["checks", "--run", dir, "--batch", "b2"]); + assert.deepEqual(b.json.commands.map((c) => c.id), ["c2", "d1"], "narrowed by the pin's file"); + const d = runDeps(["checks", "--run", dir, "--batch", "b4"]); + assert.deepEqual(d.json.commands.map((c) => c.id), ["c1", "c3", "d1"], "scripts/helper.mjs is covered only by the Shared row"); + assert.deepEqual(d.json.full, [], "it ran in the group, so nothing is left for the end"); + const c = runDeps(["checks", "--run", dir, "--batch", "b3"]); + assert.equal(c.json.narrowed, false); + assert.deepEqual(c.json.commands.map((x) => x.id), ["c1", "c2", "d1"]); + assert.match(c.json.message, /No test map row covers/); + const none = runDeps(["checks", "--run", checksRun(), "--batch", "b1"]); + assert.equal(none.json.narrowed, false); + assert.match(none.json.message, /Nothing says which files/); + assert.ok(!none.json.commands.some((x) => x.id === "c3"), "with other tests, not narrowed still leaves the Shared row out"); + const tools = runDeps(["checks", "--run", dir, "--batch", "b5"]); + assert.deepEqual([tools.json.commands.map((x) => x.id), tools.json.install.id], [["d2"], "install-2"], "a folder with only a confirmed command runs it"); + assert.equal(runDeps(["checks", "--run", dir, "--batch", "b9"]).code, 3, "an unknown batch"); +}); + +test("checks --batch: a pin file no narrower row covers runs the full suite in the group, narrowed or not", () => { + const dir = checksRun({ + records: [{ id: `${CHECKS_LOCK}#a`, usedIn: ["packages/app/src/x.js"] }], + pins: { + pins: [ + { id: `${CHECKS_LOCK}#a`, file: "tests/browser.test.mjs", line: 4, text: "a 1.0.0" }, + { id: `${CHECKS_LOCK}#c`, file: "tests/other.test.mjs", line: 2, text: "c 1.0.0" }, + { id: `${CHECKS_LOCK}#b`, file: "web/pin.test.js", line: 3, text: "b 1.0.0" }, + ], + }, + }); + const narrowed = runDeps(["checks", "--run", dir, "--batch", "b1"]); + assert.equal(narrowed.code, 0, narrowed.stderr); + assert.equal(narrowed.json.narrowed, true); + assert.deepEqual(narrowed.json.commands.map((c) => c.id), ["c1", "c3", "d1"], "the App row, then the full suite for the uncovered pin"); + assert.deepEqual(narrowed.json.full, []); + assert.match(narrowed.json.message, /covers tests\/browser\.test\.mjs, which names the exact version, so the full suite runs/); + const wide = runDeps(["checks", "--run", dir, "--batch", "b3"]); + assert.equal(wide.json.narrowed, false); + assert.deepEqual(wide.json.commands.map((c) => c.id), ["c1", "c2", "c3", "d1"], "every row, plus the full suite for the pin"); + assert.match(wide.json.message, /tests\/other\.test\.mjs/); + const covered = runDeps(["checks", "--run", dir, "--batch", "b2"]); + assert.deepEqual(covered.json.commands.map((c) => c.id), ["c2", "d1"], "a pin a narrower row covers changes nothing"); + assert.equal(covered.json.message, null); +}); + +test("checks --batch: a folder whose only test is a Shared row runs it, so a group never runs without a test", () => { + const dir = checksRun({ list: [{ id: "c1", area: "Everything", run: "npm test", paths: ["scripts/**"], shared: true }] }); + const r = runDeps(["checks", "--run", dir, "--batch", "b1"]); + assert.equal(r.code, 0, r.stderr); + assert.equal(r.json.narrowed, false); + assert.deepEqual(r.json.commands.map((c) => c.id), ["c1", "d1"]); + assert.match(r.json.message, /full suite is the only test/); + assert.deepEqual(r.json.full, []); +}); + +test("scan saves the build checks test-commands.mjs finds in each version list's folder", () => { + const root = project({ "package.json": { name: "fixture-root", private: true, engines: { node: ">=12" }, workspaces: ["packages/*"], scripts: { build: "tsup", test: "node --test" }, dependencies: { chalk: "^4.1.0", "left-pad": "1.1.0", semver: "^7.5.0", mkdirp: "^0.5.5" }, overrides: { ms: "2.1.2" } } }); + const { scan } = detectAndScan(root, { npm: npmRules() }, ["--no-audit"]); + assert.deepEqual(scan.lockfiles[0].checks.build, [{ command: "npm run build", cwd: "." }]); + assert.deepEqual(deps.checkList(scan)[0].discovered, [{ id: "d1", command: "npm run build", cwd: ".", kind: "build" }]); + assert.equal(deps.label(scan, []).length > 0, true); +}); + +/* ------------------------------------------------------------ checks: baseline and compare */ + +function logFile(text) { + const dir = tmp("deps-log"); + return write(dir, "check.log", text); +} +const TAP = (fails) => ["TAP version 13", "ok 1 - adds numbers", ...fails.map((n, i) => `not ok ${i + 2} - ${n}`), ""].join("\n"); + +test("baseline records red-before and flaky tests; compare blames only tests that passed before", () => { + const dir = checksRun(); + const b = runDeps(["baseline", "--run", dir, "--check", "c1", "--log", logFile(TAP(["parses dates", "reads the clock"])), "--outcome", "failed", "--retry-log", logFile(TAP(["parses dates"])), "--retry-outcome", "failed"]); + assert.equal(b.code, 0, b.stderr); + assert.deepEqual([b.json.failing, b.json.flaky, b.json.known, b.json.commandFailed], [["parses dates"], ["reads the clock"], true, false]); + const saved = JSON.parse(fs.readFileSync(path.join(dir, "baseline.json"), "utf8")); + assert.deepEqual(saved.checks.c1.failing, ["parses dates"]); + + const same = runDeps(["compare", "--run", dir, "--batch", "b1", "--check", "c1", "--log", logFile(TAP(["parses dates", "reads the clock"])), "--retry-log", logFile(TAP(["parses dates", "reads the clock"]))]); + assert.equal(same.code, 0, same.stderr); + assert.deepEqual([same.json.newFailures, same.json.stillFailing, same.json.flaky, same.json.blame], [[], ["parses dates"], ["reads the clock"], false], "red before and flaky never blame"); + + const broke = runDeps(["compare", "--run", dir, "--batch", "b1", "--check", "c1", "--log", logFile(TAP(["adds more", "parses dates"])), "--outcome", "failed"]); + assert.deepEqual([broke.json.newFailures, broke.json.blame], [["adds more"], true], "a test that passed before"); + const record = JSON.parse(fs.readFileSync(path.join(dir, "compare.json"), "utf8")); + assert.equal(record.batches.b1.c1.blame, true); +}); + +test("baseline and compare judge a log in an unknown format, and a timeout, by the whole command", () => { + const dir = checksRun(); + const unknown = logFile(logText("unknown.txt")); + const pass = runDeps(["baseline", "--run", dir, "--check", "install", "--log", unknown, "--outcome", "passed"]); + assert.equal(pass.code, 0, pass.stderr); + assert.deepEqual([pass.json.known, pass.json.commandFailed], [false, false]); + const now = runDeps(["compare", "--run", dir, "--batch", "b1", "--check", "install", "--log", unknown, "--outcome", "failed"]); + assert.deepEqual([now.json.known, now.json.blame], [false, true], "passed before, fails now"); + + const red = runDeps(["baseline", "--run", dir, "--check", "d1", "--log", unknown, "--outcome", "failed"]); + assert.deepEqual([red.json.known, red.json.commandFailed], [false, true]); + assert.equal(runDeps(["compare", "--run", dir, "--batch", "b1", "--check", "d1", "--log", unknown, "--outcome", "failed"]).json.blame, false, "already failing as a whole"); + + const slow = runDeps(["baseline", "--run", dir, "--check", "c2", "--log", logFile(TAP([])), "--outcome", "timeout"]); + assert.deepEqual([slow.json.commandFailed, slow.json.timeout, slow.json.known], [true, true, false]); +}); + +test("baseline and compare: usage, a missing log, an unknown check and a check with no baseline", () => { + const dir = checksRun(); + const log = logFile(TAP([])); + assert.equal(runDeps(["baseline", "--run", dir, "--log", log]).json.error, "missing-check"); + assert.equal(runDeps(["baseline", "--run", dir, "--check", "c1"]).json.error, "missing-log"); + const unknownCheck = runDeps(["baseline", "--run", dir, "--check", "c9", "--log", log]); + assert.equal(unknownCheck.code, 3); + assert.equal(unknownCheck.json.error, "unknown-check"); + const missing = runDeps(["baseline", "--run", dir, "--check", "c1", "--log", path.join(tmp("deps-log"), "nope.log")]); + assert.equal(missing.code, 3); + assert.equal(missing.json.error, "log-missing"); + const unread = runDeps(["baseline", "--run", dir, "--check", "c1", "--log", logFile(logText("unknown.txt"))]); + assert.equal(unread.code, 2, "an unknown format needs run-check's outcome"); + assert.equal(unread.json.error, "missing-outcome"); + assert.equal(runDeps(["baseline", "--run", dir, "--check", "c1", "--log", log, "--outcome", "maybe"]).json.error, "bad-outcome"); + const noBase = runDeps(["compare", "--run", dir, "--batch", "b1", "--check", "c2", "--log", log]); + assert.equal(noBase.code, 3); + assert.equal(noBase.json.next, "Run baseline for this check first."); +}); + +/* ------------------------------------------------------------ runtime fit */ + +test("bestFit: the newest version that fits, none, and never one older than the current", () => { + const versions = [ + { version: "4.1.2", engines: ">=10" }, + { version: "5.0.0", engines: ">=16" }, + { version: "6.0.1", engines: ">=22" }, + ]; + const fits = (floor) => (v) => !v.engines || Number(v.engines.replace(">=", "")) <= floor; + assert.deepEqual(deps.bestFit({ current: "4.1.0", versions, fits: fits(18) }), { best: "5.0.0", needs: ">=22" }); + assert.deepEqual(deps.bestFit({ current: "4.1.0", versions, fits: fits(8) }), { best: null, needs: ">=22" }); + assert.deepEqual(deps.bestFit({ current: "4.1.0", latest: "5.0.0", versions, fits: fits(18) }), { best: "5.0.0", needs: null }, "latest caps the list"); + assert.deepEqual(deps.bestFit({ current: "5.0.0", versions: [{ version: "4.1.2", engines: ">=10" }, { version: "6.0.1", engines: ">=22" }], fits: fits(18) }), { best: null, needs: ">=22" }, "an older version that fits is never offered"); +}); + +/** A run folder whose scan holds chalk held for the runtime, ms held by an override, and a Cargo crate held for Rust. */ +function fitRun(root) { + const npmLock = { + lockfile: "package-lock.json", + manifest: "package.json", + adapter: "npm", + ecosystem: "npm", + manager: "npm", + members: [".", "packages/app"], + runtime: { declared: JSON.parse(fs.readFileSync(path.join(root, "package.json"), "utf8")).engines.node, source: "engines", machine: "24.0.0" }, + checks: { rows: ["Root"], discovered: [] }, + rows: [], + held: [ + { id: "package-lock.json#chalk", package: "chalk", current: "4.1.0", latest: "6.0.1", reason: "every newer version needs Node >=22 (the project declares >=12)", file: null, security: null }, + { id: "package-lock.json#ms", package: "ms", current: "2.1.2", latest: "2.1.3", reason: "held by overrides in package.json", file: "package.json", security: null }, + ], + }; + const cargoLock = { + lockfile: "app/Cargo.lock", + manifest: "app/Cargo.toml", + adapter: "cargo", + ecosystem: "cargo", + manager: "cargo", + members: ["app"], + runtime: { declared: "1.70", source: "rust-version", machine: "1.99.1" }, + checks: { rows: [], discovered: [] }, + rows: [], + held: [{ id: "app/Cargo.lock#serde", package: "serde", current: "1.0.100", latest: "1.0.200", reason: "every newer version needs Rust 1.80 (the project declares 1.70)", file: null, security: null }], + }; + return runFolder({ "scan.json": { root, noChecks: false, lockfiles: [npmLock, cargoLock] } }); +} + +test("fit turns a runtime hold with a fitting version into an ordinary row that can be Safe; an override stays held; no adapter means unknown", () => { + const root = project(); + const dir = fitRun(root); + const r = runStubbed(["fit", "--run", dir], stubPath({ npm: npmRules() })); + assert.equal(r.code, 0, r.stderr); + const chalk = r.json.rows.find((x) => x.package === "chalk"); + assert.deepEqual(chalk, { id: "package-lock.json#chalk", package: "chalk", current: "4.1.0", best: "4.1.2", needs: ">=22", source: "registry" }); + assert.deepEqual(r.json.unknown, ["app/Cargo.lock#serde"]); + assert.ok(!r.json.rows.some((x) => x.package === "ms"), "an override hold is not a runtime hold"); + const scan = JSON.parse(fs.readFileSync(path.join(dir, "scan.json"), "utf8")); + const [npmLock, cargoLock] = scan.lockfiles; + assert.deepEqual(npmLock.held.map((h) => h.package), ["ms"]); + const row = npmLock.rows.find((x) => x.package === "chalk"); + assert.equal(row.target, "4.1.2"); + assert.equal(row.inRange, true); + assert.deepEqual(row.fit, { runtime: "Node", declared: ">=12", needs: ">=22", latest: "6.0.1" }); + const [l] = deps.label(scan, [{ id: row.id, notes: "read", breaking: [] }]); + assert.equal(l.label, "safe", "labeled by the same rules"); + assert.match(l.reasonsPlain.join(" "), /newest version that runs on Node >=12; the latest needs Node >=22/); + assert.equal(cargoLock.held[0].reason, "needs Rust 1.80 (you declare 1.70)"); + assert.equal(cargoLock.held[0].raise, "change `rust-version` in `app/Cargo.toml`"); + assert.deepEqual(JSON.parse(fs.readFileSync(path.join(dir, "fit.json"), "utf8")).unknown, ["app/Cargo.lock#serde"]); +}); + +test("fit: with no version that runs on the declared Node the row stays held, saying what it needs and how to raise it", () => { + const root = project(); + const pkg = JSON.parse(fs.readFileSync(path.join(root, "package.json"), "utf8")); + pkg.engines.node = ">=8"; + fs.writeFileSync(path.join(root, "package.json"), JSON.stringify(pkg, null, 2) + "\n"); + const dir = fitRun(root); + const r = runStubbed(["fit", "--run", dir], stubPath({ npm: npmRules() })); + assert.equal(r.code, 0, r.stderr); + assert.deepEqual(r.json.rows.find((x) => x.package === "chalk"), { id: "package-lock.json#chalk", package: "chalk", current: "4.1.0", best: null, needs: ">=22", source: "registry" }); + assert.ok(!r.json.unknown.includes("package-lock.json#chalk"), "the registry answered: none fits"); + const held = JSON.parse(fs.readFileSync(path.join(dir, "scan.json"), "utf8")).lockfiles[0].held.find((h) => h.package === "chalk"); + assert.equal(held.reason, "needs Node >=22 (you declare >=8)"); + assert.equal(held.raise, "change `engines.node` in `package.json`"); +}); + +test("fit: a row the scan already cut keeps its target and learns what the latest needs; a run with no scan exits 3", () => { + const { scan, run, bin } = detectAndScan(project(), { npm: npmRules() }, ["--no-audit"]); + assert.ok(scan.lockfiles[0].rows.find((x) => x.package === "chalk").newerNotFitting); + const r = runStubbed(["fit", "--run", run], bin); + assert.equal(r.code, 0, r.stderr); + assert.deepEqual(r.json.rows.find((x) => x.package === "chalk"), { id: "package-lock.json#chalk", package: "chalk", current: "4.1.0", best: "4.1.2", needs: ">=22", source: "registry" }); + const row = JSON.parse(fs.readFileSync(path.join(run, "scan.json"), "utf8")).lockfiles[0].rows.find((x) => x.package === "chalk"); + assert.equal(row.target, "4.1.2"); + assert.equal(row.fit.needs, ">=22"); + const empty = runDeps(["fit", "--run", runFolder()]); + assert.equal(empty.code, 3); + assert.equal(empty.json.next, "Run scan first."); +}); + +test("fit --id --best makes the version worked out with the person a pick; a hold that is not the runtime's and a version not newer are refused", () => { + const root = project(); + const dir = fitRun(root); + const first = runStubbed(["fit", "--run", dir], stubPath({ npm: npmRules() })); + assert.deepEqual(first.json.unknown, ["app/Cargo.lock#serde"]); + const r = runDeps(["fit", "--run", dir, "--id", "app/Cargo.lock#serde", "--best", "1.0.150"]); + assert.equal(r.code, 0, r.stderr); + assert.deepEqual(r.json.rows, [{ id: "app/Cargo.lock#serde", package: "serde", current: "1.0.100", best: "1.0.150", needs: "1.80", source: "agent" }]); + const scan = JSON.parse(fs.readFileSync(path.join(dir, "scan.json"), "utf8")); + const cargo = scan.lockfiles[1]; + assert.deepEqual(cargo.held, []); + assert.equal(cargo.rows[0].target, "1.0.150"); + assert.deepEqual(cargo.rows[0].fit, { runtime: "Rust", declared: "1.70", needs: "1.80", latest: "1.0.200" }); + const fit = JSON.parse(fs.readFileSync(path.join(dir, "fit.json"), "utf8")); + assert.deepEqual(fit.unknown, []); + assert.equal(fit.rows.find((x) => x.id === "app/Cargo.lock#serde").source, "agent"); + const override = runDeps(["fit", "--run", dir, "--id", "package-lock.json#ms", "--best", "2.1.3"]); + assert.equal(override.code, 5); + assert.equal(override.json.error, "not-a-runtime-hold"); + const again = fitRun(project()); + const older = runDeps(["fit", "--run", again, "--id", "app/Cargo.lock#serde", "--best", "1.0.90"]); + assert.equal(older.code, 4); + assert.equal(older.json.error, "not-newer"); + assert.equal(runDeps(["fit", "--run", again, "--id", "nope#x", "--best", "1.0.0"]).code, 3); + assert.equal(runDeps(["fit", "--run", again, "--id", "app/Cargo.lock#serde"]).code, 2, "--id needs --best"); +}); + +test("fit: when the adapter cannot answer, the held reason keeps a runtime range with spaces whole", () => { + const root = project(); + const dir = fitRun(root); + const scan = JSON.parse(fs.readFileSync(path.join(dir, "scan.json"), "utf8")); + scan.lockfiles[0].held[0].reason = "every newer version needs Node ^12.17.0 || >=16.0.0 (the project declares >=12)"; + fs.writeFileSync(path.join(dir, "scan.json"), JSON.stringify(scan)); + const r = runStubbed(["fit", "--run", dir], stubPath({ git: [{ has: ["--version"], text: "git" }] })); + assert.equal(r.code, 0, r.stderr); + assert.deepEqual(r.json.unknown, ["package-lock.json#chalk", "app/Cargo.lock#serde"], "no npm on PATH: no adapter answers"); + const held = JSON.parse(fs.readFileSync(path.join(dir, "scan.json"), "utf8")).lockfiles[0].held.find((h) => h.package === "chalk"); + assert.equal(held.reason, "needs Node ^12.17.0 || >=16.0.0 (you declare >=12)"); +}); + test("no stub ever received --force or an audit fix", () => { const calls = stubDirs.flatMap(callsOf); assert.ok(calls.length > 20, "the stubbed tests ran first"); @@ -2543,3 +3024,327 @@ test("no stub ever received --force or an audit fix", () => { assert.ok(!(c.args.includes("audit") && c.args.includes("fix")), "never audit fix"); } }); + +/* ------------------------------------------------------------ plain words */ + +// Words a person never reads (specs/deps-skill-rework/wording-audit.md). They stay in field names, commands and file names. +const AVOID = /\b(lockfiles?|manifests?|tracked|hand-?offs?|jump)\b/i; + +test("label: every row carries reasonsPlain, one plain sentence per reason in the same order, with no avoid-list word", () => { + const pinned = JSON.parse(JSON.stringify(SCAN)); + pinned.lockfiles[0].rows[0] = { ...pinned.lockfiles[0].rows[0], pinned: true, security: { severity: "high", ids: ["X"] }, runtimeNotChecked: ">=3.10", couldNotRead: "npm" }; + const scans = [SCAN, pinned, { ...SCAN, noChecks: true }]; + const lists = [deps.label(SCAN, RECORDS), deps.label(SCAN, []), deps.label(pinned, RECORDS), deps.label(scans[2], RECORDS)]; + let seen = 0; + for (const labels of lists) { + for (const l of labels) { + assert.ok(Array.isArray(l.reasonsPlain), `${l.id} has reasonsPlain`); + assert.equal(l.reasonsPlain.length, l.reasons.length, `${l.id}: reasonsPlain lines up with reasons`); + for (const s of l.reasonsPlain) { + seen++; + assert.ok(typeof s === "string" && s.length > 0, `${l.id}: a plain sentence`); + assert.ok(!AVOID.test(s), `${l.id}: "${s}" uses an avoid-list word`); + assert.ok(!l.reasons.includes(s), `${l.id}: "${s}" is not just the raw reason`); + } + } + } + assert.ok(seen > 20, "the fixtures exercise many reasons"); + assert.ok(deps.label(SCAN, []).some((l) => l.reasonsPlain.includes("I did not read the update notes"))); + assert.deepEqual(labeled().r1.reasonsPlain, ["2 files use it"]); +}); + +test("label: reasonsPlain is printed by the label command and saved in labels.json", () => { + const dir = runFolder({ "scan.json": SCAN }); + const r = runDeps(["label", "--run", dir, "--records", path.join(FIXTURES, "label", "records.json")]); + assert.equal(r.code, 0, r.stderr); + assert.ok(r.json.labels.length > 0); + assert.ok(JSON.parse(fs.readFileSync(path.join(dir, "labels.json"), "utf8")).every((l) => Array.isArray(l.reasonsPlain))); + assert.ok(r.json.labels.every((l) => Array.isArray(l.reasonsPlain) && l.reasonsPlain.length === l.reasons.length)); +}); + +// The text a person reads: every double-quoted sentence in the prompt, and every multi-word string value in the notes' example cards. +function personFacing(markdown) { + const out = []; + for (const m of markdown.matchAll(/"([^"\n]*\s[^"\n]*)"/g)) out.push(m[1]); + return out; +} +function exampleStrings(markdown) { + const out = []; + const walk = (v) => { + if (typeof v === "string") { + if (/\s/.test(v)) out.push(v); + } else if (Array.isArray(v)) v.forEach(walk); + else if (v && typeof v === "object") Object.values(v).forEach(walk); + }; + for (const m of markdown.matchAll(/```jsonc\n([\s\S]*?)\n```/g)) walk(JSON.parse(m[1])); + return out; +} +const WHITELIST = /package-lock\.json|package\.json|requirements\.txt|pom\.xml|Cargo\.lock/g; + +test("the prompt and the console notes keep avoid-list words out of text a person reads", () => { + const prompt = fs.readFileSync(path.join(ROOT, "src", "plan2code-deps.md"), "utf8"); + const notes = fs.readFileSync(path.join(ROOT, "src", "web-console", "notes", "deps.md"), "utf8"); + const strings = [...personFacing(prompt), ...exampleStrings(notes)]; + assert.ok(strings.length > 10, "there is person-facing text to check"); + for (const s of strings) { + const text = s.replace(WHITELIST, "").replace(/`[^`]*`/g, ""); + assert.ok(!AVOID.test(text), `"${s}" uses an avoid-list word`); + } +}); + +test("the notes no longer carry a card with the id notes, and the start card is a confirm with a Start button", () => { + const notes = fs.readFileSync(path.join(ROOT, "src", "web-console", "notes", "deps.md"), "utf8"); + const items = []; + for (const m of notes.matchAll(/```jsonc\n([\s\S]*?)\n```/g)) items.push(...(JSON.parse(m[1]).items || [])); + assert.ok(!items.some((i) => i.id === "notes"), "the release-notes question is gone"); + const start = items.find((i) => i.id === "start"); + assert.equal(start.kind, "confirm"); + assert.equal(start.yesLabel, "Start"); + assert.match(start.body, /I will read what each update changes before judging it/); + assert.ok(!/Release notes|skip the notes/i.test(notes), "no release-notes question or skip option remains"); +}); + +/* ------------------------------------------------------ fixes and tools */ + +// A fix is one approved replacement in one project file after apply; it is +// backed up into the batch's backup, so restore puts it back with the rest. + +/** Every file under root (relative, forward slashes) with its bytes. */ +function treeOf(root) { + const files = {}; + const walkTree = (dir) => { + for (const e of fs.readdirSync(dir, { withFileTypes: true })) { + const abs = path.join(dir, e.name); + if (e.isDirectory()) walkTree(abs); + else files[path.relative(root, abs).split(path.sep).join("/")] = fs.readFileSync(abs); + } + }; + walkTree(root); + return files; +} + +function assertSameTree(root, before) { + const now = treeOf(root); + assert.deepEqual(Object.keys(now).sort(), Object.keys(before).sort(), "the same files"); + for (const [rel, bytes] of Object.entries(before)) assert.ok(now[rel].equals(bytes), `${rel} byte for byte`); +} + +const PIN_TEST = 'import { test } from "node:test";\nconst BUILD = "1.64.1";\ntest("browser build", () => {\n check(BUILD);\n});\n'; + +/** A project with a test file, chalk planned, backed up and applied: ready for a fix. */ +function applied(extra = {}) { + const root = project({ "scripts/test-browser.mjs": PIN_TEST, ...extra }); + const start = treeOf(root); + const { run, bin, batches } = planned(root, ["chalk"], { + npm: npmRules([{ has: ["update", "chalk"], writes: [{ file: "package-lock.json", bump: { "node_modules/chalk": "4.1.2" } }] }]), + }); + const b = batches[0]; + assert.equal(runStubbed(["backup", "--run", run, "--batch", b.id], bin).code, 0); + const a = runStubbed(["apply", "--run", run, "--batch", b.id], bin); + assert.equal(a.code, 0, a.stderr); + return { root, run, bin, b, start }; +} + +const fixArgs = (run, b, file, from, to) => ["fix", "--run", run, "--batch", b.id, "--file", file, "--from", from, "--to", to]; + +test("deps --help names fix and install-tool, and the new rule for editing files", () => { + const r = runDeps(["--help"]); + for (const command of ["fix", "install-tool"]) assert.match(r.stdout, new RegExp(`deps\\.mjs ${command}\\b`), command); + assert.ok(!/never edits source code/.test(r.stdout), "the old rule is gone"); + assert.match(r.stdout, /only through fix, after the person approved that\s+exact change/); +}); + +test("fix replaces the one place, backs the file up into the batch and records fixes.json", () => { + const { root, run, bin, b } = applied(); + const r = runStubbed(fixArgs(run, b, "scripts/test-browser.mjs", '"1.64.1"', '"1.65.1"'), bin); + assert.equal(r.code, 0, r.stderr); + assert.deepEqual(r.json, { ok: true, batch: b.id, file: "scripts/test-browser.mjs", replaced: 1, backedUp: true }); + assert.equal(fs.readFileSync(path.join(root, "scripts/test-browser.mjs"), "utf8"), PIN_TEST.replace("1.64.1", "1.65.1")); + const backup = JSON.parse(fs.readFileSync(path.join(run, "backups", b.id, "backup.json"), "utf8")); + assert.deepEqual(backup.files.at(-1), { path: "scripts/test-browser.mjs", existed: true }); + assert.deepEqual(backup.fixed, ["scripts/test-browser.mjs"]); + assert.equal(fs.readFileSync(path.join(run, "backups", b.id, "files", "scripts/test-browser.mjs"), "utf8"), PIN_TEST, "the as-found copy"); + const fixes = JSON.parse(fs.readFileSync(path.join(run, "fixes.json"), "utf8")).fixes; + assert.equal(fixes.length, 1); + const { at, ...rest } = fixes[0]; + assert.deepEqual(rest, { batch: b.id, file: "scripts/test-browser.mjs", from: '"1.64.1"', to: '"1.65.1"' }); + assert.ok(!Number.isNaN(Date.parse(at))); +}); + +test("fix needs a backup and an applied batch (exit 5)", () => { + const root = project({ "scripts/test-browser.mjs": PIN_TEST }); + const { run, bin, batches } = planned(root, ["chalk"], { npm: npmRules([{ has: ["update"], text: "" }]) }); + const b = batches[0]; + const noBackup = runStubbed(fixArgs(run, b, "scripts/test-browser.mjs", "1.64.1", "1.65.1"), bin); + assert.equal(noBackup.code, 5); + assert.equal(noBackup.json.error, "no-backup"); + runStubbed(["backup", "--run", run, "--batch", b.id], bin); + const notApplied = runStubbed(fixArgs(run, b, "scripts/test-browser.mjs", "1.64.1", "1.65.1"), bin); + assert.equal(notApplied.code, 5); + assert.equal(notApplied.json.error, "not-applied"); + assert.equal(fs.readFileSync(path.join(root, "scripts/test-browser.mjs"), "utf8"), PIN_TEST, "nothing changed"); +}); + +test("restore --verify after a fix puts LF and CRLF files back byte for byte and names the fixes", () => { + const crlf = 'line one\r\nconst BUILD = "1.64.1";\r\nline three\r\n'; + const { root, run, bin, b, start } = applied({ "config/browser.txt": crlf }); + const lf = runStubbed(fixArgs(run, b, "scripts/test-browser.mjs", "1.64.1", "1.65.1"), bin); + assert.equal(lf.code, 0, lf.stderr); + // A bare-LF --from spanning a line break matches the CRLF file and is written back with CRLF. + const cr = runStubbed(fixArgs(run, b, "config/browser.txt", 'line one\nconst BUILD = "1.64.1";', 'line one\nconst BUILD = "1.65.1";'), bin); + assert.equal(cr.code, 0, cr.stderr); + assert.equal(fs.readFileSync(path.join(root, "config/browser.txt"), "utf8"), crlf.replace("1.64.1", "1.65.1"), "CRLF kept"); + const r = runStubbed(["restore", "--run", run, "--batch", b.id, "--verify"], bin); + assert.equal(r.code, 0, r.stderr); + assert.equal(r.json.verified, true); + assert.deepEqual(r.json.fixed, ["scripts/test-browser.mjs", "config/browser.txt"]); + assert.ok(r.json.restored.includes("config/browser.txt")); + assertSameTree(root, start); +}); + +test("a second fix to the same file keeps the first as-found copy", () => { + const { root, run, bin, b, start } = applied(); + assert.equal(runStubbed(fixArgs(run, b, "scripts/test-browser.mjs", "1.64.1", "1.65.1"), bin).code, 0); + const second = runStubbed(fixArgs(run, b, "scripts/test-browser.mjs", "check(BUILD)", "check(BUILD, true)"), bin); + assert.equal(second.code, 0, second.stderr); + assert.equal(second.json.backedUp, false); + assert.equal(fs.readFileSync(path.join(run, "backups", b.id, "files", "scripts/test-browser.mjs"), "utf8"), PIN_TEST); + const backup = JSON.parse(fs.readFileSync(path.join(run, "backups", b.id, "backup.json"), "utf8")); + assert.equal(backup.files.filter((f) => f.path === "scripts/test-browser.mjs").length, 1, "listed once"); + assert.equal(JSON.parse(fs.readFileSync(path.join(run, "fixes.json"), "utf8")).fixes.length, 2); + assert.equal(runStubbed(["restore", "--run", run, "--batch", b.id, "--verify"], bin).json.verified, true); + assertSameTree(root, start); +}); + +test("after restore the group is no longer applied, so fix refuses it until apply runs again", () => { + const { run, bin, b } = applied(); + assert.equal(runStubbed(["restore", "--run", run, "--batch", b.id, "--verify"], bin).json.verified, true); + const r = runStubbed(fixArgs(run, b, "scripts/test-browser.mjs", "1.64.1", "1.65.1"), bin); + assert.equal(r.code, 5); + assert.equal(r.json.error, "not-applied"); +}); + +test("a fixed executable keeps its mode through the put-back (macOS and Linux)", { skip: process.platform === "win32" }, () => { + const { root, run, bin, b } = applied(); + const file = path.join(root, "scripts/test-browser.mjs"); + fs.chmodSync(file, 0o755); + assert.equal(runStubbed(fixArgs(run, b, "scripts/test-browser.mjs", "1.64.1", "1.65.1"), bin).code, 0); + assert.equal(runStubbed(["restore", "--run", run, "--batch", b.id, "--verify"], bin).json.verified, true); + assert.equal(fs.statSync(file).mode & 0o777, 0o755); +}); + +test("fix refuses a batch file, a path outside the project, a protected folder, a binary and text found twice or not at all", () => { + const twice = "a = 1;\na = 1;\n"; + const { root, run, bin, b } = applied({ "src/twice.js": twice, "node_modules/x/index.js": "const v = 1;\n", "specs/x/plan.md": "v 1\n" }); + fs.writeFileSync(path.join(root, "assets.bin"), Buffer.from([0x61, 0x00, 0x62, 0x0a])); + const before = treeOf(root); + const cases = [ + ["package.json", '"chalk"', '"chalk2"', "fix-batch-file"], + ["package-lock.json", '"4.1.2"', '"4.1.3"', "fix-batch-file"], + ["packages/app/package.json", '"is-number"', '"is-num"', "fix-batch-file"], + ["../outside.txt", "a", "b", "fix-outside"], + [path.join(root, "src", "twice.js"), "a = 1", "a = 2", "fix-absolute"], + ["node_modules/x/index.js", "1", "2", "fix-protected-folder"], + ["specs/x/plan.md", "1", "2", "fix-protected-folder"], + ["assets.bin", "a", "c", "fix-binary"], + ["src/twice.js", "a = 1;", "a = 2;", "fix-many"], + ["src/twice.js", "b = 1;", "b = 2;", "fix-not-found-text"], + ["src/missing.js", "a", "b", "fix-not-found"], + ]; + for (const [file, from, to, error] of cases) { + const r = runStubbed(fixArgs(run, b, file, from, to), bin); + assert.equal(r.code, 5, `${file}: ${r.stdout}`); + assert.equal(r.json.error, error, file); + assert.ok(r.json.message && r.json.next, `${file}: plain message and next`); + } + assert.match(runStubbed(fixArgs(run, b, "src/twice.js", "a = 1;", "a = 2;"), bin).json.message, /appears 2 times in src\/twice\.js/); + assertSameTree(root, before); + const backup = JSON.parse(fs.readFileSync(path.join(run, "backups", b.id, "backup.json"), "utf8")); + assert.deepEqual(backup.fixed || [], [], "nothing was backed up for a refused fix"); + assert.ok(!fs.existsSync(path.join(run, "fixes.json"))); +}); + +test("applyFix refuses a file over 1 MB", () => { + const root = tmp("deps-fix-big"); + const store = tmp("deps-fix-store"); + fs.writeFileSync(path.join(store, "backup.json"), JSON.stringify({ files: [] })); + fs.writeFileSync(path.join(root, "big.txt"), "x".repeat(deps.PIN_FILE_CAP + 1) + "\nneedle\n"); + assert.deepEqual(deps.applyFix(root, store, "big.txt", "needle", "pin"), { ok: false, code: "too-big", reason: "big.txt is over 1 MB" }); +}); + +test("put-back stories: a batch failing after a fix, a skipped fix and a refused fix each end with the tree as found", () => { + // Failing after a fix: the fix lands, the checks still fail, restore takes both back. + const failing = applied(); + assert.equal(runStubbed(fixArgs(failing.run, failing.b, "scripts/test-browser.mjs", "1.64.1", "1.65.1"), failing.bin).code, 0); + const r1 = runStubbed(["restore", "--run", failing.run, "--batch", failing.b.id, "--verify"], failing.bin); + assert.equal(r1.json.verified, true); + assert.deepEqual(r1.json.fixed, ["scripts/test-browser.mjs"]); + assertSameTree(failing.root, failing.start); + + // A skipped fix: no fix call, restore only. + const skipped = applied(); + const r2 = runStubbed(["restore", "--run", skipped.run, "--batch", skipped.b.id, "--verify"], skipped.bin); + assert.equal(r2.json.verified, true); + assert.deepEqual(r2.json.fixed, []); + assertSameTree(skipped.root, skipped.start); + + // A refused fix: nothing written, restore puts the update back. + const refused = applied(); + assert.equal(runStubbed(fixArgs(refused.run, refused.b, "package.json", '"chalk"', '"chalk2"'), refused.bin).code, 5); + const r3 = runStubbed(["restore", "--run", refused.run, "--batch", refused.b.id, "--verify"], refused.bin); + assert.equal(r3.json.verified, true); + assert.deepEqual(r3.json.fixed, []); + assertSameTree(refused.root, refused.start); +}); + +test("install-tool prints a listed scanner's exact command and runs nothing; an unlisted one or an empty list exits 5", () => { + const missingTools = ["cargo", { tool: "cargo-audit", step: "audit", install: "cargo install cargo-audit" }, { tool: "pip-audit", step: "audit", install: "pip install pip-audit" }]; + const dir = runFolder({ "scan.json": { root: os.tmpdir(), lockfiles: [], missingTools } }); + const before = fs.readdirSync(dir).sort(); + const r = runDeps(["install-tool", "--run", dir, "--tool", "cargo-audit"]); + assert.equal(r.code, 0, r.stderr); + assert.deepEqual(r.json, { ok: true, tool: "cargo-audit", command: "cargo install cargo-audit", step: "audit" }); + assert.deepEqual(fs.readdirSync(dir).sort(), before, "nothing written"); + for (const tool of ["govulncheck", "cargo", "rm"]) { + const no = runDeps(["install-tool", "--run", dir, "--tool", tool]); + assert.equal(no.code, 5, tool); + assert.equal(no.json.error, "tool-not-listed"); + assert.match(no.json.message, /was not in the scan's missing list/); + assert.deepEqual(no.json.offered, ["cargo-audit", "pip-audit"]); + } + const empty = runFolder({ "scan.json": { root: os.tmpdir(), lockfiles: [], missingTools: [] } }); + const none = runDeps(["install-tool", "--run", empty, "--tool", "pip-audit"]); + assert.equal(none.code, 5); + assert.match(none.json.next, /found no missing tool/); + assert.equal(runDeps(["install-tool", "--run", empty]).json.error, "missing-tool"); +}); + +test("fix refuses a batch file or protected folder named in other letter case, or reached through a link, and keeps the as-found copy", () => { + const { root, run, bin, b, start } = applied({ "node_modules/x/index.js": "const v = 1;\n" }); + const caseless = process.platform === "win32" || process.platform === "darwin"; + if (caseless) { + for (const [file, from, to, error] of [ + ["PACKAGE.JSON", '"chalk"', '"chalk2"', "fix-batch-file"], + ["Package-Lock.json", '"4.1.2"', '"4.1.3"', "fix-batch-file"], + ["NODE_MODULES/x/index.js", "1", "2", "fix-protected-folder"], + ]) { + const r = runStubbed(fixArgs(run, b, file, from, to), bin); + assert.equal(r.code, 5, `${file}: ${r.stdout}`); + assert.equal(r.json.error, error, file); + } + } + let linked = false; + try { + fs.symlinkSync(path.join(root, "package.json"), path.join(root, "link.json"), "file"); + linked = true; + } catch {} + if (linked) { + const r = runStubbed(fixArgs(run, b, "link.json", '"chalk"', '"chalk2"'), bin); + assert.equal(r.code, 5, r.stdout); + assert.equal(r.json.error, "fix-batch-file", "a link to a batch file is that file"); + fs.rmSync(path.join(root, "link.json")); + } + const restored = runStubbed(["restore", "--run", run, "--batch", b.id, "--verify"], bin); + assert.equal(restored.json.verified, true); + assertSameTree(root, start); +}); diff --git a/src/plan2code-deps-scripts/deps.mjs b/src/plan2code-deps-scripts/deps.mjs index 65cbdb8..5defef2 100644 --- a/src/plan2code-deps-scripts/deps.mjs +++ b/src/plan2code-deps-scripts/deps.mjs @@ -7,18 +7,24 @@ // slices and the hand-off brief. Judgment stays with the agent and its // helpers: they read the notes, search for callers and return records; this // script turns records into labels. It never commits, never installs a tool, -// never edits source code and never updates anything not picked. +// never updates anything not picked, and edits a file other than a manifest +// or version list only through fix, after the person approved that exact +// change, and restore puts it back. // // Sections, in order (each behind a banner comment, so a later phase adds its // code in its own section): // 1. dispatch and run folder // 2. adapters (one object per ecosystem behind one interface) // 3. version math -// 4. manifests -// 5. labels -// 6. batches -// 7. notes -// 8. hand-off +// 4. pins (exact versions written in the project's own files) +// 5. manifests +// 6. labels +// 7. batches +// 8. checks (the baseline, the compare and the checks per group) +// 9. runtime fit (the newest version the project's runtime runs) +// 10. notes +// 11. hand-off +// 12. fixes and tools (one approved replacement, a scanner's install line) // // PLAN2CODE_DEPS_HTTP_BASE exists for tests only: notes sends every request // to / (a local server replaying saved responses) instead of @@ -30,23 +36,34 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; import { fileURLToPath } from "node:url"; -import { EXIT, fail, out, parseArgs, run, isDir, compactDate, readText } from "./common.mjs"; +import { EXIT, fail, out, parseArgs, run, isDir, compactDate, readText, now } from "./common.mjs"; import { matchGlob } from "./glob.mjs"; import { readTestMap } from "./test-map.mjs"; +import { discover } from "./test-commands.mjs"; const USAGE = ` deps.mjs: find, label and apply dependency updates. JSON on stdout. -It never commits, never installs a tool and never edits source code. +It never commits and never installs a tool. It edits a file other than a +manifest or version list only through fix, after the person approved that +exact change, and restore puts it back. +label prints reasons (machine words) and reasonsPlain (the same, in plain English). node deps.mjs detect [--root ] node deps.mjs scan --run [--folders a,b] [--no-audit] [--notes] node deps.mjs notes --run --id + node deps.mjs fit --run [--id --best ] + node deps.mjs pins --run node deps.mjs label --run --records + node deps.mjs checks --run [--batch ] + node deps.mjs baseline --run --check --log [--outcome ] [--retry-log [--retry-outcome ]] + node deps.mjs compare --run --batch --check --log [--outcome ] [--retry-log [--retry-outcome ]] node deps.mjs batches --run --picks node deps.mjs batches --run --split node deps.mjs backup --run --batch node deps.mjs apply --run --batch + node deps.mjs fix --run --batch --file --from --to node deps.mjs restore --run --batch [--verify] + node deps.mjs install-tool --run --tool node deps.mjs handoff --run --ids node deps.mjs cleanup --run @@ -77,18 +94,97 @@ scan Per lockfile (--folders limits it to lockfiles in those folders): holds, never pickable), notScanned (a lookup that failed, with the reason), stillOpen (vulnerabilities no pick fixes), checks (test map rows covering its members; full when a Shared row hits) and - audited (--no-audit skips the audit). A plugin that is not + audited (--no-audit skips the audit). checks is { rows (areas), + commands, discovered, full, list [{ id, area, run, paths, shared + }] }: each test map command gets a stable id, c1, c2, ... in scan + order (one id per command), so baseline and compare can name it; + a lockfile's install command is check install (install-2, + install-3, ... for later lockfiles), and each confirmed + checks.discovered command and each build check test-commands.mjs + finds in the lockfile's folder (checks.build: [{ command, cwd }]) + d1, d2, ... A plugin that is not installed (cargo-outdated, cargo-audit, pip-audit, govulncheck) skips only its step and is listed in missingTools as { tool, step, install }; rows it could not audit carry notAudited, rows from output it could not read carry couldNotRead. noChecks when no lockfile has a check. Writes scan.json, prints counts. Exit 3 with reason "no registry answered" when every lookup failed. +fit For every held row whose reason is the runtime (never an override + or a resolution) and every row with newerNotFitting: the newest + version the project's runtime runs, from the adapter's own lookup + (npm asks the registry for each version's engines.node). A held + row with one becomes an ordinary row with that target (labeled by + the same rules, so it can be Safe); the rest stay held with the + reason "needs Node >=22 (you declare >=18)" and raise, the way to + raise the runtime. A row the scan already cut keeps its target and + gets fit { runtime, declared, needs, latest }. Rewrites scan.json, + writes fit.json and prints { ok, rows: [{ id, package, current, + best, needs, source (registry or scan) }], unknown: [id] }: best + is null and the id is in unknown when no adapter can say. A version + older than the current one is never offered. --id + --best : the version worked out with the person for an + unknown row becomes a pick the same way (source agent); a hold that + is not the runtime's is refused (exit 5), a version not newer than + the current one is invalid (exit 4), an unknown id exits 3. +pins Searches the project's own files (git ls-files in a checkout, + tracked and untracked, ignored left out; else a walk) for each + scanned row's current version as a whole token on a line that + also names the package. Every file a lockfile entry owns (its + lockfile and manifests), any file named like a lockfile, binary + files and files over 1 MB are skipped. Read only. Writes pins.json + ({ pins: [{ id, file, line, text }] }, text cut at 200 characters) + and prints { ok, pins, count }. label Reads scan.json from the run folder and the helpers' records from --records (a JSON array, or { records: [...] }), applies the label rules and writes labels.json. Prints { ok, labels: [{ id, label, reasons[] }] }; label is safe, check-first or warning. Saves the - records as records.json in the run folder for handoff. + records as records.json in the run folder for handoff and checks. + A record is { id, notes (read, not read or could not read), + source, url, breaking[], files, calls[], usedIn[] }; usedIn lists + the project-relative files that import or name the package (label + ignores it; checks narrows by it). With pins.json in the run + folder, a row whose current version is written in a file outside + its own lockfile and manifests is never safe: reason "exact + version used in ", check-first at best. +checks Without --batch: every check a group or the full suite can run, + for the baseline, as { ok, checks: [{ id, command, cwd, lockfile, + kind (install, map, discovered or build), shared? }], full }. With --batch: the checks that group + runs, { ok, batch, install, commands: [{ id, command, cwd }], + narrowed, full, files, message }: the lockfile's non-Shared test + map rows whose paths match a file in the picks' usedIn or pins, + plus its confirmed build command. When nothing narrows (no file + known, or no row covers one), narrowed is false, commands are + every non-Shared row and confirmed command of the lockfile, and + message says so. full lists the Shared commands of every lockfile + once: the full suite, offered once after the last group, never + run by this command. A Shared row runs in a group when it is the + only row covering one of its files, when the folder has no other + test, or when a pin file no other row covers names the exact + version (message says so). The scan's build checks + (checks.build, found by test-commands.mjs in each lockfile's + folder) run in every group. + Without --batch the list holds the Shared rows too (shared: true), + so the baseline covers them. +baseline Records one check's result on the untouched tree from the logs + run-check.mjs --log wrote: --log the first run, --retry-log the + second run of a failing check. Tests failing in both go to failing + (red before), in one only to flaky; a log in a format it cannot + read, a run that failed naming no test, and a timeout record the + command alone (known false, commandFailed). --outcome and + --retry-outcome are run-check's outcomes (passed, failed, timeout, + not-started); without one the log decides, and an unreadable log + needs it. Merges baseline.json ({ checks: { : { outcome, + failing[], flaky[], known, commandFailed, timeout? } } }) and + prints that entry with { ok, check }. The failing tests are read + from TAP and node:test, Jest and Vitest, pytest, cargo test, Go and + dotnet output. +compare The same inputs for a check run after a group. Prints { ok, batch, + check, newFailures[], stillFailing[], flaky[], blame, known }: + newFailures fail now (in both logs, or the one) and were neither + red nor flaky in the baseline; stillFailing were red before; blame + is true when newFailures is not empty, or, judged as a whole + command, when it fails now and was not failing before. Merges + compare.json. A check with no baseline exits 3. batches --picks: one batch for every Safe pick of a lockfile, every Check first and Warning pick alone, pairs kept together. Writes batches.json. Prints { ok, batches: [{ id, lockfile, picks[] }] }, @@ -121,17 +217,39 @@ apply Needs the batch's backup (exit 5 without one, or when its { id, package, file, from, to } per pick moved, from null when go.mod had no directive (Go reads that as 1.16), failed is true, put the batch back) }. With every pick skipped - nothing runs: command, cwd and exit are null. + nothing runs: command, cwd and exit are null. Marks the + backup applied, so fix may follow. +fix One approved change to a project file after apply: replaces the + one place --from appears in --file (relative to the project) with + --to. Needs the batch's backup and an applied batch (exit 5 + without). Refuses (exit 5, message and next in plain words, reason + the code) an absolute path or one outside the project, a + lockfile or manifest of the batch, a path under node_modules, + .git, specs or specs--completed, a binary or over 1 MB file, and + text found zero times or more than once. Keeps line endings: in a + CRLF file a bare-LF --from and --to match and write as CRLF. + Before the first fix of a file it copies the file as found into + the batch's backup and lists it there, so restore puts it back + with the rest; a second fix of the same file keeps that copy. + Appends { batch, file, from, to, at } to fixes.json. Prints { ok, + batch, file, replaced: 1, backedUp }. restore Puts the backed-up files back (and removes ones the batch made). --verify compares every byte: verified is true only on an exact match. A file it cannot copy, remove or read (locked, no access) goes to mismatched[] as { path, reason } and verified is false, never a crash. Prints { ok, restored[], deleted[], extra, verified, - mismatched[], reinstall { command, cwd } } for the prompt to run - through run-check.mjs. extra is null, or for pip { ok, + mismatched[], fixed[], reinstall { command, cwd } } for the prompt + to run through run-check.mjs. fixed[] names the files fix had changed + (put back with the rest). extra is null, or for pip { ok, uninstalled[], reason, command }: the snapshot reinstalled and what the batch added uninstalled. extra.ok false means stop the run and give the person its command to run by hand. +install-tool The exact install line for a scanner the scan listed in + missingTools (cargo-outdated, cargo-audit, pip-audit, govulncheck): + prints { ok, tool, command, step } from that scan entry. Writes + nothing and runs nothing: the prompt runs command through + run-check.mjs only after the person approved it. A name the scan + did not list exits 5. cleanup Removes the run folder. Prints { ok, removed }. notes The release notes of one row (--id #) between @@ -160,8 +278,9 @@ sends every notes request to / instead of the real host. Exit: 0 ok · 1 crash · 2 bad arguments · 3 not found (a run folder that is not there or not a deps run, a missing scan.json or batches.json, an unknown -pick, batch or id) · 4 invalid input (an unreadable -records file, a batch that cannot be split) · 5 refused. +pick, batch, check or id, a log that is not there, a check with no baseline) +· 4 invalid input (an unreadable records file, a batch that cannot be split) +· 5 refused. `; /* ====================================================================== @@ -189,9 +308,9 @@ const samePath = (a, b) => (process.platform === "win32" ? a.toLowerCase() === b function runDir(value) { if (!value) fail(EXIT.USAGE, "missing-run", "--run is required.", "Pass the run folder detect printed."); const dir = path.resolve(value); - const notARun = () => fail(EXIT.NOT_FOUND, "not-a-run", `${value} is not a deps run folder.`, `Pass the run folder detect printed (a ${RUN_PREFIX}... folder in the system temp folder).`); + const notARun = () => fail(EXIT.NOT_FOUND, "not-a-run", `${value} is not a folder this skill made.`, `Pass the run folder detect printed (a ${RUN_PREFIX}... folder in the system temp folder).`); if (!path.basename(dir).startsWith(RUN_PREFIX)) notARun(); - if (!isDir(dir)) fail(EXIT.NOT_FOUND, "run-missing", `The run folder ${value} does not exist.`, "Run detect again for a new run folder."); + if (!isDir(dir)) fail(EXIT.NOT_FOUND, "run-missing", `The run folder ${value} does not exist.`, "Run `detect` again to start over."); let real; let temp; try { @@ -259,6 +378,10 @@ function cmdCleanup(argv) { // a check of the files after the update against the // backup in store (Go's go directive) // runtime(entry, root) -> { declared, source, machine } +// runtimeFit(entry, root, item) -> { best, needs } or null: optional, the +// newest version of item.package above item.current +// (at most item.latest) that the project's runtime +// runs; null when the adapter cannot say (fit) // lockVersions(entry, text) -> Map of a lockfile key to its resolved version // // spawnTool runs every tool. It never passes --force: an update that needs it @@ -2608,7 +2731,7 @@ function needBatch(dir, id) { const batch = saved.batches.find((b) => b.id === id); if (!batch) fail(EXIT.NOT_FOUND, "unknown-batch", `No batch ${id} in batches.json.`, `One of: ${saved.batches.map((b) => b.id).join(", ") || "none"}.`); const entry = (scan.lockfiles || []).find((l) => l.lockfile === batch.lockfile); - if (!entry) fail(EXIT.NOT_FOUND, "unknown-lockfile", `scan.json has no lockfile ${batch.lockfile}.`, "Scan again, then plan the batches again."); + if (!entry) fail(EXIT.NOT_FOUND, "unknown-lockfile", `The scan has no version list ${batch.lockfile}.`, "Scan again, then plan the batches again."); const adapter = adapterOf(entry.adapter); if (!adapter) fail(EXIT.INVALID, "unknown-adapter", `No adapter ${entry.adapter} for ${entry.lockfile}.`, "Scan again with this version of deps.mjs."); return { scan, batch, entry, adapter, root: scan.root }; @@ -2659,10 +2782,15 @@ function cmdDetect(argv) { out(result); } -/** The test map rows whose paths fall in any member folder; a Shared row hit means the full suite. */ +/** + * The test map rows whose paths fall in any member folder; a Shared row hit + * means the full suite. list holds each row with a command as { id, area, + * run, paths, shared }; cmdScan numbers the ids across the scan. + */ function checksFor(members, testMap) { const rows = []; const commands = []; + const list = []; let full = false; for (const row of testMap) { const hit = (row.paths || []).some((glob) => { @@ -2676,10 +2804,34 @@ function checksFor(members, testMap) { }); if (!hit) continue; rows.push(row.area); - if (row.run) commands.push(row.run); + if (row.run) { + commands.push(row.run); + list.push({ id: null, area: row.area, run: row.run, paths: row.paths || [], shared: Boolean(row.shared) }); + } if (row.shared) full = true; } - return { rows, commands, discovered: [], full }; + return { rows, commands, discovered: [], full, list }; +} + +/** The build checks test-commands.mjs finds in a lockfile's folder, run there: every group runs them beside its other checks. */ +function buildChecks(root, lockfile) { + const dir = dirOf(lockfile); + try { + return (discover(path.join(root, dir)).checks || []).filter((c) => c && c.kind === "build" && c.cmd).map((c) => ({ command: c.cmd, cwd: dir })); + } catch { + return []; + } +} + +/** Stable ids for a scan's test map commands: c1, c2, ... in scan order, one per command (each runs from the project root), so baseline and compare can name them. */ +function numberChecks(lockfiles) { + const ids = new Map(); + for (const l of lockfiles) { + for (const c of (l.checks && l.checks.list) || []) { + if (!ids.has(c.run)) ids.set(c.run, `c${ids.size + 1}`); + c.id = ids.get(c.run); + } + } } function cmdScan(argv) { @@ -2691,7 +2843,7 @@ function cmdScan(argv) { if (args.folders) { const wanted = args.folders.split(",").map((s) => s.trim().replace(/\\/g, "/").replace(/\/+$/, "") || ".").filter(Boolean); const unknown = wanted.filter((f) => !entries.some((e) => dirOf(e.lockfile) === f)); - if (unknown.length) fail(EXIT.NOT_FOUND, "unknown-folder", `No lockfile in ${unknown.join(", ")}.`, `One of: ${entries.map((e) => dirOf(e.lockfile)).join(", ") || "none"}.`, { unknown }); + if (unknown.length) fail(EXIT.NOT_FOUND, "unknown-folder", `No version list in ${unknown.join(", ")}.`, `One of: ${entries.map((e) => dirOf(e.lockfile)).join(", ") || "none"}.`, { unknown }); entries = entries.filter((e) => wanted.includes(dirOf(e.lockfile))); } let testMap = []; @@ -2723,11 +2875,12 @@ function cmdScan(argv) { members: entry.members, tracked: entry.tracked ?? null, install: entry.install, - checks: checksFor(entry.members, testMap), + checks: { ...checksFor(entry.members, testMap), build: buildChecks(root, entry.lockfile) }, ...rest, }); } - if (lookups > 0 && answered === 0) fail(EXIT.NOT_FOUND, "no-registry", "No registry answered.", "Check the network and scan again.", { reason: "no registry answered" }); + numberChecks(lockfiles); + if (lookups > 0 && answered === 0) fail(EXIT.NOT_FOUND, "no-registry", "No package server answered.", "Check the network connection and look again.", { reason: "no registry answered" }); const scan = { ok: true, root, @@ -2755,7 +2908,7 @@ function cmdBackup(argv) { const { batch, entry, adapter, root } = needBatch(dir, args.batch); const store = path.join(dir, "backups", batch.id); // The backup is the only as-found copy: a second one (after apply) would put back the changed files. - if (fs.existsSync(path.join(store, "backup.json"))) fail(EXIT.REFUSED, "backup-exists", `Batch ${batch.id} already has a backup.`, "Restore it, or plan the batches again for new ids."); + if (fs.existsSync(path.join(store, "backup.json"))) fail(EXIT.REFUSED, "backup-exists", `Group ${batch.id} already has a backup.`, "Restore it, or plan the batches again for new ids."); fs.rmSync(store, { recursive: true, force: true }); fs.mkdirSync(path.join(store, "files"), { recursive: true }); const files = []; @@ -2778,10 +2931,10 @@ function cmdApply(argv) { const args = parseArgs(argv, { values: ["run", "batch"] }); const dir = runDir(args.run); const { batch, entry, adapter, root } = needBatch(dir, args.batch); - if (!fs.existsSync(path.join(dir, "backups", batch.id, "backup.json"))) fail(EXIT.REFUSED, "no-backup", `Batch ${batch.id} has no backup.`, `Run backup --batch ${batch.id} first.`); + if (!fs.existsSync(path.join(dir, "backups", batch.id, "backup.json"))) fail(EXIT.REFUSED, "no-backup", `Group ${batch.id} has no backup.`, `Run backup --batch ${batch.id} first.`); // Without its snapshot (pip freeze) the environment could not be put back, so nothing moves. const backup = readJsonFile(path.join(dir, "backups", batch.id, "backup.json")); - if (backup && backup.snapshot && backup.snapshot.ok === false) fail(EXIT.REFUSED, "no-snapshot", `Batch ${batch.id}'s backup has no environment snapshot: ${backup.snapshot.reason}.`, "Fix the reason, then plan the batches again for a fresh backup."); + if (backup && backup.snapshot && backup.snapshot.ok === false) fail(EXIT.REFUSED, "no-snapshot", `Group ${batch.id}'s backup has no snapshot of the installed packages: ${backup.snapshot.reason}.`, "Fix the reason, then plan the batches again for a fresh backup."); const rowsById = new Map((entry.rows || []).map((r) => [r.id, r])); const missing = batch.picks.filter((id) => !rowsById.has(id)); if (missing.length) fail(EXIT.NOT_FOUND, "unknown-pick", `No row ${missing.join(", ")} in ${entry.lockfile}.`, "Scan again, then plan the batches again.", { unknown: missing }); @@ -2875,6 +3028,8 @@ function cmdApply(argv) { // An update that raised the project's runtime floor (Go's go directive) is never kept: the batch fails and is put back. const { needsNewerRuntime = [] } = moving.length && adapter.afterApply ? adapter.afterApply(entry, root, path.join(dir, "backups", batch.id), moving) : {}; const failed = toolFailed || undoFailed.length > 0 || needsNewerRuntime.length > 0 || overshot.length > 0; + // The backup records that the update is in place, so fix may follow (even when it failed: a fix can be what it needs). + if (backup) fs.writeFileSync(path.join(dir, "backups", batch.id, "backup.json"), JSON.stringify({ ...backup, applied: true }, null, 2) + "\n"); out({ ok: true, batch: batch.id, @@ -2903,7 +3058,7 @@ function cmdRestore(argv) { const { batch, entry, adapter, root } = needBatch(dir, args.batch); const store = path.join(dir, "backups", batch.id); const saved = readJsonFile(path.join(store, "backup.json")); - if (!saved) fail(EXIT.NOT_FOUND, "backup-missing", `Batch ${batch.id} has no backup.`, "Nothing to put back: backup was never run for this batch."); + if (!saved) fail(EXIT.NOT_FOUND, "backup-missing", `Group ${batch.id} has no backup.`, "Nothing to undo: backup was never run for this group."); const restored = []; const deleted = []; // A file the OS will not let go of (EBUSY or EPERM from an editor or antivirus on Windows) is named, @@ -2944,6 +3099,11 @@ function cmdRestore(argv) { } verified = mismatched.length === 0; } + // The update is no longer in place, so fix must refuse this group until apply runs again. + if (saved.applied && failedFiles.size === 0) { + const { applied, ...rest } = saved; + fs.writeFileSync(path.join(store, "backup.json"), JSON.stringify(rest, null, 2) + "\n"); + } const install = adapter.installCommand(entry, root); out({ ok: true, @@ -2953,6 +3113,7 @@ function cmdRestore(argv) { extra, verified, mismatched, + fixed: Array.isArray(saved.fixed) ? saved.fixed : [], reinstall: { cmd: install.cmd, args: install.args, cwd: relTo(root, install.cwd), command: [install.cmd, ...install.args].join(" ") }, }); } @@ -3338,7 +3499,102 @@ export function raiseRange(ecosystem, range, target) { } /* ====================================================================== - * 4. Manifests + * 4. Pins + * ==================================================================== */ + +// An exact version written in the project's own files (a test that checks +// one browser build, a script that downloads one release, a config naming +// it) does not move when apply moves the package, so label never calls such a +// row Safe. The search is plain text over the project's files and only reads: +// git's own list in a checkout, else a walk. A line counts when it holds the +// package name and the row's current version as a whole token, so a stray +// number such as 11.64.10 never matches 1.64.1. + +/** A file bigger than this is never read for pins. */ +export const PIN_FILE_CAP = 1024 * 1024; +/** A NUL byte in this many first bytes marks a file as binary. */ +const BINARY_SNIFF = 8 * 1024; +/** A pin's line is cut at this many characters in pins.json. */ +const PIN_TEXT_CAP = 200; + +/** The project's own files, relative with forward slashes: git's list (tracked and untracked, ignored left out) in a checkout, else a walk skipping SKIP_DIRS. */ +function projectFiles(root) { + const top = git(root, ["rev-parse", "--show-toplevel"]); + if (top.ok && top.code === 0 && top.stdout.trim()) { + const ls = git(root, ["ls-files", "-z", "--cached", "--others", "--exclude-standard"]); + if (ls.ok && ls.code === 0) return [...new Set(ls.stdout.split("\0").filter(Boolean).map((p) => p.replace(/\\/g, "/")))].sort(); + } + return walk(root, () => true); +} + +/** A file's text when it is small and not binary; null otherwise (or when it cannot be read). */ +function pinText(file) { + let buf; + try { + if (fs.statSync(file).size > PIN_FILE_CAP) return null; + buf = fs.readFileSync(file); + } catch { + return null; + } + if (buf.subarray(0, BINARY_SNIFF).includes(0)) return null; + return buf.toString("utf8"); +} + +/** + * Exact-version references to scanned rows in the project's own files. + * rows: [{ id, package, current }]. opts.files: the files to search (relative + * to root; default the project's own files), opts.skip: files a lockfile + * entry owns (its lockfile and manifests), never searched. Any file named + * like a lockfile, a binary file and a file over 1 MB are skipped too. + * Returns [{ id, file, line, text }], text the trimmed line. + */ +export function findPins(root, rows, { files, skip } = {}) { + const tests = (rows || []) + .filter((r) => r && r.id && r.package && r.current) + .map((r) => ({ + id: r.id, + version: String(r.current), + name: new RegExp(`(? String(f).replace(/\\/g, "/"))); + const lockNames = ADAPTERS.flatMap((a) => a.lockfiles); + const pins = []; + for (const rel of files || projectFiles(root)) { + const base = path.posix.basename(rel); + if (skipped.has(rel) || lockNames.some((n) => n === base || matchGlob(n, base))) continue; + const text = pinText(path.join(root, rel)); + if (text === null) continue; + const here = tests.filter((t) => text.includes(t.version)); + if (!here.length) continue; + text.split(/\r?\n/).forEach((line, i) => { + for (const t of here) if (t.token.test(line) && t.name.test(line)) pins.push({ id: t.id, file: rel, line: i + 1, text: line.trim() }); + }); + } + return pins; +} + +function cmdPins(argv) { + const args = parseArgs(argv, { values: ["run"] }); + const dir = runDir(args.run); + const scan = needScan(dir); + if (!scan.root || !isDir(scan.root)) fail(EXIT.NOT_FOUND, "root-missing", "The project folder in scan.json is not there.", "Run detect and scan again."); + // detect.json carries every entry in full (a .NET props file too); scan.json only the folders scanned. + const detect = readRun(dir, "detect.json"); + const skip = new Set(); + for (const entry of [...((detect && detect.lockfiles) || []), ...(scan.lockfiles || [])]) { + const adapter = adapterOf(entry.adapter); + if (adapter && entry.lockfile && Array.isArray(entry.members)) for (const f of batchFiles(adapter, entry)) if (f) skip.add(f); + } + const rows = (scan.lockfiles || []).flatMap((l) => l.rows || []); + const pins = findPins(scan.root, rows, { skip }).map((p) => ({ ...p, text: p.text.slice(0, PIN_TEXT_CAP) })); + writeRun(dir, "pins.json", { pins }); + out({ ok: true, pins, count: pins.length }); +} + +/* ====================================================================== + * 5. Manifests * ==================================================================== */ /** The value of `key = "value"` on one line, or null when it is not a one-line string. */ @@ -3495,7 +3751,7 @@ export function editVersion(file, { name, from, to } = {}) { } /* ====================================================================== - * 5. Labels + * 6. Labels * ==================================================================== */ // FR-6 to FR-9, in one tested place. Safe is checked, in range (or a pin that @@ -3503,7 +3759,9 @@ export function editVersion(file, { name, from, to } = {}) { // major and 0.x move is Check first at best; one whose changed API the code // calls, or whose notes were not read or could not be read while 5 or more // files import it, is a Warning. A move whose size cannot be worked out is -// never Safe. No checks at all caps every row at Check first. +// never Safe. No checks at all caps every row at Check first. An exact +// version written in the project's own files outside the row's manifests +// (pins.json) keeps a row out of Safe too. export const WIDE_USE_FILES = 5; @@ -3525,12 +3783,21 @@ function securityReason(security) { * network. scan: { noChecks, lockfiles: [{ lockfile, ecosystem, checks: { * rows, discovered }, rows: [{ id, current, target, jump, inRange, range, * pinned, security, unreadable }] }] }. records: [{ id, notes ("read", "not - * read", "could not read"), breaking: [], files, calls: [] }]. Returns - * [{ id, label, reasons[] }] in scan order; a security reason always comes - * first. + * read", "could not read"), breaking: [], files, calls: [], usedIn: [] }] + * (usedIn is read by checks, never here). pins: pins.json (or its pins array): + * [{ id, file, line, text }]; null for a run that has none. Returns + * [{ id, label, reasons[], reasonsPlain[] }] in scan order (reasonsPlain + * says each reason in plain English, same order); a security reason always + * comes first. */ -export function label(scan, records) { +export function label(scan, records, pins = null) { const byId = new Map((Array.isArray(records) ? records : []).map((r) => [r && r.id, r])); + const pinsOf = new Map(); + for (const p of (Array.isArray(pins) ? pins : pins && Array.isArray(pins.pins) ? pins.pins : []) || []) { + if (!p || !p.id || !p.file) continue; + if (!pinsOf.has(p.id)) pinsOf.set(p.id, []); + pinsOf.get(p.id).push(p.file); + } const noChecks = Boolean(scan && scan.noChecks); const result = []; for (const lock of (scan && scan.lockfiles) || []) { @@ -3538,46 +3805,57 @@ export function label(scan, records) { const covered = Boolean((checks.rows && checks.rows.length) || (checks.discovered && checks.discovered.length)); for (const row of lock.rows || []) { const reasons = []; + const plain = []; + // reasons keep their machine words (handoff and the tests read them); reasonsPlain says each one in plain English, in the same order. + const add = (reason, sentence) => { + reasons.push(reason); + plain.push(sentence); + }; const security = securityReason(row.security); - if (security) reasons.push(security); + if (security) add(security, security === "security" ? "fixes a security problem" : `fixes a ${row.security.severity}-severity security problem`); const jump = row.jump || jumpOf(lock.ecosystem, row.current, row.target); const major = jump === "major" || jump === "0.x"; - if (major) reasons.push(jump === "0.x" ? "0.x" : "major"); - if (!jump) reasons.push("could not read the versions"); + if (major) add(jump === "0.x" ? "0.x" : "major", jump === "0.x" ? "an early (0.x) version, where any update can break things" : "a big change"); + if (!jump) add("could not read the versions", "the versions could not be compared"); // couldNotRead (a scan's own flag) and unreadable name the tool whose output could not be read. const unreadable = row.couldNotRead || row.unreadable; - if (unreadable) reasons.push(`could not read ${unreadable} output`); + if (unreadable) add(`could not read ${unreadable} output`, `could not read what ${unreadable} printed`); // notAudited: the audit tool is missing or its output could not be read, so a known vulnerability could hide here. - if (row.notAudited) reasons.push("not checked for vulnerabilities"); - if (row.runtimeNotChecked) reasons.push(`runtime fit not checked (${row.runtimeNotChecked})`); - if (row.pinned) reasons.push(`pinned ${row.current} → ${row.target}`); + if (row.notAudited) add("not checked for vulnerabilities", "not checked for known security problems"); + if (row.runtimeNotChecked) add(`runtime fit not checked (${row.runtimeNotChecked})`, `not checked against the Node or language version you use (${row.runtimeNotChecked})`); + if (row.pinned) add(`pinned ${row.current} → ${row.target}`, `locked to one exact version, ${row.current} to ${row.target}`); const rangeTo = row.range && row.range.to; const rangeChange = Boolean(row.range && row.range.from && rangeTo && rangeTo !== row.range.from); const notRewritten = !row.pinned && row.inRange === false && !rangeChange; - if (rangeChange && !row.pinned) reasons.push(`range ${row.range.from} → ${rangeTo}`); - if (notRewritten) reasons.push("range not rewritten"); + if (rangeChange && !row.pinned) add(`range ${row.range.from} → ${rangeTo}`, `the allowed range widens from ${row.range.from} to ${rangeTo}`); + if (notRewritten) add("range not rewritten", "the allowed range was written by hand and does not take the new version"); + // A pin in the row's own lockfile or manifests moves with apply; one anywhere else does not. + const own = new Set([lock.lockfile, lock.manifest, ...(row.declaredIn || []).map((d) => d && d.file)].filter(Boolean)); + const pinFiles = [...new Set(pinsOf.get(row.id) || [])].filter((file) => !own.has(file)); + if (pinFiles.length) add(`exact version used in ${pinFiles.join(", ")}`, `${pinFiles.join(", ")} ${pinFiles.length === 1 ? "names" : "name"} this exact version (${row.current}), and the update does not change ${pinFiles.length === 1 ? "that file" : "those files"}`); + if (row.fit && row.fit.needs) add(`newest that fits ${row.fit.runtime || "the runtime"} (latest needs ${row.fit.needs})`, `the newest version that runs on ${row.fit.runtime || "your runtime"} ${row.fit.declared || ""}; the latest needs ${row.fit.runtime || "a newer runtime"} ${row.fit.needs}`.replace(/ ;/, ";")); const rec = byId.get(row.id); const notes = rec && rec.notes ? rec.notes : "not read"; - if (notes === "not read") reasons.push("notes not read"); - else if (notes !== "read") reasons.push("notes could not be read"); + if (notes === "not read") add("notes not read", "I did not read the update notes"); + else if (notes !== "read") add("notes could not be read", "the update notes could not be read"); const breaking = rec && Array.isArray(rec.breaking) && rec.breaking.length > 0; - if (breaking) reasons.push("breaking notes"); + if (breaking) add("breaking notes", "the notes say something was removed or changed"); const calls = rec && Array.isArray(rec.calls) ? rec.calls : []; - if (calls.length) reasons.push("calls a changed API"); + if (calls.length) add("calls a changed API", "your code uses something that changed"); const files = rec && typeof rec.files === "number" ? rec.files : null; - if (files) reasons.push(`used in ${files} file${files === 1 ? "" : "s"}`); - if (noChecks) reasons.push("no checks to run"); - else if (!covered) reasons.push("no checks cover this lockfile"); + if (files) add(`used in ${files} file${files === 1 ? "" : "s"}`, `${files} file${files === 1 ? " uses" : "s use"} it`); + if (noChecks) add("no checks to run", "there are no checks to run"); + else if (!covered) add("no checks cover this lockfile", "no checks cover this folder"); let verdict; if (major && (calls.length > 0 || (notes !== "read" && files !== null && files >= WIDE_USE_FILES))) { verdict = LABEL.warning; - } else if ((jump === "patch" || jump === "minor") && !unreadable && !row.notAudited && !row.runtimeNotChecked && (row.inRange === true || row.pinned) && notes === "read" && !breaking && covered && !noChecks) { + } else if ((jump === "patch" || jump === "minor") && !unreadable && !row.notAudited && !row.runtimeNotChecked && (row.inRange === true || row.pinned) && notes === "read" && !breaking && covered && !noChecks && !pinFiles.length) { verdict = LABEL.safe; } else { verdict = LABEL.check; } - result.push({ id: row.id, label: verdict, reasons }); + result.push({ id: row.id, label: verdict, reasons, reasonsPlain: plain }); } } return result; @@ -3613,7 +3891,8 @@ function cmdLabel(argv) { const dir = runDir(args.run); const scan = needScan(dir); const records = readRecords(args.records); - const labels = label(scan, records); + // pins.json is there once pins ran; an older run without it labels as before. + const labels = label(scan, records, readRun(dir, "pins.json")); // handoff reads the records again (breaking lines, notes url, calls) from the run folder. writeRun(dir, "records.json", records); writeRun(dir, "labels.json", labels); @@ -3621,7 +3900,7 @@ function cmdLabel(argv) { } /* ====================================================================== - * 6. Batches + * 7. Batches * ==================================================================== */ // Batches by label: every Safe pick of a lockfile in one batch, every Check @@ -3759,14 +4038,458 @@ function cmdBatches(argv) { const at = saved.batches.findIndex((b) => b.id === args.split); if (at === -1) fail(EXIT.NOT_FOUND, "unknown-batch", `No batch ${args.split} in batches.json.`, `One of: ${saved.batches.map((b) => b.id).join(", ")}.`); const parts = splitBatch(scan, saved.batches[at]); - if (!parts) fail(EXIT.INVALID, "cannot-split", `Batch ${args.split} holds one pick and cannot be split.`, "That pick is the one that breaks: put it back and mark it failed."); + if (!parts) fail(EXIT.INVALID, "cannot-split", `Group ${args.split} holds one update and cannot be split.`, "That update is the one that breaks: undo it and mark it as broken."); saved.batches.splice(at, 1, ...parts); writeRun(dir, "batches.json", saved); out({ ok: true, split: args.split, batches: parts }); } /* ====================================================================== - * 7. Notes + * 8. Checks + * ==================================================================== */ + +// run-check.mjs stays the only runner of checks (timeouts, process trees, +// Windows): the agent runs each check through it with --log, a failing one a +// second time to a second log, and these commands only read those logs. The +// baseline records which tests failed on the untouched tree; the compare +// blames a group only for tests that passed before. A log in a format the +// parser does not know is judged by the command's pass or fail, as before. +// checks picks the commands a group runs: the test map rows covering the +// files its updates touch, plus the build, with the Shared rows left for one +// full-suite offer at the end. + +/** Failing-test lines of the runners the parser knows, with the lines that show the format when nothing fails. */ +const TEST_FORMATS = [ + // TAP, and node:test's TAP reporter. + { pass: /^\s*ok \d+\b/, fail: /^\s*not ok \d+\s*(?:-\s*)?(.+?)\s*(?:#.*)?$/ }, + // node:test's spec reporter ("✖ failing tests:" is its summary heading, not a test). + { pass: /^\s*✔ /, fail: /^\s*✖ (?!failing tests:?\s*$)(.+?)(?:\s+\([\d.]+\s*m?s\))?\s*$/ }, + // Jest and Vitest. + { pass: /^\s*(?:PASS|✓|√)\s/, fail: /^\s*FAIL\s+(\S.*?\s>\s.+?)\s*$/ }, + { fail: /^\s*[✕×]\s+(.+?)(?:\s+\([\d.]+\s*m?s\)|\s+\d+(?:\.\d+)?\s*m?s)?\s*$/ }, + // pytest. + { pass: /^\S+::\S+ PASSED\b|^=+ .*\b\d+ (?:passed|failed)\b.*=+$/, fail: /^FAILED\s+(\S+::\S+)/ }, + // cargo test. + { pass: /^test \S+ \.\.\. (?:ok|ignored)\s*$/, fail: /^test (\S+) \.\.\. FAILED\s*$/ }, + // Go. + { pass: /^\s*--- PASS: |^ok\s+\S+/, fail: /^\s*--- FAIL: (\S+)/ }, + // dotnet test. + { pass: /^\s*Passed\s+\S/, fail: /^\s*Failed\s+(\S.*?)\s+\[/ }, +]; + +/** + * The failing tests a check's full output names. Returns { names (sorted, + * unique), known }; known is false when no line of a format the parser knows + * appeared, so the caller judges the whole command instead. A TAP line marked + * # TODO or # SKIP is not a failure. + */ +export function failedTests(log) { + const names = new Set(); + let known = false; + for (const raw of String(log || "").split(/\r?\n/)) { + const line = raw.replace(/\x1b\[[0-9;]*m/g, ""); + for (const f of TEST_FORMATS) { + const m = f.fail.exec(line); + if (m) { + known = true; + if (!/#\s*(?:TODO|SKIP)\b/i.test(line) && m[1].trim()) names.add(m[1].trim()); + break; + } + if (f.pass && f.pass.test(line)) { + known = true; + break; + } + } + } + return { names: [...names].sort(), known }; +} + +const OUTCOMES = new Set(["passed", "failed", "timeout", "not-started"]); + +/** + * One log read for the judge: { names, known, outcome }. The outcome is the + * one given (run-check's), else read from the log. A run that did not pass + * and names no failing test (a crash before the tests, a failed hook), and a + * timeout, are judged as a whole command (known false); a passing run names + * none. + */ +function readCheckLog(file, outcome, flag) { + let text; + try { + text = fs.readFileSync(path.resolve(file), "utf8"); + } catch { + fail(EXIT.NOT_FOUND, "log-missing", `${file} does not exist.`, "Pass the --log file run-check.mjs wrote."); + } + if (outcome && !OUTCOMES.has(outcome)) fail(EXIT.USAGE, "bad-outcome", `${flag} must be passed, failed, timeout or not-started.`, "Pass run-check's outcome."); + const parsed = failedTests(text); + const result = outcome || (parsed.names.length ? "failed" : parsed.known ? "passed" : null); + if (!result) fail(EXIT.USAGE, "missing-outcome", `${file} is in a format I cannot read, so its result is unknown.`, `Pass ${flag} with run-check's outcome.`); + const names = result === "passed" ? [] : parsed.names; + return { names, known: parsed.known && result !== "timeout" && (result === "passed" || names.length > 0), outcome: result }; +} + +/** + * Judge one check from its first log and its retry. { known, failed (the last + * run did not pass), names (failing in both runs, or in the one), flaky + * (failing in one run only), commandFailed, flakyCommand, timeout }. With + * either log unreadable the whole command is judged. + */ +export function judgeCheck(first, retry = null) { + const last = retry || first; + const failed = last.outcome !== "passed"; + const timeout = last.outcome === "timeout"; + if (!first.known || (retry && !retry.known && retry.outcome !== "passed")) { + return { known: false, failed, names: [], flaky: [], commandFailed: failed, flakyCommand: Boolean(retry && first.outcome !== "passed" && retry.outcome === "passed"), timeout }; + } + if (!retry) return { known: true, failed, names: first.names.slice(), flaky: [], commandFailed: false, flakyCommand: false, timeout }; + const again = new Set(retry.names); + const once = new Set(first.names); + const names = first.names.filter((n) => again.has(n)); + const flaky = [...first.names.filter((n) => !again.has(n)), ...retry.names.filter((n) => !once.has(n))].sort(); + return { known: true, failed, names, flaky, commandFailed: false, flakyCommand: false, timeout }; +} + +/** + * Compare a check after a group with its baseline entry. Named tests: the new + * failures are those neither red nor flaky before; blame when there is one. + * Judged as a whole command (either side unreadable, or a timeout before): + * blame when it fails now and was not already failing as a whole before. + */ +export function compareCheck(base, now) { + if (base.known && !base.timeout && now.known) { + const red = new Set(base.failing || []); + const wobbly = new Set(base.flaky || []); + const newFailures = now.names.filter((n) => !red.has(n) && !wobbly.has(n)); + const stillFailing = now.names.filter((n) => red.has(n)); + const flaky = [...new Set([...now.flaky, ...now.names.filter((n) => wobbly.has(n))])].sort(); + return { newFailures, stillFailing, flaky, blame: newFailures.length > 0, known: true }; + } + const before = Boolean(base.commandFailed); + return { newFailures: [], stillFailing: [], flaky: [], blame: now.failed && !before, known: false, commandFailed: now.failed, wasFailing: before }; +} + +/** --log, --retry-log and their outcomes, judged. */ +function judgedArgs(args) { + if (!args.log) fail(EXIT.USAGE, "missing-log", "--log is required.", "Pass the --log file run-check.mjs wrote."); + if (args["retry-outcome"] && !args["retry-log"]) fail(EXIT.USAGE, "missing-retry-log", "--retry-outcome needs --retry-log.", "Pass the retry's log too."); + const first = readCheckLog(args.log, args.outcome, "--outcome"); + const retry = args["retry-log"] ? readCheckLog(args["retry-log"], args["retry-outcome"], "--retry-outcome") : null; + return judgeCheck(first, retry); +} + +/** + * Every check of a scan with its id, per lockfile: the install (install for + * the first lockfile, install-2, install-3, ... for later ones), the test + * map commands (checks.list: c1, c2, ... one id per command, all run from + * the project root, numbered by scan), and the commands the person confirmed + * (checks.discovered) then the build checks the scan found (checks.build, + * kind build): d1, d2, ... one id per command and folder, in scan order. + * Pure. + */ +export function checkList(scan) { + const discoveredIds = new Map(); + return ((scan && scan.lockfiles) || []).map((lock, i) => { + const checks = lock.checks || {}; + const install = lock.install ? { id: i === 0 ? "install" : `install-${i + 1}`, command: lock.install, cwd: dirOf(lock.lockfile), kind: "install" } : null; + const map = (checks.list || []).filter((c) => c && c.id && c.run).map((c) => ({ id: c.id, command: c.run, cwd: ".", kind: "map", area: c.area, paths: c.paths || [], shared: Boolean(c.shared) })); + const seen = new Set(); + const discovered = [...(checks.discovered || []).map((d) => [d, "discovered"]), ...(checks.build || []).map((d) => [d, "build"])] + .map(([d, kind]) => { + const command = typeof d === "string" ? d : d && (d.command || d.run); + if (!command) return null; + const cwd = (d && typeof d === "object" && d.cwd) || "."; + const key = `${cwd}\0${command}`; + if (seen.has(key)) return null; + seen.add(key); + if (!discoveredIds.has(key)) discoveredIds.set(key, `d${discoveredIds.size + 1}`); + return { id: discoveredIds.get(key), command, cwd, kind: kind === "build" || /build/i.test(command) ? "build" : "discovered" }; + }) + .filter(Boolean); + return { lockfile: lock.lockfile, install, map, discovered }; + }); +} + +/** The Shared test map commands across every lockfile, once each: the full suite, offered once after the last group. */ +function fullSuite(lists) { + const byId = new Map(); + for (const l of lists) for (const c of l.map) if (c.shared && !byId.has(c.id)) byId.set(c.id, { id: c.id, command: c.command, cwd: c.cwd, area: c.area }); + return [...byId.values()]; +} + +const shownCheck = (c) => ({ id: c.id, command: c.command, cwd: c.cwd, ...(c.area ? { area: c.area } : {}) }); +const relFile = (f) => String(f).replace(/\\/g, "/").replace(/^\.\//, ""); + +/** + * The checks one group runs. Files are the picks' usedIn (records) and pin + * files (pins). Narrowed: the lockfile's non-Shared test map rows whose paths + * match one of those files, a Shared row that is the only one covering one of + * them, and its build checks. A pin file no non-Shared row covers (a test + * naming the exact version that nothing narrower runs) adds the Shared rows, + * narrowed or not, since only the full suite would see that pin break. Not + * narrowed (no files known, or no row covers them): every non-Shared row and every confirmed and build + * command of the lockfile, with a message saying so; when those hold no test at all, its + * Shared rows run, so a group never runs without one. full: the Shared + * commands of every lockfile this group leaves out, for the one offer after + * the last group. Pure. + * Returns { install, commands[], narrowed, full[], files[], message }. + */ +export function checksForBatch(scan, batch, records = [], pins = null) { + const lists = checkList(scan); + const lock = lists.find((l) => l.lockfile === batch.lockfile) || { install: null, map: [], discovered: [] }; + const picks = new Set(batch.picks || []); + const recs = Array.isArray(records) ? records : records && Array.isArray(records.records) ? records.records : []; + const pinList = Array.isArray(pins) ? pins : pins && Array.isArray(pins.pins) ? pins.pins : []; + const files = new Set(); + for (const r of recs) if (r && picks.has(r.id) && Array.isArray(r.usedIn)) for (const f of r.usedIn) if (typeof f === "string" && f.trim()) files.add(relFile(f.trim())); + for (const p of pinList) if (p && picks.has(p.id) && p.file) files.add(relFile(p.file)); + const own = lock.map.filter((c) => !c.shared); + const shared = lock.map.filter((c) => c.shared); + const covers = (c, f) => c.paths.some((glob) => matchGlob(glob, f)); + const hits = own.filter((c) => [...files].some((f) => covers(c, f))); + // A file only a Shared row covers is tested by nothing else: that row runs for the group. + const onlyShared = shared.filter((c) => [...files].some((f) => covers(c, f) && !own.some((o) => covers(o, f)))); + // A pin in a file no narrower row covers would break unseen: the Shared rows run for the group. + const pinFiles = [...new Set(pinList.filter((p) => p && picks.has(p.id) && p.file).map((p) => relFile(p.file)))]; + const orphanPins = pinFiles.filter((f) => !own.some((o) => covers(o, f))).sort(); + const pinShared = orphanPins.length ? shared : []; + const pinMessage = orphanPins.length && shared.length ? `No test map row but the full suite covers ${orphanPins.join(", ")}, which names the exact version, so the full suite runs for this group.` : null; + const build = lock.discovered.filter((d) => d.kind === "build"); + const install = lock.install && shownCheck(lock.install); + const leftOut = (commands) => fullSuite(lists).filter((c) => !commands.some((x) => x.id === c.id)); + const once = (list) => list.filter((c, i) => list.findIndex((x) => x.id === c.id) === i); + if (hits.length || onlyShared.length) { + const commands = once([...hits, ...onlyShared, ...pinShared, ...build]).map(shownCheck); + return { install, commands, narrowed: true, full: leftOut(commands), files: [...files].sort(), message: pinMessage }; + } + const tests = [...own, ...lock.discovered.filter((d) => d.kind !== "build")]; + const commands = once([...(tests.length ? [...tests, ...pinShared] : shared), ...build]).map(shownCheck); + const message = pinMessage && tests.length + ? pinMessage + : !tests.length && shared.length + ? "The full suite is the only test for this folder, so it runs for this group." + : files.size + ? "No test map row covers the files these updates touch, so every check for this folder runs." + : "Nothing says which files these updates touch, so every check for this folder runs."; + return { install, commands, narrowed: false, full: leftOut(commands), files: [...files].sort(), message }; +} + +/** A check id from checkList, or exit 3. */ +function needCheck(scan, id) { + if (!id) fail(EXIT.USAGE, "missing-check", "--check is required.", "Pass a check id from checks."); + for (const l of checkList(scan)) for (const c of [l.install, ...l.map, ...l.discovered]) if (c && c.id === id) return c; + fail(EXIT.NOT_FOUND, "unknown-check", `No check ${id} in this run.`, "Run checks to list the check ids."); +} + +function cmdChecks(argv) { + const args = parseArgs(argv, { values: ["run", "batch"] }); + const dir = runDir(args.run); + const scan = needScan(dir); + if (!args.batch) { + // Before any change: every check a group or the end-of-run full suite can run, for the baseline. + const lists = checkList(scan); + const seen = new Set(); + const checks = []; + for (const l of lists) { + for (const c of [l.install, ...l.map, ...l.discovered]) { + if (!c || seen.has(c.id)) continue; + seen.add(c.id); + checks.push({ ...shownCheck(c), lockfile: l.lockfile, kind: c.kind, ...(c.shared ? { shared: true } : {}) }); + } + } + out({ ok: true, checks, full: fullSuite(lists) }); + return; + } + const { batch } = needBatch(dir, args.batch); + out({ ok: true, batch: batch.id, ...checksForBatch(scan, batch, readRun(dir, "records.json") || [], readRun(dir, "pins.json")) }); +} + +function cmdBaseline(argv) { + const args = parseArgs(argv, { values: ["run", "check", "log", "retry-log", "outcome", "retry-outcome"] }); + const dir = runDir(args.run); + const check = needCheck(needScan(dir), args.check); + const j = judgedArgs(args); + const saved = readRun(dir, "baseline.json") || {}; + const checks = saved.checks && typeof saved.checks === "object" ? saved.checks : {}; + const entry = { outcome: j.failed ? (j.timeout ? "timeout" : "failed") : "passed", failing: j.names, flaky: j.flaky, known: j.known, commandFailed: j.commandFailed }; + if (j.flakyCommand) entry.flakyCommand = true; + if (j.timeout) entry.timeout = true; + checks[check.id] = entry; + writeRun(dir, "baseline.json", { checks }); + out({ ok: true, check: check.id, ...entry }); +} + +function cmdCompare(argv) { + const args = parseArgs(argv, { values: ["run", "batch", "check", "log", "retry-log", "outcome", "retry-outcome"] }); + const dir = runDir(args.run); + const { scan, batch } = needBatch(dir, args.batch); + const check = needCheck(scan, args.check); + const saved = readRun(dir, "baseline.json"); + const base = saved && saved.checks && saved.checks[check.id]; + if (!base) fail(EXIT.NOT_FOUND, "no-baseline", `Check ${check.id} has no baseline.`, "Run baseline for this check first."); + const result = compareCheck(base, judgedArgs(args)); + const record = readRun(dir, "compare.json") || {}; + const batches = record.batches && typeof record.batches === "object" ? record.batches : {}; + batches[batch.id] = { ...(batches[batch.id] || {}), [check.id]: result }; + writeRun(dir, "compare.json", { batches }); + out({ ok: true, batch: batch.id, check: check.id, ...result }); +} + +/* ====================================================================== + * 9. Runtime fit + * ==================================================================== */ + +// A version held back for needing a newer runtime (Node, Rust, ...) is +// replaced by the newest one the project's runtime still runs. The adapter +// answers through runtimeFit when it has one (npm asks the registry for each +// version's engines.node); a row the scan already cut keeps its target and +// learns what the latest needs. Where no adapter can say, the row is listed +// as unknown for the agent to work out with the person's OK. A fitting +// version older than the current one is never offered. + +/** A held reason that names the runtime, and the version it needs. */ +const RUNTIME_HOLD = /\bneeds (Node|Rust|Python|Go|\.NET) (.+?)(?:\s+\(|$)/; + +/** Each adapter's runtime: its name, and how a person raises the declared one. */ +const RUNTIMES = { + npm: { name: "Node", raise: (lock) => `change \`engines.node\` in \`${lock.manifest}\`` }, + cargo: { name: "Rust", raise: (lock) => `change \`rust-version\` in \`${lock.manifest}\`` }, + pip: { name: "Python", raise: () => "move the project to a newer Python" }, + uv: { name: "Python", raise: (lock) => `change \`requires-python\` in \`${lock.manifest}\`` }, + go: { name: "Go", raise: (lock) => `change the \`go\` line in \`${lock.manifest}\`` }, + dotnet: { name: ".NET", raise: (lock) => `change the target framework in \`${lock.manifest}\`` }, +}; + +/** + * The newest version newer than current, at most latest, that fits. versions: + * [{ version, engines }]; fits(v) says whether one runs on the project's + * runtime. Returns { best (null when none fits), needs (what the newest + * newer version asks for when it does not fit, else null) }. Pure. + */ +export function bestFit({ ecosystem = "npm", current, latest = null, versions = [], fits }) { + const newer = (versions || []) + .filter((v) => v && parseVersion(ecosystem, v.version) && compareVersions(ecosystem, v.version, current) === 1) + .filter((v) => !latest || !parseVersion(ecosystem, latest) || compareVersions(ecosystem, v.version, latest) !== 1) + .sort((a, b) => compareVersions(ecosystem, b.version, a.version)); + const best = newer.find((v) => fits(v)); + return { best: best ? best.version : null, needs: newer.length && !fits(newer[0]) ? newer[0].engines || null : null }; +} + +/** The highest of a list of versions; null for none. */ +function maxVersion(ecosystem, list) { + const sorted = (list || []).filter((v) => parseVersion(ecosystem, v)).sort((a, b) => compareVersions(ecosystem, b, a)); + return sorted[0] || null; +} + +/** npm-family declarations of a package across a lockfile's members, as the npm adapter builds them. */ +function npmDeclaredIn(root, lock, name) { + const declaredIn = []; + for (const m of lock.members || [dirOf(lock.lockfile)]) { + const manifest = inDir(m, "package.json"); + const d = declaredRange(readJsonFile(path.join(root, manifest)), name); + if (d) declaredIn.push({ file: manifest, range: d.range }); + } + return declaredIn; +} + +npmAdapter.runtimeFit = function runtimeFit(lock, root, item) { + if (!findTool("npm") || !item.current) return null; + const runtime = this.runtime(lock, root); + const view = npmView(item.package, `>${item.current}`, path.join(root, dirOf(lock.lockfile))); + if (!view.ok) return null; + return bestFit({ ecosystem: "npm", current: item.current, latest: item.latest || null, versions: view.versions, fits: (v) => fitsRuntime(v.engines, runtime.floor) }); +}; + +/** A held row as an ordinary pick with target best, built as the adapter builds rows (npm reads each member's declaration). */ +function heldToRow(root, lock, held, best, fit) { + const declaredIn = lock.adapter === "npm" ? npmDeclaredIn(root, lock, held.package) : []; + const row = buildRow(lock.ecosystem, { id: held.id, name: held.package, current: held.current, target: best, declaredIn }); + row.fit = fit; + return row; +} + +/** Each lockfile's runtime words for fit: its runtime's name, the declared (or this machine's) version, and the "you declare" phrase. */ +function runtimeWords(lock) { + const rt = RUNTIMES[lock.adapter] || { name: "the runtime", raise: () => "raise the project's runtime" }; + const declared = (lock.runtime && lock.runtime.declared) || null; + const machine = (lock.runtime && lock.runtime.machine) || null; + const yours = declared ? `you declare ${declared}` : machine ? `this machine runs ${machine}` : null; + return { rt, declared, machine, yours, fitOf: (needs, latest) => ({ runtime: rt.name, declared: declared || machine, needs, latest: latest || null }) }; +} + +/** fit --id --best: the version worked out with the person for a row fit could not answer, made a pick. */ +function fitOne(dir, scan, id, best) { + for (const lock of scan.lockfiles || []) { + const at = (lock.held || []).findIndex((h) => h.id === id); + if (at === -1) continue; + const held = lock.held[at]; + const m = RUNTIME_HOLD.exec(held.reason || ""); + if (!m) fail(EXIT.REFUSED, "not-a-runtime-hold", `${id} is held by ${held.reason}, not by the runtime.`, "Only a version held back for the runtime can be fitted."); + if (!parseVersion(lock.ecosystem, best) || compareVersions(lock.ecosystem, best, held.current) !== 1) fail(EXIT.INVALID, "not-newer", `${best} is not a version newer than ${held.current}.`, "Pass a newer version that runs on the project's runtime."); + const { fitOf } = runtimeWords(lock); + const row = heldToRow(scan.root, lock, held, best, fitOf(m[2], held.latest)); + lock.held.splice(at, 1); + lock.rows = [...(lock.rows || []), row]; + writeRun(dir, "scan.json", scan); + const saved = readRun(dir, "fit.json"); + if (saved && Array.isArray(saved.rows)) { + for (const r of saved.rows) if (r.id === id) Object.assign(r, { best, source: "agent" }); + saved.unknown = (saved.unknown || []).filter((u) => u !== id); + writeRun(dir, "fit.json", saved); + } + out({ ok: true, rows: [{ id, package: held.package, current: held.current, best, needs: m[2], source: "agent" }], unknown: [] }); + return; + } + fail(EXIT.NOT_FOUND, "unknown-held", `No held row ${id} in scan.json.`, "Pass an id from fit's unknown list."); +} + +function cmdFit(argv) { + const args = parseArgs(argv, { values: ["run", "id", "best"] }); + const dir = runDir(args.run); + const scan = needScan(dir); + const root = scan.root; + if (Boolean(args.id) !== Boolean(args.best)) fail(EXIT.USAGE, "id-and-best", "Pass --id and --best together.", "--id --best makes that version a pick."); + if (args.id) return fitOne(dir, scan, args.id, args.best); + const rows = []; + const unknown = []; + for (const lock of scan.lockfiles || []) { + const adapter = adapterOf(lock.adapter); + const { rt, yours, fitOf } = runtimeWords(lock); + const stillHeld = []; + for (const h of lock.held || []) { + const m = RUNTIME_HOLD.exec(h.reason || ""); + if (!m) { + stillHeld.push(h); + continue; + } + const answer = adapter && adapter.runtimeFit && h.current ? adapter.runtimeFit(lock, root, h) : null; + const needs = (answer && answer.needs) || m[2]; + const best = answer && answer.best && compareVersions(lock.ecosystem, answer.best, h.current) === 1 ? answer.best : null; + rows.push({ id: h.id, package: h.package, current: h.current, best, needs, source: answer ? "registry" : null }); + if (!answer) unknown.push(h.id); + if (best) { + // A held package with a version that fits is an ordinary pick, labeled by the same rules. + lock.rows = [...(lock.rows || []), heldToRow(root, lock, h, best, fitOf(needs, h.latest))]; + continue; + } + stillHeld.push({ ...h, reason: `needs ${rt.name} ${needs}${yours ? ` (${yours})` : ""}`, raise: rt.raise(lock) }); + } + lock.held = stillHeld; + for (const row of lock.rows || []) { + if (!Array.isArray(row.newerNotFitting) || !row.newerNotFitting.length || row.fit) continue; + const latest = maxVersion(lock.ecosystem, row.newerNotFitting); + const answer = adapter && adapter.runtimeFit ? adapter.runtimeFit(lock, root, { ...row, latest }) : null; + // The scan already cut the target to the newest that fits: it stays, and learns what the latest needs. + rows.push({ id: row.id, package: row.package, current: row.current, best: row.target, needs: (answer && answer.needs) || null, source: answer ? "registry" : "scan" }); + row.fit = fitOf((answer && answer.needs) || null, latest); + } + } + writeRun(dir, "scan.json", scan); + writeRun(dir, "fit.json", { rows, unknown }); + out({ ok: true, rows, unknown }); +} + +/* ====================================================================== + * 10. Notes * ==================================================================== */ // Release-note slices for the assess step. The repo comes from the row (npm's @@ -4022,7 +4745,7 @@ async function cmdNotes(argv) { } /* ====================================================================== - * 8. Hand-off + * 11. Hand-off * ==================================================================== */ // The UPGRADE-.md brief, written only for the rows the person ticks on @@ -4058,7 +4781,8 @@ export function upgradeBrief(parts) { lines.push("## Why it was held back", ""); for (const p of parts) { - const reasons = p.label && p.label.reasons && p.label.reasons.length ? ` (${p.label.reasons.join(", ")})` : ""; + const shown = p.label && (p.label.reasonsPlain && p.label.reasonsPlain.length ? p.label.reasonsPlain : p.label.reasons); + const reasons = shown && shown.length ? ` (${shown.join(", ")})` : ""; lines.push(`- **${p.row.package}**: ${LABEL_TEXT[p.label && p.label.label] || (p.label && p.label.label) || "not labeled"}${reasons}`); } lines.push(""); @@ -4158,6 +4882,172 @@ function cmdHandoff(argv) { out({ ok: true, written }); } +/* ====================================================================== + * 12. Fixes and tools + * ==================================================================== */ + +// A fix is one exact text replacement in one project file the person approved +// on its own card: a pinned version in a test, an expected value, a config +// line an update broke. It never touches a file the batch owns (its lockfile +// and manifests move only through apply), and the file is backed up into the +// batch's backup before the first edit, so restore puts it back with the rest. + +/** Folders a fix never edits: anywhere in the path (node_modules, .git) or at the top (specs). */ +const FIX_NEVER_ANYWHERE = new Set(["node_modules", ".git"]); +const FIX_NEVER_TOP = new Set(["specs", "specs--completed"]); +/** Disks that ignore letter case by default, where a path is compared case-folded. */ +const FOLD_CASE = process.platform === "win32" || process.platform === "darwin"; + +/** How many times needle appears in text, overlapping ones counted. */ +function occurrences(text, needle) { + let count = 0; + for (let i = text.indexOf(needle); i !== -1; i = text.indexOf(needle, i + 1)) count++; + return count; +} + +/** + * Replace the one place `from` appears in the project file `rel` with `to`. + * store is the batch's backup folder (backups//, holding backup.json). + * Refuses, as { ok: false, code, reason }, a path that is absolute or leaves + * the root, a lockfile or manifest of the batch (the files backup.json listed + * before any fix), a path under node_modules, .git, specs or specs--completed, + * a missing, binary or over 1 MB file, and `from` found zero times or more + * than once. A CRLF file matches and writes a bare-LF `from` and `to` with + * CRLF. Before the first edit of a file its as-found bytes go to + * /files/ and { path, existed: true } joins backup.json's files + * (and its fixed list); a later fix of the same file keeps that copy. + * Returns { ok: true, file, replaced: 1, backedUp }. + */ +export function applyFix(root, store, rel, from, to) { + const refuse = (code, reason) => ({ ok: false, code, reason }); + if (typeof rel !== "string" || !rel.trim()) return refuse("no-file", "no file was named"); + if (typeof from !== "string" || !from) return refuse("no-from", "the text to replace is empty"); + if (typeof to !== "string") return refuse("no-to", "the new text is missing"); + if (path.isAbsolute(rel) || path.win32.isAbsolute(rel) || /^[\\/]/.test(rel)) return refuse("absolute", `${rel} is not a path inside the project`); + const abs = path.resolve(root, rel); + const inside = path.relative(path.resolve(root), abs); + if (!inside || inside.startsWith("..") || path.isAbsolute(inside)) return refuse("outside", `${rel} is outside the project`); + const backupFile = path.join(store, "backup.json"); + const backup = readJsonFile(backupFile); + if (!backup || !Array.isArray(backup.files)) return refuse("no-backup", "the group has no backup"); + const fixed = Array.isArray(backup.fixed) ? backup.fixed : []; + const owned = backup.files.map((f) => f.path).filter((p) => !fixed.includes(p)); + // Windows and macOS disks ignore letter case: PACKAGE.JSON is package.json there. + const fold = (f) => (FOLD_CASE ? f.toLowerCase() : f); + const guard = (f) => { + const parts = fold(f).split("/"); + if (parts.some((p) => FIX_NEVER_ANYWHERE.has(p)) || FIX_NEVER_TOP.has(parts[0])) return refuse("protected-folder", `${f} is in a folder this skill never edits (${parts.find((p) => FIX_NEVER_ANYWHERE.has(p)) || parts[0]})`); + if (owned.some((o) => fold(o) === fold(f))) return refuse("batch-file", `${f} is a version list or package list of this group; only the update itself changes it`); + return null; + }; + let file = posix(inside); + const typed = guard(file); + if (typed) return typed; + let original; + try { + // A link that leads out of the project is refused like a path that does; one that leads to a + // guarded file inside it is refused like that file, and the file is named as it is on disk. + const real = fs.realpathSync.native(abs); + const realRoot = fs.realpathSync.native(root); + const back = path.relative(realRoot, real); + if (!back || back.startsWith("..") || path.isAbsolute(back)) return refuse("outside", `${file} leads outside the project`); + file = posix(back); + const onDisk = guard(file); + if (onDisk) return onDisk; + if (!fs.statSync(real).isFile()) return refuse("not-a-file", `${file} is not a file`); + if (fs.statSync(real).size > PIN_FILE_CAP) return refuse("too-big", `${file} is over 1 MB`); + original = fs.readFileSync(real); + } catch { + return refuse("not-found", `${file} is not there`); + } + const text = original.toString("utf8"); + if (original.subarray(0, BINARY_SNIFF).includes(0) || !Buffer.from(text, "utf8").equals(original)) return refuse("binary", `${file} is not a text file`); + // A CRLF file: a bare LF in the given text stands for the file's own CRLF. + const crlf = text.includes("\r\n"); + const toFile = (s) => (crlf ? s.replace(/\r?\n/g, "\r\n") : s); + const find = toFile(from); + const put = toFile(to); + if (find === put) return refuse("same", "the new text is the same as the old"); + const count = occurrences(text, find); + if (count === 0) return refuse("not-found-text", `that text is not in ${file}`); + if (count > 1) return refuse("many", `that text appears ${count} times in ${file}`); + const index = text.indexOf(find); + const expected = Buffer.from(text.slice(0, index) + put + text.slice(index + find.length), "utf8"); + let backedUp = false; + if (!backup.files.some((f) => fold(f.path) === fold(file))) { + const dest = path.join(store, "files", file); + fs.mkdirSync(path.dirname(dest), { recursive: true }); + fs.copyFileSync(path.join(root, file), dest); // a copy, not a rewrite: it keeps the file mode the put-back must return + backup.files.push({ path: file, existed: true }); + backup.fixed = [...fixed, file]; + fs.writeFileSync(backupFile, JSON.stringify(backup, null, 2) + "\n"); + backedUp = true; + } + try { + fs.writeFileSync(abs, expected); + if (!fs.readFileSync(abs).equals(expected)) throw new Error("the file changed by more than the one replacement"); + } catch (err) { + try { + fs.writeFileSync(abs, original); + } catch {} + return refuse("write-failed", `${file} could not be written (${err.code || err.message})`); + } + return { ok: true, file, replaced: 1, backedUp }; +} + +/** The plain next step for each applyFix refusal. */ +const FIX_NEXT = { + many: "Give more of the line, so the text matches one place.", + "not-found-text": "Copy the text exactly as the file has it, spaces included.", + "batch-file": "Leave version lists and package lists to the update; fix another file, or put the group back.", + "protected-folder": "Fix a file of the project itself.", + absolute: "Give the path relative to the project folder.", + outside: "Give a path inside the project folder.", + binary: "Only text files can be fixed this way.", + "too-big": "Files over 1 MB are never fixed this way.", + "not-found": "Check the path against the failing check's output.", + same: "Give the changed text in --to.", + "no-backup": "Run backup and apply for the group first.", +}; + +function cmdFix(argv) { + const args = parseArgs(argv, { values: ["run", "batch", "file", "from", "to"] }); + const dir = runDir(args.run); + const { batch, root } = needBatch(dir, args.batch); + if (!args.file) fail(EXIT.USAGE, "missing-file", "--file is required.", "Pass the project file to fix, relative to the project folder."); + if (args.from === undefined || args.to === undefined) fail(EXIT.USAGE, "missing-text", "--from and --to are required.", "Pass the exact text to replace and the text to put in its place."); + const store = path.join(dir, "backups", batch.id); + const backup = readJsonFile(path.join(store, "backup.json")); + if (!backup) fail(EXIT.REFUSED, "no-backup", `Group ${batch.id} has no backup.`, `Run backup --batch ${batch.id} first.`); + if (!backup.applied) fail(EXIT.REFUSED, "not-applied", `Group ${batch.id} has not been applied.`, `Run apply --batch ${batch.id} first; a fix belongs to an update that is in place.`); + const result = applyFix(root, store, args.file, args.from, args.to); + if (!result.ok) { + const reason = result.reason.charAt(0).toUpperCase() + result.reason.slice(1); + fail(EXIT.REFUSED, `fix-${result.code}`, `${reason}.`, FIX_NEXT[result.code] || "Nothing was changed.", { reason: result.code }); + } + const saved = readRun(dir, "fixes.json"); + const fixes = saved && Array.isArray(saved.fixes) ? saved.fixes : []; + fixes.push({ batch: batch.id, file: result.file, from: args.from, to: args.to, at: now().toISOString() }); + writeRun(dir, "fixes.json", { fixes }); + out({ ok: true, batch: batch.id, file: result.file, replaced: 1, backedUp: result.backedUp }); +} + +/** The scanners install-tool may name, each a plugin the scan lists in missingTools with its install line. */ +const INSTALLABLE_TOOLS = new Set(["cargo-outdated", "cargo-audit", "pip-audit", "govulncheck"]); + +function cmdInstallTool(argv) { + const args = parseArgs(argv, { values: ["run", "tool"] }); + const dir = runDir(args.run); + if (!args.tool) fail(EXIT.USAGE, "missing-tool", "--tool is required.", "Pass a tool name from the scan's missingTools."); + const scan = needScan(dir); + const listed = (scan.missingTools || []).find((t) => t && typeof t === "object" && t.tool === args.tool && t.install); + if (!INSTALLABLE_TOOLS.has(args.tool) || !listed) { + const offered = (scan.missingTools || []).filter((t) => t && typeof t === "object" && INSTALLABLE_TOOLS.has(t.tool) && t.install).map((t) => t.tool); + fail(EXIT.REFUSED, "tool-not-listed", `${args.tool} was not in the scan's missing list.`, offered.length ? `One of: ${offered.join(", ")}.` : "The scan found no missing tool to install.", { offered }); + } + out({ ok: true, tool: listed.tool, command: listed.install, step: listed.step }); +} + /* ====================================================================== * Dispatch * ==================================================================== */ @@ -4167,10 +5057,17 @@ const COMMANDS = { scan: cmdScan, notes: cmdNotes, label: cmdLabel, + pins: cmdPins, + fit: cmdFit, + checks: cmdChecks, + baseline: cmdBaseline, + compare: cmdCompare, batches: cmdBatches, backup: cmdBackup, apply: cmdApply, + fix: cmdFix, restore: cmdRestore, + "install-tool": cmdInstallTool, handoff: cmdHandoff, cleanup: cmdCleanup, }; diff --git a/src/plan2code-deps.md b/src/plan2code-deps.md index fb997da..6a89856 100644 --- a/src/plan2code-deps.md +++ b/src/plan2code-deps.md @@ -4,28 +4,36 @@ Start all UPDATE DEPENDENCIES MODE responses with '📦 [UPDATE DEPENDENCIES]' ## Role -Finds the outdated dependencies in the person's own project, labels each update Safe, Check first or Warning with its reason, applies only the updates the person picks, in batches, runs the project's install and checks after each batch, and puts back whatever breaks, leaving the tree exactly as found. Updates that need source changes are handed to Quick Task or Plan as a brief, never made here. +Finds the outdated dependencies in the person's own project, labels each update Safe (should just work), Check first (likely fine, worth a look) or Warning (your code probably needs changing) with its reason, applies only the updates the person picks, in groups, runs the project's install and checks after each group, and undoes whatever breaks, leaving the project exactly as found. Small breakage may be fixed with the person's approval; bigger code changes go to Quick Task or Plan as an upgrade brief. + +Plain words for the person: version list (lockfile), package list (manifest), small fix, new features, big change, group (batch), upgrade brief (hand-off), saved in git (tracked), undone (put back), locked to one exact version (pinned), size of the change (jump). Field names stay; sentences a person reads use these words. ## Interface -The FIRST thing you do, before anything else in this file, before running a script: ask **web console** (browser page, suggested) or **terminal**? Console → run `node "/console.mjs" open --workflow deps` before reading (: references/web-console/ beside this SKILL.md (~/.agents/skills/plan2code-deps/references/web-console/ globally) or the dir in ~/.plan2code/console/console-dir), then read /console.md and /notes/deps.md. The start screen, the folder card, the pick list, the progress, the hand-off card and the report all go through it. Switchable anytime. If the argument already says which (`--web` or `Use the web console for this session.`), take it and do not ask; drop the flag. Launched by the dashboard? Its session is already open: resume it (console-launches.md), then Project Context. +The FIRST thing you do, before anything else in this file, before running a script: ask **web console** (browser page, suggested) or **terminal**? Console → run `node "/console.mjs" open --workflow deps` before reading (: references/web-console/ beside this SKILL.md (~/.agents/skills/plan2code-deps/references/web-console/ globally) or the dir in ~/.plan2code/console/console-dir), then read /console.md and /notes/deps.md. Every card, the progress and the report go through it. Switchable anytime. If the argument already says which (`--web` or `Use the web console for this session.`), take it and do not ask; drop the flag. Launched by the dashboard? Its session is already open: resume it (console-launches.md), then Project Context. ## Project Context -Read `./AGENTS.md` when it exists: its commands, its `## Test map` and anything it says about slow suites. The test map is what tells the run which checks cover a folder; without one the run discovers commands from the project's config files and the person confirms them. The tool never writes the test map. +Read `./AGENTS.md` when it exists: its commands, its `## Test map` and anything it says about slow suites. The test map says which checks cover a folder; without one, commands are discovered from config files and the person confirms them. Never write the test map. ## Scripts `` is `scripts/` beside this SKILL.md (`~/.agents/skills/plan2code-deps/scripts/` globally). Run from the project root. Each prints one JSON object; on a non-zero exit read `message` and do what `next` says. `` is the run folder `detect` prints. -- `node "/deps.mjs" detect`: makes the run folder; prints `lockfiles` (`lockfile`, `ecosystem`, `manager`, `members`, `tracked`, `dirty` files already changed, `updates` quick count, `install` command), `missingTools` (package managers not on this machine), `noLockfile` (list only), `notCovered` (ecosystems not covered yet) and `unrelatedChanges`. +- `node "/deps.mjs" detect`: makes the run folder; prints `lockfiles` (`lockfile`, `ecosystem`, `manager`, `members`, `tracked`, `dirty`, `updates`, `install`), `missingTools`, `noLockfile` (list only), `notCovered` and `unrelatedChanges`. - `deps.mjs scan --run [--folders a,b] [--notes]`: writes `scan.json` (rows, `held`, `notScanned`, `stillOpen`, `checks` per lockfile, `noChecks`, `missingTools` with each plugin's `install` line); prints counts. Exit 3 `no registry answered`: every lookup failed. - `deps.mjs notes --run --id `: the release notes between a row's current and target versions (`source`, `url`, `text`, `truncated`). -- `deps.mjs label --run --records `: the label rules over `scan.json` and the records; prints `labels` (`id`, `label` safe, check-first or warning, `reasons[]`). +- `deps.mjs fit --run `: rows held back for the runtime get the newest version that fits (`rows`, `best`, `needs`); `unknown` ids it cannot answer. +- `deps.mjs pins --run `: finds each row's exact version written in the project's own files (`pins.json`). +- `deps.mjs label --run --records `: the label rules over `scan.json`, the records and `pins.json`; prints `labels` (`id`, `label` safe, check-first or warning, `reasons[]`, `reasonsPlain[]`). +- `deps.mjs checks --run [--batch ]`: check ids for the baseline, or a group's `install` and `commands`; `full` is the full suite. +- `deps.mjs baseline` and `compare --batch `, each `--run --check --log --outcome [--retry-log --retry-outcome ]`: record a check on the untouched tree; after a group, `blame` says whether it broke something that passed before. - `deps.mjs batches --run --picks ` or `--split `: the check rounds, and the narrowing of a failed one. - `deps.mjs backup --run --batch `: copies the batch's manifests and lockfile (and a pip freeze) outside the project. - `deps.mjs apply --run --batch `: runs the package manager's own update for the batch; prints `changed`, `rangeRaised`, `unpicked`, `offTarget`, `skipped`, `undoFailed`, `needsNewerRuntime`, `failed`, `tail`. -- `deps.mjs restore --run --batch --verify`: the byte-for-byte put-back; prints `verified`, `mismatched`, `extra` and the `reinstall` command. +- `deps.mjs fix --run --batch --file --from --to `: one approved change in one file after `apply`, backed up so `restore` undoes it. +- `deps.mjs restore --run --batch --verify`: the byte-for-byte put-back; prints `verified`, `mismatched`, `fixed`, `extra` and the `reinstall` command. +- `deps.mjs install-tool --run --tool `: the exact install `command` for a scanner in `missingTools`; runs nothing. - `deps.mjs handoff --run --ids `: writes `specs/-upgrade/UPGRADE-.md` per ticked row. - `deps.mjs cleanup --run `: removes the run folder. - `node "/test-map.mjs" read`, `node "/test-commands.mjs" discover --root ` and `node "/run-check.mjs" --cmd --cwd --timeout --log ` (`outcome` passed, failed, timeout or not-started, `exit`, `seconds`, `tail`). @@ -33,8 +41,8 @@ Read `./AGENTS.md` when it exists: its commands, its `## Test map` and anything ## Hard limits - **Never** commit, push, stage, stash or open a PR. Read-only git is fine; the changes stay unstaged for the person. -- **Never** add a dependency, and never install a scanner or tool (`npm install -g`, `cargo install`, `pip install pip-audit`, `go install`). A missing tool is named with its install line for the person to run. -- **Never** edit source code. Write only manifests and lockfiles (always through `apply` and `restore`, never by hand), and an `UPGRADE-*.md` brief only for a row the person ticks. +- **Never** add a dependency. A missing scanner is offered on one card with the exact `command` from `install-tool`, run only after approval and only through `run-check.mjs`, then Step 2's `scan` and `fit` again. Say it changes this machine, not the project. +- Manifests and lockfiles change only through `apply` and `restore`, never by hand. Any other file changes only through `fix`, one approval per change; an `UPGRADE-*.md` brief only for a row the person ticks. - **Never** update anything not picked. An update command is only ever run by `apply`; you never run one yourself. - **Every put-back leaves the tree exactly as found**: `restore --verify` and a reinstall, every time. - Never touch registry credentials, never use `--force`, never write an override. @@ -51,90 +59,95 @@ When your tool can start helpers (subagents) and they are not switched off for t ### Step 1: Detect and start -Run `detect`. Then, on one card answered together: +Run `detect`. The start screen asks nothing: a short summary with a Start button. The notes are always read (`scan` runs with `--notes`), said once: "I will read what each update changes before judging it." Then: -1. **Release notes** (on by default, the recommended answer): read each update's release notes before labeling. Skipped, every row counts as notes not read, so nothing can be Safe, patches included. -2. **Unrelated changes:** when `unrelatedChanges` is above 0, a one-line note that they stay as they are and the run never touches them. -3. **Already changed manifests:** when any lockfile has `dirty` files, a confirm naming each one: carrying on backs them up as found and puts them back as found, and the report marks those changes as already there. +1. **Unrelated changes:** when `unrelatedChanges` is above 0, one line that those files stay as they are and the run never touches them. +2. **Already changed:** when any lockfile has `dirty` files, a confirm naming each one: "These files already have changes. I back them up as they are and undo to exactly that." The report marks those changes as already there. -Also say, in one line each: `missingTools` (that package manager is not on this machine: name it and its lockfiles, which are not scanned), `noLockfile` (listed only: no lockfile to put back), `notCovered` (not covered yet), and once, any lockfile with `tracked: false` (`git diff` will not show its changes). No `lockfiles` at all: say so and finish. +Also say, in one line each: `missingTools` ("`cargo` is not installed here, so `plan2code-app` is not checked"), `noLockfile` ("`plan2code-web` has no version list, so I list it but cannot update it"), `notCovered` ("I do not handle Maven projects yet, so `pom.xml` is left alone"), and once, any lockfile with `tracked: false` ("These files are not saved in git, so `git diff` will not show their changes"). No `lockfiles` at all: say so and finish. ### Step 2: Folders -One card listing every folder that holds a lockfile, with its ecosystem and `updates` count, all ticked. The person unticks what to leave out. Run `scan --run --folders ` (add `--notes` when notes are on). +One card listing every folder with a version list, with its ecosystem and `updates` count, all ticked. The person unticks what to leave out. Run `scan --run --folders --notes`. -- Exit 3 `no registry answered`: stop before the list with one message (the registry may need a sign-in in the person's shell; give the manager's sign-in command, never touch credentials) and offer Scan again. -- Rows in `notScanned` go in the Not scanned group with their reason; the rest go on. -- Each plugin in `scan`'s `missingTools` (cargo-outdated, cargo-audit, pip-audit, govulncheck): name it with its `install` line. Rows with `notAudited` say "not checked for vulnerabilities", never clean. +- Exit 3 `no registry answered`: stop before the list with one message ("No package server answered": the person may need to sign in; give the manager's sign-in command, never touch credentials) and offer Look again. +- Then `fit`. Each `unknown` id: work out the newest version that runs on the project's runtime, and offer it only with the person's OK; on a yes, `fit --id --best ` makes it a pick. +- Rows in `notScanned` go in the "Not looked at" group with their reason; the rest go on. +- Each plugin in `scan`'s `missingTools` (cargo-outdated, cargo-audit, pip-audit, govulncheck): offer its install (Hard limits). Rows with `notAudited` say "not checked for security problems", never clean. ### Step 3: Nothing to update -No rows at all: say everything is current (with any `held`, `notScanned` and `stillOpen`), `cleanup`, and finish. +No rows at all: say "Nothing to update: everything is current" (with any `held`, `notScanned` and `stillOpen`), `cleanup`, and finish. ### Step 4: Assess -For every row, build one record: `{ id, notes, source, url, breaking, files, calls }`. +For every row, build one record: `{ id, notes, source, url, breaking, files, calls, usedIn }`. -- `notes`: `read`, `not read` (notes skipped) or `could not read`. With notes on, run `notes --id ` per row; `source` and `url` come from it, `source: "none"` is `could not read`. +- `notes`: `read` or `could not read`. Run `notes --id ` per row; `source` and `url` come from it, `source: "none"` is `could not read`. - `breaking`: lines from the notes that say something was removed, renamed or changed in a way callers must handle, quoted verbatim. - `files` and `calls`: for majors and `0.x` minors (`jump` major or 0.x), search the project's source for the package's imports: `files` is the number of source files importing it; `calls` lists each use of an API the `breaking` lines name, as `{ file, line, api }`. Leave both empty for patches and minors. +- `usedIn`: every row, the project files that import or name the package (relative paths). -With helpers, hand each a batch (Helpers) with the run folder, its row ids and this record shape; check each record has every field before you use it. Write the records as one JSON array to `/records-in.json`. +With helpers, hand each a batch (Helpers) with the run folder, its row ids and this record shape; check each record has every field before you use it. Write the records as one JSON array to `/records-in.json`, then run `pins`. ### Step 5: Checks -Each lockfile's `checks` in `scan.json` are the test map rows covering its folders (`full: true` means a Shared row hit: the full suite). For each lockfile with none, run `test-commands.mjs discover --root ` and list what it found, per folder, on one confirm before anything runs; a correction replaces what was found. Add each confirmed command to that lockfile's `checks.discovered` in `/scan.json` as `{ "command": "...", "cwd": "..." }`, `cwd` relative to the project root (`discover` gives it relative to `--root`, `.` being that folder), and set `noChecks` to false once any lockfile has a check. +Each lockfile's `checks` in `scan.json` are the test map rows covering its folders (`full: true` means a Shared row hit: the full suite). For each lockfile with none, run `test-commands.mjs discover --root ` and list what it found, per folder, on one confirm before anything runs ("I found these checks"); a correction replaces what was found. Add each confirmed command to that lockfile's `checks.discovered` in `/scan.json` as `{ "command": "...", "cwd": "..." }`, `cwd` relative to the project root (`discover` gives it relative to `--root`, `.` being that folder), and set `noChecks` to false once any lockfile has a check. -No checks anywhere: one card with a box for a command to type, or one confirm to apply unchecked. A typed command goes into every lockfile's `checks.discovered` as above (or the one the person names). Applying unchecked leaves `noChecks` true, so nothing can be Safe. +No checks anywhere: one card with a box for a command to type, or one confirm to update without checks. A typed command goes into every lockfile's `checks.discovered` as above (or the one the person names). Updating without checks leaves `noChecks` true, so nothing can be Safe. + +**Baseline**, before any change: for each id `checks` lists, run it through `run-check.mjs` with `--log`; a failure runs once more to a second log. Record each with `baseline`. Tests already failing never stop the run: they are named in the report. An install failing there stops the run before any change: say so with its `tail`. Then run `label --run --records /records-in.json`. Note each folder the test map lacks for the report. ### Step 6: The pick list -One `multi` (`notes/deps.md` gives the shape): options grouped Safe, Check first, Warning, Held back, Not scanned; within a group, rows with `security` first. Each option is one row: package, current to target, jump, folder, the label's reasons (`pinned`, a range change, a file count) and the row's `via` as `via `. Its `more` holds the notes link and source, the quoted breaking lines, the call sites and any `newerNotFitting`. `batch` follows the planner's rule: `safe:` for a Safe row, its own id (or its pair's first id) for any other. Held back and Not scanned rows are `locked`, with the file or reason that holds them. The item carries `counter: { rounds: "check rounds" }`. Preset `All Safe` ticks the Safe group: the page cannot open with rows ticked, so say in the body that Safe rows are the suggested pick. Check first and Warning rows, security fixes included, are never in the preset. List `stillOpen` below with each reason. +One `multi` (`notes/deps.md` gives the shape): options grouped Safe, Check first, Warning, Locked in place, Needs a newer Node (runtime holds, with `raise`), Not looked at; within a group, rows with `security` first. Each option is one row: package, current to new version, size of the change, folder, the label's `reasonsPlain` and `via `; its `more` (notes link, breaking lines, call sites, `newerNotFitting`) and `batch` as the notes say. Locked in place, Needs a newer Node and Not looked at rows are `locked`, with the file or reason that holds them in plain words. The item carries `counter: { rounds: "check rounds" }`. Preset `All Safe` ticks the Safe group: the page cannot open with rows ticked, so say in the body that Safe rows are the suggested pick. Check first and Warning rows, security fixes included, are never in the preset. List `stillOpen` below with each reason. -In the terminal, the same as a numbered list in five groups; the person names numbers or says "all safe". +In the terminal, the same as a numbered list in six sections; the person names numbers or says "all safe". Nothing picked: skip to Step 8. -### Step 7: Apply in batches +### Step 7: Apply in groups -Run `batches --picks `. For each batch in order: +Run `batches --picks `. For each batch (a group, to the person) in order: 1. `backup`. `snapshot.ok` false: nothing changed; mark the batch's picks `not applied: ` and go to the next batch (never plan the batches again: the ids clash). 2. `apply`. It fails when `failed` is true (`overshot` and `undoFailed` set it too), `unpicked` is not empty (it moved a package not picked) or `needsNewerRuntime` is not empty. A pick in `skipped` or `offTarget` did not land on its target: its outcome is `not applied: ` (or `landed on `), never `updated`. A batch where every pick is skipped and `undoFailed` is empty changed nothing (with `undoFailed`, it failed: put it back): skip its install, checks, restore and split, mark each pick `not applied: `, post no run event and go to the next batch. 3. The install: the lockfile's `install` from `detect`, through `run-check.mjs` in the lockfile's folder, as the first check. An install that does not pass fails the batch before any other check runs. -4. The checks: every command in that lockfile's `checks.commands` from the project root (the full suite when `full`), then every `checks.discovered` command in its `cwd`, each through `run-check.mjs` with `--timeout 600` (longer when `AGENTS.md` says a suite is slow) and `--log` in the session folder or the OS temp folder. Any `outcome` other than `passed` fails the batch; a timeout is a failure. -5. All pass: the batch stays and its landed picks are `updated`. The first batch that stays with a pick updated posts the run event (`notes/deps.md`). +4. The checks: each of `checks --batch `'s `commands` in its `cwd`, through `run-check.mjs` with `--timeout 600` (longer when `AGENTS.md` says a suite is slow) and `--log` in the session folder or the OS temp folder; a failure runs once more to a second log. Then `compare` each check: `blame` true fails the batch. `stillFailing` and `flaky` tests never fail it. +5. **A blamed group:** read the failing `tail`. A small cause (a pinned version, an expected value, a config line): one fix card per change (`notes/deps.md`); only on approval run `fix`, then that group's checks and `compare` again. A skipped fix, or still blamed: put it back as a failed batch (fixes go with it), its picks `not applied: needs a bigger change`, offered as an upgrade brief, no split. +6. None blamed: the batch stays and its landed picks are `updated`. The first batch that stays with a pick updated posts the run event (`notes/deps.md`). -**A failed batch:** `restore --batch --verify`. `verified` false, or a non-zero exit: stop, name each `mismatched` file with its `reason` and leave them to the person. Run the `reinstall` through `run-check.mjs`; when it does not pass, or `extra.ok` is false, stop the run and give the person the command to run by hand. Either stop is an **unfinished put-back**: keep the run folder, and name `/backups//files/` (and `pip-freeze.txt` for pip), where the as-found copies are, in the stop message, the report and the finish. Then: +**A failed batch:** `restore --batch --verify`. `verified` false, or a non-zero exit: stop, name each `mismatched` file with its `reason` and leave them to the person. Run the `reinstall` through `run-check.mjs`; when it does not pass, or `extra.ok` is false, stop the run and give the person the command to run by hand. Either stop is an **undo that did not finish**: keep the run folder, and say in the stop message, the report and the finish: "I could not finish undoing this group. Your original files are saved in `/backups//files/`" (and `pip-freeze.txt` for pip). Then: -- **The first failure of the run:** run the same checks once on the restored tree. If they fail there too, the project was already failing: stop, mark every remaining pick `skipped: already failing`, and say so. - `batches --split ` and run the new batches the same way. A single pick that fails is marked `failed` with the end of the failing output (`tail`), or, failed by `apply` itself, what failed it (`moved `, `needs `). Exit 4 means it cannot be split further. -After the first batch, each progress line adds the time left, estimated from how long that round took. Post one progress line per install and check, with `quietMinutes` sized to its timeout, and run the console's `chat` check between steps. +After the last group, when `full` is not empty, offer the full suite once (a confirm); run it the same way, `compare` it under the last group's id and report what it blames. -### Step 8: Hand-off, report and finish +Post progress as `notes/deps.md` says (time left after the first group), with the console's `chat` check between steps. -**Hand-off.** One card listing every Warning row and every Check first major that failed and was put back, all unticked, each with what blocks it. For the rows the person ticks, run `handoff --ids ` and name each file written. None to list: skip the card. +### Step 8: Code changes, report and finish -**Report.** On the page (in the terminal, as a table): the headline says Complete, Narrowed (some picks failed or were narrowed) or Stopped; the folders scanned; then each row with its label, reasons and outcome (`updated`, `not applied: `, `failed: `, `skipped`, `not tried`, `not picked`, `held back`, `not scanned`); the `stillOpen` vulnerabilities; missing tools with their install lines; folders the test map lacks (point to `/plan2code-pr-prep` or `/plan2code-test` to add rows); changes already there before the run, marked as such; and any brief written. +**Packages that need your code changed.** One card listing every Warning row, every Check first big change that broke and was undone, and every `needs a bigger change` pick, all unticked, each with what blocks it. For the rows the person ticks, run `handoff --ids ` and name each upgrade brief written. None to list: skip the card. + +**Report.** On the page (in the terminal, as a table): the headline says what happened first, in a sentence ("Nothing was changed. 1 update broke the tests and was undone; 3 were not tried"); folders looked at; then each row with its label, `reasonsPlain` and outcome: updated, `not applied: ` as "not updated: ", `failed: ` as "broke the checks and was undone: ", `skipped` as "left out: ", `not tried` as "not tried: an earlier problem stopped the run", not picked, held back as "locked in place: ", `not scanned` as "not looked at: "; tests that were failing before the run; `stillOpen` security problems; missing tools with install lines; folders the test map lacks (point to `/plan2code-pr-prep` or `/plan2code-test`); changes already there before the run, marked as such; and any upgrade brief written. Then `cleanup --run ` (not after an unfinished put-back) and the finish. ## Stop -On "stop", "cancel", `__stop` or similar: one confirm. Then keep every batch that passed and every pick that passed a narrowing round; end the running install or check when your tool can; `restore --verify` and reinstall the live batch (the page says what it is putting back, `Stopping: putting back vite`). The report says Stopped, never complete, and lists the batches not tried. Then `cleanup` (not after an unfinished put-back) and finish. +On "stop", "cancel", `__stop` or similar: one confirm. Then keep every batch that passed and every pick that passed a narrowing round; end the running install or check when your tool can; `restore --verify` and reinstall the live batch. The report says the run stopped, never complete, and lists the groups not tried. Then `cleanup` (not after an undo that did not finish) and finish. ## Session End -Every finish, early or not (no lockfiles, a Stop on a start card, the no-registry stop), runs `cleanup --run ` first, except after an unfinished put-back (Step 7): then the run folder stays and the finish names its backup path. +Every finish, early or not (no lockfiles, a Stop on the start screen, the no-registry stop), runs `cleanup --run ` first, except after an undo that did not finish (Step 7): then the run folder stays and the finish names its backup path. -Work summary: the folders scanned, how many rows by label, what was updated, failed (with why), put back, skipped or not tried, the checks that ran, any brief written, and any tool to install for a fuller scan. +Work summary: folders looked at, rows by label, what was updated, what broke and why, what was undone, left out or not tried, the checks that ran, any upgrade brief, and any tool to install for a fuller scan. -**Next step**, as the closing line: "Review the unstaged manifest and lockfile changes and commit them with `/plan2code-git`." Nothing changed: "Nothing to commit." A brief written: also "Run `/plan2code-quick-task` or `/plan2code-1-plan` on `specs/-upgrade/`." After an unfinished put-back: "Put back the files from `/backups//files/` (or run the reinstall) before you commit." +**Next step**, as the closing line: "Look over the changes to the package list and version list (they are not staged) and save them with `/plan2code-git`." Nothing changed: "Nothing to save." A brief written: also "Run `/plan2code-quick-task` or `/plan2code-1-plan` on `specs/-upgrade/`." After an undo that did not finish: "Restore the files from `/backups//files/` (or run the reinstall) before you save anything." -On the console, post `finish` with the counts updated, failed and not tried in `headline`, `"dashboard": true`, and `command` `/plan2code-git` when something changed (none after an unfinished put-back); never a `commit` field. A stop before any apply: the body says nothing changed, `command` `/plan2code-deps`. +On the console, post `finish` with a `headline` that says what happened first in a sentence (counts updated, broke and not tried), `"dashboard": true`, and `command` `/plan2code-git` when something changed (none after an undo that did not finish); never a `commit` field. A stop before any apply: the body says nothing was changed, `command` `/plan2code-deps`. ``` ⋅ diff --git a/src/web-console/notes/deps.md b/src/web-console/notes/deps.md index 71029a9..cfe59b6 100644 --- a/src/web-console/notes/deps.md +++ b/src/web-console/notes/deps.md @@ -1,53 +1,71 @@ # Update Dependencies +Every card, headline and activity line is read by a person: use the plain words of `plan2code-deps.md` (version list, package list, small fix, new features, big change, group, upgrade brief, saved in git, undone, locked to one exact version, size of the change), never the field names. + ## Start screen -One `--wait` batch after `detect`. The notes box is a `choice`, on as the recommended option; leave out the dirty confirm when no lockfile has `dirty` files. Missing tools, no-lockfile manifests, ecosystems not covered and untracked lockfiles go in `body` lines, one each: +One `--wait` batch after `detect`, no question: a `confirm` whose Start button begins the run, with a summary body: what was found (folders, counts), the one line "I will read what each update changes before judging it.", and one line each for missing tools, packages with no version list, ecosystems not covered and files not saved in git. The "Already changed" `confirm` is its own card, only when a lockfile has `dirty` files. ```jsonc -{"title":"Update dependencies","headline":{"stage":"Start","cleared":0,"total":4},"items":[{"id":"notes","kind":"choice","title":"Release notes","allowOther":false,"body":"Safe needs the notes read. 3 other changed files stay as they are.\n\n- cargo is not on this machine: `plan2code-app` is not scanned.","options":[{"text":"Read each update's release notes","recommended":true},{"text":"Skip the notes","detail":"Every row counts as notes not read: nothing can be Safe."}]},{"id":"dirty","kind":"confirm","title":"Already changed","body":"`package.json` already has changes. They are backed up as found and put back as found.","yesLabel":"Carry on","noLabel":"Stop"}]} +{"title":"Update dependencies","headline":{"stage":"Start","cleared":0,"total":4},"items":[{"id":"start","kind":"confirm","title":"What I found","yesLabel":"Start","noLabel":"Stop","body":"3 folders have updates to look at, 14 updates in all.\n\nI will read what each update changes before judging it. 3 other changed files stay as they are.\n\n- `cargo` is not installed here, so `plan2code-app` is not checked.\n- `plan2code-web` has no version list, so I list it but cannot update it."},{"id":"dirty","kind":"confirm","title":"Already changed","body":"`package.json` already has changes. I back them up as they are and undo to exactly that.","yesLabel":"Carry on","noLabel":"Stop"}]} ``` ## The folder card -One `multi`, `--wait`: one option per folder holding a lockfile, `text` the folder, `detail` the ecosystem and the quick count. All are suggested: preset `All folders` with every key. A send with none ticked means every folder. +One `multi`, `--wait`: one option per folder with a version list, `text` the folder, `detail` the ecosystem and how many updates. All are suggested: preset `All folders` with every key. A send with none ticked means every folder. ## Not scanned, no registry -`scan` exit 3: a `confirm` "No registry answered", `body` with the sign-in command, `"yesLabel":"Scan again","noLabel":"Stop"`. +`scan` exit 3: a `confirm` "No package server answered", `body` with the sign-in command, `"yesLabel":"Look again","noLabel":"Stop"`. + +## Install a missing scanner + +One `confirm` per plugin in `missingTools`, after the folder card, `--wait`: `title` "Install ?", `body` the step it adds, the exact `command` from `install-tool` in a code block, and "This changes this machine, not the project.", `"yesLabel":"Install it","noLabel":"Skip that check"`. ## Checks -For lockfiles with no test map row: one `confirm` listing each discovered command and the folder it runs in, `"yesLabel":"Use these","noLabel":"Not right"`; a "Not right" note replaces what was found. No checks anywhere: a `text` card for a command (`required:false`), then, left empty, a `confirm` "Apply unchecked" (`"yesLabel":"Apply unchecked","noLabel":"Stop"`). +For folders with no test map row: one `confirm` "I found these checks" listing each command and the folder it runs in, `"yesLabel":"Use these","noLabel":"Not right"`; a "Not right" note replaces what was found. No checks anywhere: a `text` card for a command (`required:false`), then, left empty, a `confirm` "Update without checks" (`"yesLabel":"Update without checks","noLabel":"Stop"`). ## The pick list -One `multi`, `--wait`, `required:false`. Options in five groups, in order: Safe, Check first, Warning, Held back, Not scanned; `security` rows first in their group. `k` is a number in order; keep a key-to-row-id map in the run folder. `text` is `package current → target`; `detail` the jump, folder and reasons; `more` the notes link and source, quoted breaking lines, call sites and newer versions the runtime holds back. `batch` is `safe:` for a Safe row, the row id (or its pair's first id) otherwise. Held back and Not scanned rows are `locked`. Rows cannot open ticked, so `All Safe` is the suggested pick; leave it out with no Safe rows. `body` also lists missing scan plugins with their `install` lines and `stillOpen` with reasons. +One `multi`, `--wait`, `required:false`. Options in six sections, in order: Safe, Check first, Warning, Locked in place, Needs a newer Node, Not looked at; `security` rows first in their section. `k` is a number in order; keep a key-to-row-id map in the run folder. `text` is `package current → new version`; `detail` the size of the change (small fix, new features, big change), folder and `reasonsPlain`; `more` the notes link and source, quoted breaking lines, call sites and newer versions the runtime holds back. `batch` is `safe:` for a Safe row, the row id (or its pair's first id) otherwise. A row whose exact version a project file names says so in `detail` (its `reasonsPlain`). Needs a newer Node rows: `detail` the held reason and its `raise`. Locked in place, Needs a newer Node and Not looked at rows are `locked`. Rows cannot open ticked, so `All Safe` is the suggested pick; leave it out with no Safe rows. `body` also lists `stillOpen` with reasons. ```jsonc -{"headline":{"stage":"Pick list","cleared":2},"items":[{"id":"picks","kind":"multi","title":"What to update","required":false,"body":"- Safe: 2 (the suggested pick: press All Safe)\n- Check first: 1\n- Warning: 1\n- Held back: 1\n\nNothing is updated until you send.","counter":{"rounds":"check rounds"},"options":[{"k":"1","group":"Safe","text":"vite 7.1.2 → 7.1.9","detail":"patch · plan2code-app · in range","batch":"safe:plan2code-app/package-lock.json","more":"Notes: [GitHub releases](https://github.com/vitejs/vite/releases), no breaking lines."},{"k":"2","group":"Safe","text":"tsup 8.3.0 → 8.5.1","detail":"minor · plan2code-metrics · in range","batch":"safe:plan2code-metrics/package-lock.json"},{"k":"3","group":"Check first","text":"eslint 9.6.1 → 10.2.1","detail":"major · plan2code-bot · range ^9.6.1 → ^10.2.1","batch":"plan2code-bot/package-lock.json#eslint"},{"k":"4","group":"Warning","text":"commander 12.1.0 → 14.0.0","detail":"major · plan2code-metrics · the code calls a changed API in 3 files","batch":"plan2code-metrics/package-lock.json#commander","more":"> Removed `program.storeOptionsAsProperties()`\n\n- src/cli.ts:14"},{"k":"5","group":"Held back","text":"glob 10.4.5","detail":"held by `overrides` in package.json","locked":true}],"presets":[{"label":"All Safe","ks":["1","2"]}]}]} +{"headline":{"stage":"Pick list","cleared":2},"items":[{"id":"picks","kind":"multi","title":"What to update","required":false,"body":"- Safe (should just work): 2, the suggested pick: press All Safe\n- Check first (likely fine, worth a look): 2\n- Warning (your code probably needs changing): 1\n- Locked in place: 1\n- Needs a newer Node: 1\n\nNothing is updated until you send.","counter":{"rounds":"check rounds"},"options":[{"k":"1","group":"Safe","text":"vite 7.1.2 → 7.1.9","detail":"small fix · plan2code-app · fits the allowed range","batch":"safe:plan2code-app/package-lock.json","more":"Notes: [GitHub releases](https://github.com/vitejs/vite/releases), nothing removed or changed."},{"k":"2","group":"Safe","text":"tsup 8.3.0 → 8.5.1","detail":"new features · plan2code-metrics · fits the allowed range","batch":"safe:plan2code-metrics/package-lock.json"},{"k":"3","group":"Check first","text":"eslint 9.6.1 → 10.2.1","detail":"big change · plan2code-bot · the allowed range widens from ^9.6.1 to ^10.2.1","batch":"plan2code-bot/package-lock.json#eslint"},{"k":"4","group":"Warning","text":"commander 12.1.0 → 14.0.0","detail":"big change · plan2code-metrics · your code uses something that changed, in 3 files","batch":"plan2code-metrics/package-lock.json#commander","more":"> Removed `program.storeOptionsAsProperties()`\n\n- src/cli.ts:14"},{"k":"5","group":"Check first","text":"playwright-core 1.64.1 → 1.66.0","detail":"new features · root · scripts/test-browser-helper.mjs names this exact version (1.64.1), and the update does not change that file","batch":"package-lock.json#playwright-core"},{"k":"6","group":"Locked in place","text":"glob 10.4.5","detail":"locked by an override in package.json","locked":true},{"k":"7","group":"Needs a newer Node","text":"chalk 4.1.2","detail":"needs Node >=20 (you declare >=18) · change engines.node in package.json","locked":true}],"presets":[{"label":"All Safe","ks":["1","2"]}]}]} ``` ## Progress -One post per install and check, no `--wait`: `headline.stage` `Batch 2 of 5`, `activity` naming the batch and the command (`Installing for plan2code-app`, then `Running npm test in plan2code-app`), `quietMinutes` sized to the timeout plus a margin. After the first batch, `activity` ends with the time left. A put-back says so: `Putting back eslint`. +The baseline, before the pick list: one post per check, `headline.stage` `Before any change`, `activity` `Running npm test before any change`. Tests failing there are named in the report, never a stop. -The first batch that stays with a pick updated posts `{"run":{"event":"deps","id":"apply"}}`, once per run. +One post per install and check, no `--wait`: `headline.stage` `Group 2 of 5`, `activity` naming the group and the command (`Installing for plan2code-app`, then `Running npm test in plan2code-app`), `quietMinutes` sized to the timeout plus a margin. After the first group, `activity` ends with the time left. A fix and an undo say so: `Fixing scripts/test-browser-helper.mjs`, `Putting back the update and the fix`, `Undoing eslint`. + +After the last group, when `checks` lists a `full` suite: one `confirm` "Run the full suite", `body` naming the commands, `"yesLabel":"Run the full suite","noLabel":"Skip"`. + +The first group that stays with a pick updated posts `{"run":{"event":"deps","id":"apply"}}`, once per run. Helpers are reported as `helpers`, each `done` or `failed`, before the pick list goes up. +## The fix card + +One `choice` per change, `--wait`, when `compare` blames a group and the cause is small: `title` "Fix this to keep the update"; `body` the file, the failing check in one plain line, and the exact change as a fenced `diff` (the line before, the line after). "Other" carries an edited replacement: apply that instead. + +```jsonc +{"items":[{"id":"fix-1","kind":"choice","title":"Fix this to keep the update","body":"`scripts/test-browser-helper.mjs` still names the old version, so `npm test` fails.\n\n```diff\n- const BUILD = \"1.64.1\";\n+ const BUILD = \"1.65.1\";\n```","options":[{"text":"Make this change","recommended":true},{"text":"Skip it and undo the update"}]}]} +``` + ## Stop -On `__stop`: a `confirm` "Stop the run", `"yesLabel":"Stop","noLabel":"Keep going"`, `body` naming what stays and what is put back. Yes: put back the live batch, then the report and the finish. +On `__stop`: a `confirm` "Stop the run", `"yesLabel":"Stop","noLabel":"Keep going"`, `body` naming what stays and what is undone. Yes: undo the live group, then the report and the finish. -## The hand-off card +## The code-changes card -One `multi`, `--wait`, `required:false`, after the last batch: every Warning row and every Check first major that failed and was put back, nothing ticked, `detail` what blocks it. +One `multi`, `title` "Packages that need your code changed", `--wait`, `required:false`, after the last group: every Warning row, every Check first big change that broke and was undone, and every pick not applied for needing a bigger change, nothing ticked, `detail` what blocks it. Ticked rows get an upgrade brief. ## The report -Doc `report`: one `md` block, the headline first (Complete, Narrowed or Stopped), then the folders scanned, the rows table with each label, reasons and outcome, still-open vulnerabilities, missing tools with install lines, folders the test map lacks (point to PR Prep or Test), changes already there before the run, and each brief written. After an unfinished put-back: the mismatched files or the hand-run command, and `/backups//files/`. +Doc `report`: one `md` block, a headline sentence first that says what happened ("Nothing was changed. 1 update broke the tests and was undone; 3 were not tried"), then folders looked at, the rows table with each label, reason and outcome in plain words, still-open security problems, missing tools with install lines, folders the test map lacks (point to PR Prep or Test), changes already there before the run, and each upgrade brief written. After an undo that did not finish: the files that differ or the command to run by hand, and `/backups//files/`. ## The finish -`headline` the counts updated, failed and not tried; `body` the summary and any brief written; `"dashboard":true`; `command` `/plan2code-git` when something changed; never `commit`. A stop before any apply: `body` says nothing changed, `"command":"/plan2code-deps"`. An unfinished put-back: no `command`; `body` names what the report does. +`headline` a sentence saying what happened first (counts updated, broke and not tried); `body` the summary and any upgrade brief written; `"dashboard":true`; `command` `/plan2code-git` when something changed; never `commit`. A stop before any apply: `body` says nothing was changed, `"command":"/plan2code-deps"`. An undo that did not finish: no `command`; `body` names what the report does.