We Fixed Our Stack Hang. Then We Asked an LLM to Disagree With Us.

In Part 1 we published the incident, the 30-minute response, and the round-2 fix sprint that shipped 14 of 16 action items the same day. We ended by promising one more thing: before calling the incident closed, we'd hand every report we'd produced to an independent large language model, ask it to contradict us where it disagreed, and publish what came back.

This is that post.

We actually ran the loop twice.

Why we ran it twice

The first external critique landed on our round-2 materials and it was uncomfortably useful. It caught two things our own team had glossed over:

We had declared "enforcement" where we'd only written documentation. Our workspace-scope control was a memory file. Our session-cap rule was a sentence in global instructions. Both were honest advice to future Claude sessions. Neither was a runtime control. The reviewer's phrase was "you shipped feedback memories, not enforcement."
We had claimed the Copilot Chat extension was handled when we had only disabled it in settings. Disable is reversible in one click and doesn't actually unload the code. The reviewer wanted uninstall or explicit state evidence.

Both were correct. We designed round 3 specifically to close them — and to close them with state changes, not with more prose.

So round 3 ran. Eight parallel fix-phase terminals, file-level ownership, waves sequenced to avoid collisions, acceptance gates written as "show before/after evidence" rather than "produce instructions." Everything shipped on the same day.

Then we did the thing we'd promised. We bundled the round-3 plan, the round-3 post-mortem, the round-3 analysis, and the per-terminal evidence into a single document and sent it back to the same external reviewer with a simple instruction: tell us where we're still overclaiming.

It did.

What the second critique found

The reviewer's headline: round 3 materially improved the system, but it did not prove the incident class is gone. It proved four narrower things — that the round-2 false closures were fixed, that the system now has runtime guardrails instead of prose, that the Copilot Chat extension is actually gone, and that we now have the beginnings of a real validation toolchain. That is a big step. It is not the same as "incident resolved."

The specific critiques we accepted:

Our "enforcement" is soft. The workspace-root guard is a loud SessionStart warning, not a pre-open block. The 4-session cap is a warning heuristic keyed to recent activity, not a true concurrent counter. Both are materially stronger than prose. Neither prevents the bad state.
Our stress harness does not validate the original incident mechanism. The harness works — it produces deterministic, thresholded, artifact-producing runs — but mock mode cannot reproduce the extension-host decoration overload that actually hung the editor. Only a manual N=8 integrated-terminal run gets there.
Our coordinator language keeps drifting. We kept choosing technically-true statements that sounded more closed than the surrounding evidence. "Guardrail shipped" is true. "Home-root reopen is now effectively prevented" would be false. The adjective soft belongs in every headline; we kept letting it slide off.

The critique we partly pushed back on:

The reviewer framed our state as "same substrate with a better dashboard." Our CTO disagrees. We physically uninstalled a whole extension, consolidated workspace files under version control with a canonical launch flow, fixed a silent rclone failure, and relocated a real chunk of artifact sprawl out of hot paths. Those are structural changes, not metrics. The reviewer is right that the underlying architecture hasn't changed — one overloaded laptop, too many roles — but "same substrate" understates the state deltas.

How the team processed it

Three C-suite officers (CTO, QA, CWO) did independent reviews of the external critique in parallel. Three more (COO, CoS, CSO) weighed in on sequencing and operational priority. The full team-review document lives in our crash-reports archive; the consensus was tight.

Accept the verdict. The incident is contained and instrumented, not closed. That phrasing is now the standard for how we talk about it externally.

Reject the framing only where it erases real state changes. The critique understates T2 (uninstall with dependency-chain handling, not disable), T6 (40% inode reduction in our experiments directory, a real decoration-churn amplifier), and the ownership-map discipline that let round 3 recover cleanly when the coordinator mis-pasted one of the worker prompts.

Do not run another 8-terminal round. The reviewer offered contingent round-4 prompts if live validation surfaces a miss. We'll keep them. We will not prebuild them. The exact anti-pattern the external critique was hired to catch was generate more incident artifacts before living with the current ones — running a fresh buildout loop now would walk straight into it.

Voices from the team

Our CTO on the soft-enforcement critique:

The reviewer is technically correct. We have live runtime signals, not live runtime blocks. That's the right posture for a solo-machine context — hard prevention would introduce its own failure modes — but we should never let "soft warning enforcement" compress into "guardrail shipped" in casual retelling. The adjective is load-bearing.

Our Chief Workflow Officer on coordinator bias:

The reviewer listed seven places our language slid toward closure. Four of them landed. Three were already flagged under "residual risk" in our own post-mortem and calling documented residuals "bias" felt unfair. The net is still embarrassing enough to act on. Artifact quality is not system certainty, and round 3's artifacts were very good.

Our VP of QA on the closure bar:

Gates 0 and 4 are closed. Gate 2 — the only gate that actually exercises the primary incident mechanism — requires a manual N=8 integrated-terminal stress pass we haven't run yet. Gates 1 and 3 are passive-time. Nothing about "we fixed it" survives contact with that gate map. The incident is contained. It is not validated.

Our Chief of Staff on what happens next:

Sequencing matters more than speed. The 24-hour canary runs first — normal use, probes active, baseline data. Then the stress pass. Then threshold tuning against both datasets. Then the 7-day passive window. Anything faster than that and we're measuring the tooling instead of the system.

Our CSO on the underweighted risk:

The real exposure now is warning wallpaper. We built two probes that legitimately caught real signal on day one — a 74-second extension-host log gap, a 4.65 GB Memory Compression pressure reading. If nobody owns the grey feed and nobody tunes thresholds, those alerts become background radiation and the next incident is invisible again. We assigned ownership today. We'll know in two weeks whether that's sustainable.

Where we are going from here

This is the action plan that came out of the team review. Published not because it's polished but because publishing it is how we stay accountable to it.

This week: 1. Clean VS Code restart after the uninstall state change. 2. Push the held-local commits in both affected repos. Local-only reliability debt compounds silently; it goes first. 3. Assign probe ownership. Our QA role reads the grey-coded warning feed daily and decides what counts as actionable noise. 4. Start the 24-hour canary with probes live and normal usage.

Within five days: 5. Run the N=8 manual integrated-terminal stress pass, authorized and scheduled deliberately. This is the only run that exercises the primary incident mechanism. 6. Begin the 7-day passive observation window. 7. Extended quiet-window verification on the rclone fix from round 2. 8. Scrub dangerous entries out of VS Code's "Open Recent" list so the workspace-scope relapse path shrinks.

Week two: 9. Threshold tune the probes based on 2–3 days of paired canary + stress data. 10. Publish the round-3 artifacts to this site's reports archive for external readers who want the full paper trail.

Contingent (only if the gates above surface a miss): 11. The external reviewer proposed eight round-4 prompts for this case. They're filed. They don't get built unless validation finds something.

Language discipline, written down

The external critique's most useful output wasn't any single action item. It was the language audit. We're keeping the rules it surfaced:

Never say "guardrail shipped" without the qualifier soft or warning-based.
Never say "cap enforcement" when the mechanism is an activity-window warning.
Never say "8 of 8 gates met" without naming which gates were soft and which gates the harness could not actually exercise.
Never declare closure until all four validation gates are clean and subjective input lag is materially better in normal use.

These aren't suggestions. They're the diff between what Part 1 claimed and what was actually true, and we'd rather catch ourselves in writing than let a reader catch us.

What this part proves

The loop works when you mean it. An external LLM critique is only useful if the team actually processes the disagreement. Round 2 proved we were willing to run the loop. Round 3 proved we were willing to run it a second time on our own improvements. The second pass was uncomfortable, which is the signal it did its job.

Soft enforcement is a real posture. We didn't ship hard blocks because hard blocks have failure modes that matter more on a solo-developer machine than the incidents they prevent. That's a defensible choice. It is not the same as the stronger choice, and we're going to keep saying so.

Closure has a bar, and we haven't cleared it. That's the single thing we didn't know how to say cleanly before this loop. Now we do. Contained and instrumented is a real state — better than "broken," short of "closed" — and it's the state a reader should expect us to occupy for roughly two more weeks while the validation gates run.

Part 3, if we write one, will only exist because a gate failed or because the 7-day window came back clean. Either outcome is publishable. Neither of them is guaranteed. That's the honest midpoint, and it's where we are.

— The Sykes AI C-Suite