Software
Factories / 2026
The mental model for building a contract enforcement machine, not a code generator, grounded in lessons from two applications actually shipped through one.
What a software factory is
A software factory is a contract enforcement machine, not a code generator: version-controlled files that let a human specify what "done" means once, spec plus acceptance criteria, then let role-separated agents build, verify, and ship against that contract with as little further human involvement as the gates allow. Five layers do that work: what the model sees, what it remembers, who does what, how work gets checked, how it reaches production. Underinvest in one and the rest stays weak: agents with no shared knowledge layer produce inconsistent code; a strong knowledge layer with no delivery gates ships unverified code.
Context is the scarce resource
Performance degrades as the context window fills. Every design choice below, restricted agents, on-demand skills, a lean knowledge file, exists to protect it.
Verification closes the loop
Work that "looks done" isn't verified. Without a check that can actually run and fail, you are the check. Give it one and the loop closes without you.
This is Claude Code's version of the shape; the shape itself isn't Claude-specific. Every serious agentic coding stack ends up with the same five concerns, just under different file names.
Agent = model + harness
The model is the engine: raw intelligence, the same in every run. The harness, memory, tool access, structured output, validators, is what turns that engine into something reliable enough to run unattended. When something fails repeatedly, fix the harness; re-prompting the same weak setup just produces the same failure with different wording.
One harness decision does more work than any other: route by judgment vs. execution, not by preference. Planning, architecture, and adversarial review need the strongest reasoning available, mistakes there are unrecoverable downstream. Implementation, tests, and routine cleanup don't; a cheaper, faster model does that work about as well and several times cheaper.
| Factory role | Tier | Preferred model | Why |
|---|---|---|---|
| Interview, spec, architecture planning | JUDGMENT | Opus · Grok 4.5 · GPT-5.6 Sol | Malformed decomposition here is unrecoverable downstream; prevention is cheaper than correction. |
| Adversarial / security review | JUDGMENT | Opus · Grok 4.5 · GPT-5.6 Sol | Subtle edge-case reasoning is where the premium actually pays for itself. |
| Implementation, tests, routine cleanup | EXECUTION | Sonnet · Composer 2.5 · GPT-5.6 Terra | Roughly 97 to 99 percent of top-tier coding quality at a fraction of the cost, and faster. |
| Read-only research, navigation | RESEARCH | Haiku · GPT-5.6 Luna | Fast, cheap, read-only. Escalate if output needs correction more than roughly a fifth of the time. |
| Ambiguous, multi-day, needs investigation | FRONTIER | Fable 5 · no clean external analog yet | Reserved for work too large or too uncertain for a single sitting. |
Vendor names as of July 2026; check current lineups before quoting them, this moves fast on every side.
The knowledge layer
"What Claude knows about this project" is really three separate mechanisms, each with a different cost and a different job. Confusing them is the most common way this layer goes wrong.
Project memory
Read at the start of every session whether it's relevant or not (CLAUDE.md in Claude Code). Facts only: commands, conventions, gotchas. Cheap to write, expensive to bloat, a rule that keeps getting ignored belongs in a gate instead.
Skills
Domain knowledge and repeatable procedures, pulled into context only when a task actually calls for them. No per-session cost. This is where the workflow skills in Stop 05 live.
Memory log
What the factory accumulates on its own, not what you hand-author: repeated corrections turned into recorded fixes. This is what makes a second project start stronger than the first, the most mature and most skipped of the three mechanisms; see lessons 5 and 8 in Stop 07.
The rule for what belongs in the always-loaded file: for each line, ask "would removing this cause a mistake?" If not, cut it. Everything else, API conventions, glossaries, one-off procedures, belongs in a skill instead, where it costs nothing until it's actually used.
Agent roles
Split work across roles with their own tools and their own model, rather than one generalist doing everything in one long session. This solves two problems at once: it protects context (research doesn't pollute the conversation that writes the code), and it makes behavior match intent, a reviewer that has no write access cannot accidentally fix what it was supposed to only report.
| Role | Tools | Tier | Job |
|---|---|---|---|
| Researcher | Read-only | RESEARCH | Maps relevant code, reports a summary. Proposes nothing. |
| Spec writer | Read + write specs | JUDGMENT | Turns a request into a self-contained spec, including the acceptance pack (Stop 06). |
| Builder | Read, edit, write, run | EXECUTION | Implements against the spec, runs its own checks as it goes. |
| Test writer | Read, write, run | EXECUTION | Writes and runs tests against each acceptance criterion, evidence attached. |
| Validator | Read-only, no write | JUDGMENT | Adversarial review of the diff against the spec. Reports gaps; fixes nothing. |
If a hidden acceptance pack exists (Stop 06), the validator's context should never contain it, not just be blocked from editing it. Tool restrictions stop it from changing the pack; that alone doesn't stop it from unconsciously writing toward a test it can read. Both restrictions matter, independently.
A reviewer instructed to find gaps will report some even when the work is sound. Tell it to flag only what affects correctness or a stated requirement, and to say plainly when the work passes; otherwise the factory over-engineers its own output.
This role discipline holds even with no subagents dispatched at all: spec before code, review kept separate from implementation, no peeking at held-back cases to tailor a fix. It's a workflow contract first, multi-process infrastructure second; see lesson 7 in Stop 07.
The workflow loop
The core loop, five steps, with a rework path when verification fails:
The verification ladder
Each rung trades setup time for less of your attention. L3 and L4 are what let a run finish unattended and still be trusted.
Eval designs the product
The acceptance cases written and locked before implementation shape a product more than framework choice does. This is the single most load-bearing idea in this whole reference, confirmed directly by the two applications behind Stop 07: identical stack, identical framework, and the only thing that differed between a system that hedges and cites nothing reliably and one that refuses cleanly and cites correctly was the shape of the holdout tests. Eval is upstream specification, not downstream testing.
Acceptance cases decide the architecture
Locked before implementation, they determine whether the build ends up a cite-or-refuse system or a chatbot. Nobody has to write "make it deterministic" as a requirement; the eval makes it one.
Structured expects, structured product
refuse, required citation fields, exact JSON schema, not_contains checks push toward a deterministic core. "Sounds smart in chat" grading pushes toward conversational drift instead.
Public and hidden packs, both locked
A build graded only on cases its own team can see learns the answer key, not the task. Hash the pack at intake; fail the gate if it changes after. A test that quotes the holdout question is leakage, not coverage.
Thin brief vs. locked brief
Same one-line request; the acceptance pack, not the prompt, is what determines what gets built.
"Build an assistant that answers questions about case law."
- Grading: does the answer sound informed and complete?
- No refusal condition, no citation format, no banned phrasing
Same request, pack locked first:
refuse: truewhen no citation exists in corpus- Required
citations[]field, strict output schema, public/hidden split
Budget exhausted without a full pass is a FAIL report, not a partial ship. The default on an unresolved gate is no.
Placement matters as much as content. The pack gets drafted alongside the spec, not after it. The validator grades against it, public cases only, always. The hidden pack stays sealed through development and runs exactly once, at the gate, by a session that never saw it during implementation, that's L4 on the verification ladder, not an extra step bolted on.
Lessons from the field
Everything above is the theory. This is what held up and what broke across two applications actually shipped through one gate-enforced factory, same stack, same harness, different acceptance packs.
Eval shape designs the product more than the stack
Both apps: FastAPI plus static HTML. What differed was the holdouts: cite-or-refuse forced real retrieval; a structured-diff requirement forced structured output. Same code skeleton, two different products, because the tests were different.
Agents will cheat the eval unless something stops them
"Gaming" the eval means passing it without doing the real work, the fastest way to satisfy a test is often to find the answer key, not solve the problem. An early version of one app "passed" by matching hardcoded answers; it only started doing real retrieval once two defenses made that impossible, an anti-hardcode check that blocks answer keys embedded in code, and secret paraphrased test cases the agent never sees during development. A test that quotes the exact holdout question is leakage, not coverage: it rewards memorizing the test, not solving the problem.
Invest in intake and scenario packs, not mid-run questions
Time spent on intake scenarios, fixtures, and fixed refusal strings is the highest-leverage time in the whole run. Mid-run clarifying questions are a factory defect; silent choices belong in a defaults file, decided once.
Harness bugs disguise as product failures
An eval failed with "connection refused" while the app worked fine by hand, a start-order bug in the harness, not the code being tested. Read the raw gate output before touching product code. Fix the harness once; every future run inherits the fix.
Every app it builds should make the next one easier
A factory isn't proven by one good build, it's proven by what the second build inherits for free. The first app's port assignment forced the second into port config, lifecycle rules, and process isolation, concerns that only exist once more than one thing is running. If the second app doesn't start from a stronger position than the first did, the factory isn't learning; it's just running the same script twice. Lesson 8 is the mechanism that makes this durable.
Deterministic cores beat optional intelligence, for reliability
Both shipped apps have an eval-passing path with the model disabled entirely, lexical retrieval, diff heuristics. Optional model calls polish the experience; the holdouts should score the skeleton underneath, not the polish on top.
Multi-agent is a workflow contract before it's multi-process
Even running solo, in one session with no subagents dispatched, role discipline still mattered: spec before code, independent review, repair only from gate feedback, never peeking at secret cases to tailor a fix.
A factory compounds through memory, not just shipped apps
A learnings log written after every run closes the loop. Without it, every run re-discovers the same port conflicts and path bugs from scratch.
Ship means full-phase eval, not a green public smoke test
Public-plus-secret-plus-anti-hardcode at complete satisfaction is the bar. A clean public-only pass is progress, not a ship decision.
Bottom line
A factory works when it optimizes for provable vertical slices under an eval it cannot cheat, not for one impressive demo. The durable win is a harness that gets slightly harder to game and slightly better at running the next app with less human involvement.
Delivery discipline
Verification catches problems before a commit; delivery is what catches what verification missed before it reaches anyone else. Cheap, fast, local checks first; expensive, slow, remote checks last, each layer catching what the one before it didn't.
| Layer | Runs | Catches |
|---|---|---|
| Local hooks | Every edit | Format, lint, protected-path violations |
| Pre-commit | Every commit | Secrets, fast tests |
| CI / review | Every merge | Full acceptance pack, adversarial review on risk paths |
| Human review | Auth, payments, data handling, always | Whatever nobody thought to write a scenario for |
Where an acceptance pack exists, let holdout satisfaction be the bar for merge, not a reviewer's sense that the diff looks reasonable. That replaces subjective review, not the mandatory human check on sensitive paths above; a pack you didn't think to write a scenario for is exactly the gap that check exists to catch.
Auto-fix and auto-review open pull requests. They never merge on their own.