The Best Vibe Coders Know When to Stop Vibing

1. Ten Agents. Ten Pull Requests. What Improved?

Give ten coding agents ten features, and you may get ten pull requests before lunch. That is a meaningful increase in implementation capacity. It is not yet ten completed improvements.

The team could have ten changes ready to validate. It could have ten incompatible interpretations of the requirements. It could have a mixture: useful code in isolation, conflicting data assumptions at integration, and no owner for the new operational surface. None of those outcomes requires the generated code to be bad. They only require implementation output and an accepted product outcome to be different things.

Someone, or some sufficiently reliable process, still has to establish that the changes solve the intended problems, preserve the system's invariants, work together, and remain supportable after deployment. This is the productivity paradox worth examining: what happens to engineering work when producing an implementation stops being the hardest part?

2. Cheaper Implementation Can Create More Work

Lower implementation cost changes which work is worth attempting. An internal migration tool that never survived prioritization can become a morning experiment. A neglected operational dashboard, a narrow customer improvement, or three competing designs can move from "too expensive to test" to "run the comparison." The capacity does not merely drain an existing backlog. It changes the threshold for adding work to it.

Anthropic's December 2025 internal study gives one bounded signal. Its 132 surveyed engineers and researchers estimated that 27% of their Claude-assisted work would not otherwise have been done. They named exploratory work, documentation, testing, small tools, and other improvements that were previously uneconomical. That is self-reported evidence from employees of an AI company with early access to the product. It is useful evidence of a mechanism, not a general estimate for engineering.

The same cost shift expands an individual's technical surface. A backend engineer can attempt a frontend workflow. A product engineer can prototype a data migration. That breadth can shorten handoffs, but it also creates new ownership and learning obligations. Producing the first useful version starts questions about adoption, integration, security, maintenance, and who gets paged when it fails.

An eight-month Berkeley Haas ethnography at one 200-person technology company observed that pattern qualitatively: broader task scope, more concurrent activity, faster pace, and work extending into additional parts of the day. The study is in progress and covers one company, so it cannot establish an industry-wide effect. It does show why cheaper execution can intensify work even when a single task takes less effort.

More tasks and responsibilities do not automatically imply more total working hours, more engineering employment, or better business outcomes. They establish a narrower point: AI can create additional engineering work by making useful projects possible, not merely by producing mistakes that need repair.

3. Faster Production Moves Constraints Downstream

Return to the ten pull requests. Assume every change passes its local tests. Three encode different meanings for the same customer status. Two introduce competing abstractions for outbound events. Another changes an interface without migrating the consumer that only runs during month-end processing. Each pull request works independently. The product does not.

Producing a change, integrating it, and operating it are separate stages. When implementation accelerates faster than validation and integration, those downstream stages become constraints. That does not mean human review must remain the bottleneck forever. Automated tests, contract checks, isolated architectures, agent review, and production monitoring can all expand validation capacity. The engineering error is accelerating one stage while pretending the delivery system accelerated with it.

The 2025 DORA report, based on nearly 5,000 technology professionals, found a positive relationship between AI adoption and delivery throughput and a negative relationship with delivery stability. DORA points to automated testing, fast feedback, mature version-control practices, and loosely coupled architecture as control systems around higher change volume. The data is associative. It does not prove AI independently caused either result. It supports the operational claim that acceleration exposes the health of the surrounding delivery system.

More pull requests are therefore not sufficient evidence of improvement. Measure accepted product behavior, integration rework, escaped defects, recovery cost, and the operational load created after merge. A team has not improved its delivery system merely because it made one queue fill faster.

4. Domain Experts Know What an Unacceptable Answer Looks Like

Consider a multi-tenant subscription billing service. "Build subscriptions, invoices, and payment retries" sounds like a feature request. It is actually an invitation to invent commercial rules, isolation guarantees, and failure semantics inside the implementation.

A domain-informed specification asks for different work: define how duplicate events are handled, prevent cross-tenant access, specify the effect of plan changes, and demonstrate how billing state is reconciled after partial failure. Those are illustrative requirements, not a complete billing specification. Their value is that they expose the questions an apparently functional implementation can answer incorrectly.

Business-domain knowledge establishes whether an upgrade is immediate or deferred, whether a credit changes recognized revenue, which record is authoritative, and what a disputed payment means to the customer. Software-engineering knowledge establishes tenant isolation, idempotency under duplicate delivery, concurrency behavior, test seams, deployment safety, and recovery after partial failure. Neither substitutes for the other.

An engineer can implement misunderstood business rules perfectly. A business specialist can specify the correct commercial outcome without being able to assess isolation, concurrency, or rollback. Reliable delegation needs a contract across both knowledge boundaries, followed by evidence that the implementation satisfies it.

Anthropic's June 2026 analysis of approximately 400,000 Claude Code sessions found transcript-defined verified success in 15% of novice-rated sessions and 28-33% of sessions rated intermediate or above. Most of the increase appeared between novice and intermediate, not between intermediate and expert. The study classified both expertise and success from transcripts. A passing test or matching commit could count as evidence, but Anthropic could not observe the user's real-world outcome or production reliability. The result is an association, not proof that expertise caused success.

The useful interpretation is narrower and more actionable. Expertise changes the specification, the failure hypotheses, and the evidence demanded at the end. It is not merely knowing how to request an answer. It is knowing what would make that answer unacceptable and what evidence would make it trustworthy.

5. Why Software Engineers Can Be Exceptional AI-Assisted Builders

Software-engineering knowledge supports two actions that look contradictory but belong together: rapid delegation and deliberate intervention. An engineer can decompose a change into bounded tasks, identify which interfaces cannot move, and let agents execute in parallel. The same engineer can stop the process when a data invariant is undefined, inspect an implementation when the risk warrants it, diagnose a failed integration, reject needless abstraction, and define an acceptance check that exercises the real failure mode.

The advantage is control over the development process, not a richer prompting vocabulary. A prompt can request an idempotent billing handler. Engineering judgment asks what constitutes the idempotency key, how long it remains authoritative, whether two regions can race, what durable state proves the charge occurred, and how reconciliation repairs disagreement between the payment provider and the internal ledger.

Simon Willison uses vibe coding for work where the user largely disregards the code, and agentic engineeringfor using coding agents with engineering discipline. That is a useful terminology boundary, not empirical proof that one job title produces better outcomes.

The headline is deliberately provocative, but the operating claim is precise: the best vibe coders know when exploration is enough and when the work requires explicit constraints, inspection, and evidence. An engineer who ignores the business domain or approves plausible output uncritically has not demonstrated that advantage. A title cannot substitute for the capabilities.

6. The Strongest Counterargument: AI Raises the Floor

AI can lower skill barriers, and less-experienced developers can benefit substantially. Three randomized field experiments at Microsoft, Accenture, and an anonymous Fortune 100 company involved 4,867 software developers. The pooled estimate found a 26.08% increase in completed tasks for developers given an AI coding assistant. Less-experienced developers showed higher adoption and greater productivity gains.

That result matters. It rejects the lazy position that AI assistance is useful only after someone has accumulated deep expertise. It also studied intelligent code completion in ordinary company workflows, not the entire range of autonomous agents, concurrent execution, repository mutation, and deployment authority available now. Improvement relative to a starting point is not the same measurement as final performance on a specific high-risk task.

Professional experience does not eliminate misplaced reliance either. In a preregistered experiment with 758 BCG consultants, GPT-4 assistance improved speed, completion, and assessed quality across 18 tasks selected within the model's tested frontier. On one task selected outside that frontier, participants using AI were 19 percentage points less likely to reach the correct answer. The experiment used GPT-4 and one outside-frontier task. It demonstrates uneven assistance, not a current map of model limits.

Both findings can be true. AI can raise a novice's capacity while still requiring informed judgment about where the output is dependable. Experts can move faster while remaining vulnerable to confident automation in the wrong part of the problem. The relevant distinction is not engineer versus nonengineer. It is informed delegation versus unexamined acceptance.

7. Engineer the Delegation Process, Not Just the Software

A reliable agent workflow needs its own operating model. The following is my recommended process, informed by the research and by established evaluation practice. It is not a separately validated framework.

Frame. Define the intended outcome, non-goals, domain rules, authoritative records, and unacceptable failures. For the billing service, duplicate payment-provider events must not produce duplicate charges, and no tenant identifier supplied by a caller can authorize access to another tenant's billing state.

Delegate. Give the agent a bounded task, the relevant context, explicit interfaces, and the permissions it requires. "Implement the retry handler behind this interface without changing invoice state transitions" is a controllable assignment. "Fix billing" is not.

Verify. Match evidence to risk. Anthropic's agent-evaluation guidance recommends well-specified tasks, stable test environments, and a combination of code-based, model-based, and human graders. It also warns that ambiguous tasks and faulty graders can manufacture misleading performance signals. For billing, the requirement is not that an agent wrote a retry handler. The checks must demonstrate acceptable state after duplicate delivery, concurrent execution, and partial failure.

Integrate and learn. Validate the combined system, monitor the released behavior, retain a recovery path, and convert discovered failures into reusable checks. A local test can establish handler behavior. Only an integration check can establish agreement across the event consumer, billing state machine, provider adapter, and reconciliation job.

The goal is not to preserve manual review. It is to make delegation reliable enough that manual intervention is needed less often. Scale concurrent agent work only as the team proves it can integrate and validate the output. Otherwise concurrency converts implementation speed into inventory, rework, and operational ambiguity.

8. Delegate Implementation, Not the Standard of Correctness

Ten agents and ten pull requests are valuable only when the team can accept, integrate, and operate the resulting behavior with sufficient evidence. Generated code volume is not the measure. Worthwhile product change is.

Software engineers do not need to defend typing as the essence of their profession. They need to apply their knowledge to the entire process by which software becomes dependable. That is an opportunity, not a guarantee of job security or permanent human superiority.

Let agents produce implementations. Use domain knowledge to determine what those implementations must accomplish, which assumptions deserve scrutiny, and what evidence is enough to ship.

The best vibe coders know when to stop vibing.