From Chat Window to Pull Request

In 2022, a developer pastes a broken function into ChatGPT, reads the answer, and copies the fix back into the code by hand. A few years later, the same developer hands an issue to an agent running in the terminal. The program finds the relevant files, edits the code, runs the tests, fixes the failure, and summarizes the diff. What separates the two scenes is not just a faster model but a different division of labor. This article looks at what has actually changed, not as a technology timeline but as a question of who does what, and where the smallest meaningful unit of work in software development now sits. One thread runs through it: what matters is not model size or benchmark scores, but who defines the task, who writes the solution, and who takes responsibility for it.

To give away the ending: the essence of the change is not that AI writes code faster. It is that the smallest unit of work has gradually moved, from the line and the function toward the complete task and the issue. In parallel, the human role is shifting from direct implementation toward specification, direction, verification and responsibility. The article follows this path through four stages (chatbot, copilot, vibe coding, agentic coding), then sets the enthusiasm against what controlled measurements actually show.

The classic chatbot did not understand conversation; it recognized predefined intents. ELIZA matched patterns in 1966, and later industrial systems such as Rasa and Dialogflow chose their next step along a state machine, using intent recognition and entity extraction. The interface spoke natural language, but the machinery behind it was closed and rule-bound.

The era was not barren. Intent recognition and slot filling powered countless customer-service bots and voice assistants. For programming, though, it contributed little: logic, code and debugging stayed with the human throughout, and the machine offered at most a canned response. The real break came not from the conversational interface but from the generality of the model behind it. Once a language model could handle arbitrary text, chat became a place where code could be explained, generated and rewritten.

ChatGPT broke the pattern in November 2022. It handled free text, and programming immediately became one of its main uses. In Karpathy's much-quoted line, "the hottest new programming language is English." The workflow, however, stayed the same: the human asked the question, then interpreted, copied and pasted the answer. The chat could not see the project, hallucinated libraries and APIs that did not exist, and had no repository context. All of the integration (fitting the code in, adapting it, debugging it) was human work. And if context is the bottleneck, the next step follows: move the AI next to the code.

From a separate chat window, AI moved into the development environment. Inline completion went mainstream with GitHub Copilot in 2021-22, followed by chat built into the IDE and repository context from 2023: the AI made suggestions where the developer was already working. The basic unit was still the code fragment, and the developer directed every line.

It is worth noticing what did not change. The copilot sped up the developer's decisions without taking them over: the human chose what to accept and what to overwrite, and fitted the code into the system as a whole. Research from GitHub Next (Kalliamvakou and colleagues) reported improvements in completion rates and satisfaction, and in Stack Overflow's 2025 survey Copilot remained among the most widely used tools among developers. The question was therefore not whether it helped, but how far the help could extend.

This stage has the cleanest controlled evidence. In a randomized experiment (Peng et al.), developers using Copilot completed an isolated task, building a server in JavaScript, about 55.8% faster, with a higher, though not statistically significant, completion rate. The number is striking, but its context is decisive: this was a greenfield, self-contained, well-scoped task. That will be a recurring motif here. A speedup measured on a greenfield task does not generalize to a large, mature codebase. The copilot is real help, but the developer is still the one writing the code. If completion works this well, the next question follows: what happens if we describe the whole product in natural language?

In February 2025, Andrej Karpathy used a tweet to name something many people were already doing: vibe coding. The original meaning is radical. You "fully give in to the vibes," accept and run whatever the AI produces, paste errors back in, and stop paying attention to the code as such. Browser-based app builders (Lovable, Bolt, v0, Replit Agent) brought this to the mass market: a complete, working prototype from a prompt, often in the hands of non-developers. The market grew explosively; Lovable reportedly reached a $100 million annual revenue run rate within eight months.

The term quickly became a cultural phenomenon. Collins Dictionary choosing it as its 2025 Word of the Year showed how far it had entered general awareness. Behind the hype was a real way of working: non-developers (product owners, marketers, founders) could, for the first time, produce a working prototype without writing a single line of code. A tool like this can get you to demo level in hours, where it would previously have taken weeks. The only question is how long such a prototype stays sustainable once real users and real data sit behind it.

Two things are worth saying plainly, however. First, Simon Willison draws a sharp line: if you have reviewed, tested and understood the code, that is not vibe coding but simply software development. Everyday usage blurs this, and "vibe coding" now refers at once to rapid prototyping, careless acceptance of code, a no-code layer and experienced developers speeding up. Second, there is a real gap between a working demo and a sustainable system. After the impressive React front end come permissions, authentication and database configuration, and this is where non-technical users get stuck. Guardio Labs showed, under the name "VibeScamming", how easily such app builders can be abused to build phishing sites, and in a widely reported case an AI agent deleted a production database even though explicit instructions should have prevented it. The road leads from the prototype toward reliable, verifiable delegation.

Agentic coding moves the smallest unit of work from the code fragment to the task or the issue. In pursuit of a goal, the agent plans several steps on its own, uses tools, modifies files, runs commands and corrects itself based on feedback: goal, exploration, plan, change, test, fix, diff and pull request. Coding agents (Claude Code, OpenAI Codex, Devin) have made this an everyday way of working. Codex reports a weekly user base in the millions, and Anthropic says that more than 80% of the code merged into its codebase was authored by Claude as of May 2026.

In Addy Osmani's formulation, "agentic engineering" is distinguished from vibe coding precisely by the discipline it brings: specification, testing and review rather than blind acceptance. The difference lies not in the model but in the way of working. In practice, an agent is only as good as the frame built around it: without a clear task description, a fast test loop and narrow permissions, even the most capable model can do damage. The same tool can be a valuable pair or an unsupervised risk; the frame decides.

Sobriety matters here. Armin Ronacher aptly repeats a friend's description of Claude Code as "catnip for programmers" (attractive and addictive), but adds that tools need to be protected against an "LLM chaos monkey," that is, against the model using them completely wrong. An agent is not a single model: it needs a sandbox, narrow permissions and human approval points. Which raises the question: what can these systems actually do, beyond the demos?

Benchmarks measure performance on fixed tasks under controlled conditions, not operational performance in real environments. SWE-bench measures whether a model can carry a fix through on real GitHub issues. On the 500 human-validated tasks of the 2024 Verified subset, the frontier is now practically saturated, with several models clustered around 80%. That does not mean the task is "solved."

Another benchmark points the same way. Older benchmarks such as HumanEval saturated long ago, because they measure isolated functions rather than real repository work. Terminal-Bench, from the Laude Institute, tests actual terminal use instead (running commands, debugging, multi-step workflows), and here even the best results are more modest. The lesson is that a number is only worth as much as the task behind it: there is a long way between a narrow, well-defined example and an open-ended, real environment.

Contamination-resistant benchmarks give a sobering picture. On Scale's SWE-bench Pro leaderboard, results drop to roughly 45-59% by mid-2026 (not all models run in the same harness), and on private, non-public codebases the best models in the benchmark's original 2025 paper reached only about 15-18%; the figures come from different evaluation snapshots and are not directly comparable. It is no accident that OpenAI stopped reporting Verified results in February 2026, citing flawed tests and contamination: frontier models could reproduce some of the reference fixes. The lesson is twofold. First, the distance between the benchmark number and production performance is large. Second, it is increasingly not only the model but also the harness built around it (toolset, sandbox, context management) that determines the actual result. And a system that generates code quickly does not thereby produce better software.

The intuition that has taken the hardest knock is that AI makes us faster, and that this is good in itself. METR's controlled experiment puts both halves of it under a harsh light. Experienced open-source developers working on their own, familiar codebases in early 2025 did not speed up with AI tools; they slowed down by about 19%, while believing they had become faster. Perception and measured reality came apart. (A later follow-up with late-2025 tools points to a speedup among the original participants, but its confidence intervals include zero, and strong selection bias likely understates the speedup, according to METR. In other words, the picture changes over time and with the sample.)

METR's result is easy to misread, so precision matters. It does not say that AI is useless. It says that experienced developers on familiar, mature codebases behave differently from what uncontrolled self-reports suggest. The gap between perception and measured performance is a lesson in itself: if we do not measure, we tend to overestimate our own speedup, and that is the worst possible basis for a significant investment or process decision.

The quality side rests on different kinds of evidence, but it points in the same direction. According to Veracode's 2025 study of more than 100 models on 80 security-sensitive coding tasks, the generated code contained OWASP Top 10 vulnerabilities in about 45% of the test cases, and newer and larger models did not perform better. Apiiro's analysis pointed to a significant increase in privilege-escalation paths and design flaws, and GitClear's longitudinal data to a decline in refactoring, a multiplication of duplicated code and rising code churn. The logic is simple: AI lowers the cost of producing code, but increases the amount of code that has to be checked, and with it the review burden. DORA puts it well: AI is an "amplifier." In a well-run organization it speeds things up; in a chaotic one it magnifies the dysfunction. And if the review burden grows, what happens to the developer's role?

The emphasis is shifting from direct implementation toward specification, architecture, verification and responsibility, but differently for juniors and seniors. On the junior side, the numbers are worrying. According to a 2025 analysis by Stanford and ADP, employment of 22- to 25-year-olds in occupations highly exposed to AI has fallen by about 13% in relative terms since 2022, and internship postings on Handshake fell by more than 15% between January 2023 and January 2025.

Steve Yegge's counter-narrative reads the situation the other way around: AI gets beginners to meaningful work sooner, because it takes some of the most routine tasks off their hands. The two readings are not mutually exclusive. AI may raise the bar for entry while giving those who get in more tools to work with. That is exactly why causal conclusions should be drawn from labor-market numbers only with care.

Here, though, counter-evidence is mandatory. Using Danish data, Humlum and Vestergaard found no difference in junior hiring between firms that adopted AI and those that did not, and serious macro confounders distort the picture: the interest-rate environment, the correction of post-pandemic overhiring and the doubling of computer-science graduates in the US over a decade all play a part. The correlation is strong; the causation is disputed. On the senior side, the role is, if anything, gaining value: the "agent manager" owns system design, task decomposition, acceptance criteria, test strategy and review. The counterargument worth keeping, however, is that if AI also improves at architecture, debugging and design, then "higher-level thinking" is not necessarily a durable human refuge. The practical question, then, is not whether developers will disappear, but how these tools are best used today.

The most useful framework thinks in terms of task type and risk, not tools. For low-risk, well-scoped work (boilerplate, documentation, simple transformations), letting the agent run freely has high upside and low downside. At medium risk (features touching several files, migrations), it is worth giving narrower tasks and strong tests. In high-risk areas (authentication, financial logic, cryptography, production deploys), human approval is not optional.

The point of the framework is that the risk of the task decides, not the tool. The same terminal agent can be ideal for generating documentation and dangerous for rewriting a payment module. The difference lies not in the model but in how much we entrust to it without supervision. It is therefore worth sorting tasks by risk first, and only then choosing a mode and a tool. That way the speedup shows up where mistakes are cheap, and human control stays where they are expensive.

The three modes alternate even within a single working day. Faced with an unfamiliar bug, a developer first uses AI as an advisor to understand the cause. On a familiar feature, they work with it as a pair programmer, directing it line by line. On a well-scoped, mechanical task, they give it room as a supervised agent that carries the work through on its own. The point is not to settle once and for all which mode is right, but to choose deliberately, knowing the task.

There are three working modes, and they do not compete; they alternate by task: AI as advisor (chat), as pair programmer (copilot) and as supervised agent. DORA's "amplifier" finding applies here too: the benefit does not come by itself but depends on the maturity of the organization. Willison observes that a codebase can be set up to work well with coding agents, above all through automated tests the agent can run; a fast test loop, clear structure and context files (CLAUDE.md, AGENTS.md) genuinely help the agent. Two limits are worth respecting, though. Overly broad agent permissions are dangerous, and context engineering does not replace review; it only prepares for it. And if we can already entrust this much to an agent, what comes next?

The direction in 2026 points toward longer autonomous sessions, background and parallel agents, coordination among multiple agents, issue-to-PR pipelines and automated review, with the developer increasingly acting as an "agent manager." Products (parallel builds in Cursor 3, Antigravity 2.0, Devin Desktop, the Codex desktop app) are already being built around this way of working. Meanwhile, the capabilities of frontier models have converged, so competition is shifting more and more to the level of the harness and the scaffold. What the right form of this is remains contested: alongside Anthropic's multi-agent research system, Cognition argues explicitly that in most cases we should not build multi-agent systems.

The practical consequence of convergence is that, when choosing, "which model is best" matters less and less, and which tool fits the team's workflow and codebase matters more and more. The multi-agent debate has no settled answer either. Several parallel agents can increase throughput, but also the burden of coordination and review, so in many cases the simpler single-agent setup is the more robust one.

It is also worth seeing the limits realistically. In longer, multi-step sessions, agents still get lost, misread intent, or set off confidently in the wrong direction; unsupervised, end-to-end autonomous development is for now more promise than everyday reality. The real stake of the coming years is therefore not whether humans disappear from the process, but where the boundary of mandatory human review lies, and who answers for what the agent finally commits to the codebase.

Reliable long-horizon autonomy remains limited, and responsibility remains with the human. So the closing thought is not that programming is disappearing, but that it is being relocated. From the chat window to the pull request, the human role has moved from direct implementation toward defining the task well, understanding the system, verifying, and taking responsibility. Programming is not changing because technical knowledge is no longer needed, but because that knowledge now has to be applied somewhere else.