KENNY CHIEN / IDEAS / ORCHESTRATION OVER PROMPTING
Prompting is not programming. Orchestration is.
The unit of engineering is shifting from the function to the agent loop.
A prompt is a request. A loop is a system. Engineers are paid to build systems.
In one paragraph
Orchestration over prompting means the engineering value in AI systems lives not in the prompt but in the loop around it: how an agent plans, calls tools, checks its own work, recovers from failure, and reports what it did. Prompting is an input skill, like writing a clear ticket. Orchestration is a systems skill. The unit of software engineering is shifting from the function — deterministic, composable, testable in isolation — to the agent loop — probabilistic, tool-using, evaluated statistically. Teams that treat the prompt as the product plateau at impressive demos. Teams that engineer the loop ship applications.
The argument
The prompt plateau
Every team's first month with AI is prompt worship. A better prompt produces a visibly better demo, so tuning prompts feels like engineering progress. It is not. A prompt has no error handling, no state, no memory of past failures, no way to escalate. You are tuning a single call to a stochastic function and mistaking the tuning for architecture.
The ceiling arrives fast. Past a modest threshold, clever wording buys marginal gains, and the next model release moves the ceiling anyway. I have watched teams polish prompts for months that a plain verify-and-retry loop beat in a week.
The new unit: the agent loop
For fifty years the unit of engineering was the function: deterministic input to deterministic output, composable, unit-testable. The agent loop keeps the rigor and changes the shape. A loop plans, acts through tools, observes results, decides whether to retry, escalate, or proceed, and persists state across steps — sometimes across days.
Every consequential design decision now lives in the loop, not the prompt. Which tools does the agent get, and under which permissions? Where are the human checkpoints? What is the retry policy, and what triggers escalation? What state survives a restart? What gets logged so the trajectory can be read later? These are architecture questions. They have wrong answers, and the wrong answers reach production.
Old disciplines, new object
The disciplines that made software reliable do not disappear. They re-aim. Testing becomes evals: golden tasks, failure taxonomies, statistical acceptance thresholds instead of assertions. Error handling becomes recovery policy: not catching exceptions, but deciding what an agent does with a wrong answer it believes. Code review becomes trajectory review: reading the transcript of what the agent did and why. Observability becomes the craft of making agent decisions legible after the fact.
This is good news for engineers. The profession transfers almost one to one; only the object changed. The people at risk are not programmers. They are prompt collectors.
What it means for your team
Hire and train for systems thinking. Prompt tricks depreciate with every model release; loop design does not.
Make evals first-class artifacts — written before the application, rerun on every model upgrade.
Put loop design in design review: tools, permissions, checkpoints, and escalation belong on the whiteboard, not in the prompt file.
Treat prompts as configuration: versioned, cheap to change, and expected to churn.
The steelman
The strongest counterargument: models absorb the loop. Each generation plans better, self-corrects better, and needs less scaffolding — so orchestration is just prompt engineering with extra steps, obsolete on the same schedule. Half of this is true. Models do absorb the generic loop: decomposition, retry, self-critique. Build elaborate scaffolding for those and you are writing code with an expiration date.
What models cannot absorb is your loop: your tools, your permission boundaries, your human checkpoints, your definition of done. That context lives outside every foundation model by construction. The loop gets thinner every year; it does not disappear. And the team that owns the loop owns the evals — which is where an enterprise's trust in the system actually lives.
Where to go next
Building your first loop?
Tell me what the agent is supposed to do and where it keeps going wrong. I will reply with an honest read on whether it is a prompt problem or a loop problem.