The Diagram I Could Not Unsee
I was reading Sydney Runkle's article, The Art of Loop Engineering, when its four nested loops started looking familiar. An agent does the work. A grader checks it. Events trigger more runs. Production traces feed improvements back into the system.
I had seen that shape before. It looked like model training.
We spent years teaching models through examples, evaluations, and feedback. Now we apply a similar pattern after deployment, often improving the prompts, tools, memory, and graders around a model instead of updating its weights.
We trained the model. Now we train the loop.
The story is not simply that models became smarter. The unit of engineering kept moving outward.
The Unit of Engineering Kept Moving Outward
The first mainstream LLM experience was a chatbox. A person typed a question, the model returned text, and the interaction ended.
Function calling changed the boundary. A model could produce structured arguments for code to execute. Tool calling expanded that idea into search, files, databases, terminals, and external APIs. The model no longer had to describe an action when software could perform it.
Then integration became its own problem. Every tool needed schemas, permissions, error handling, and model-facing instructions. Protocols such as the Model Context Protocol made those connections reusable, but access to tools still did not guarantee good work.
That is where harness engineering became important. The harness supplies instructions, context, tools, memory, permissions, and stopping rules. Two applications can use the same model and behave differently because one gives it a better operating environment.
Loop engineering moves one layer farther out. Instead of asking only, "How should this agent work?" it asks:
What should observe, verify, trigger, and improve the agent itself?
This is not a timeline where each idea replaces the previous one. These layers coexist. What changed is where engineers found the next bottleneck.
Training and Production Started Rhyming
The analogy is not perfect, but the structures rhyme:
Model training | Loop engineering |
|---|---|
Training examples | Production tasks and traces |
Human labels or reward models | Tests, rubrics, LLM judges, and human review |
Evaluation feedback | Grader and operational feedback |
Update model weights | Update prompts, tools, memory, graders, and configuration |
Traditional training improves behavior by changing parameters inside the model. A production loop can leave those parameters untouched and change the model's environment instead.
Calling every improvement "learning" can hide where the capability lives. If an agent becomes more reliable because an engineer added a better tool schema and a deterministic test, the base model did not become smarter. The system became better designed.
For teams running open-weight models, production traces may become fine-tuning or reinforcement-learning data. But the loop can also improve retrieved context, reusable skills, permissions, or the grader deciding whether work is acceptable.
Platform Engineering for Probabilistic Workers
As a software engineer, I find loop engineering less mysterious when I compare it with systems we already build.
The agent loop is a worker executing a task. The verification loop resembles CI: run checks, report failures, and block bad output. The event-driven loop resembles the webhooks, queues, and schedules that wake ordinary services. Traces extend observability with model decisions and tool calls.
The hill-climbing loop is the unusual part. It uses traces to propose harness changes. That does not need to mean letting an agent rewrite itself and deploy at 3 AM. A safer version looks familiar: detect a repeated failure, open a pull request, run evaluations, and require human approval.
Seen this way, agents did not make established engineering disciplines obsolete. They imported those disciplines into a world where the worker is probabilistic.
I Had Already Been Moving Through These Layers
At the 14th Bicol Youth Congress in Information Technology, I gave a talk called The Agentic Revolution: From Chatbots to AI Agents. I explained the jump from models that answer questions to systems that can call functions, use tools, and keep working toward a goal.
That framing was useful, but it stopped at the agent.
Later, I built Wan Shi Tong, a workflow that analyzes GitLab merge request diffs, finds cross-project breaking-change risks, reports them inside the MR, and can generate fix MRs. I also began contributing to Nous Research's Hermes Agent, where I found a growing interest in harness engineering.
Loop engineering showed me the next layer. The agent and its harness are inner machinery. Reliability also comes from the trigger, evaluation, trace, and process that turns repeated failures into controlled improvements.
A Loop Can Improve the Wrong Thing
There is a dangerous interpretation of this idea: connect enough feedback loops and the system will inevitably improve itself. That depends entirely on what the loop rewards.
A grader can verify that links resolve, tests pass, or JSON matches a schema. It is harder to encode whether a change belongs in the architecture, suits its audience, or should be automated at all. A weak rubric does not remove judgment. It buries judgment inside a metric.
Humans therefore belong at more than the final approval step. We decide what success means, which evidence is trustworthy, and which actions require consent. Human judgment belongs where automation cannot safely substitute for it.
So, Did the AI Learn?
The progression from chatboxes to loop engineering is also a story of engineers building larger systems: first to act, then to act reliably, run continuously, and improve from experience.
The model may be the most visible component, but it is no longer the whole product.
If its weights never change, but its tools, memory, evaluations, and behavior keep improving, did the AI learn?
Or did we move learning outside the model?
