# The agent loop is an interpreter. We compiled it. > 346 agent programs compiled, 1,729 production runs measured: 40% fewer model calls per run, tool-call errors 2.2x lower, one program on its 393rd run from a single compile. The method, the cost model, the formulas, and the determinism number we would rather not print. Page: https://flymy.ai/blog/compiling-ai-agents/ [Blog](https://flymy.ai/blog/) */* [Research](https://flymy.ai/Research/) */* Compiling AI agents research · agent compiler # The agent loop is an interpreter. We compiled it. 346 agent programs, 1,729 production runs. Model calls per run down 40%, tool-call errors 2.2x lower, one program on its 393rd run from a single compile. And a determinism number we would rather not have to print. by **Denis Timonin** and the FlyMy.AI team September 2026 10 min read production data, apr - sep 2026 results.txt tl;dr ``` compiled_programs 346 tier 1: 303 · tier 2: 18 production_runs 1,729 max reuse: 393 runs model_calls / run 5 → 3 -40% paired, n = 78 tool_calls / run 4 → 2 -50% tool_call_error_rate 5.7% → 2.6% 2.2x lower completion_rate 94.9% → 95.4% break_even run 2 path_determinism 34% run-weighted · open ``` Ask an agent to pull yesterday's failed payments, cross-reference them against your CRM and email the account owners. It thinks, picks a tool, reads the result, thinks again, picks the next tool. Eleven steps later you have your emails. Run it again tomorrow and it does the same eleven steps, rediscovered from scratch, at full price. Not quite the same eleven, either: nothing guarantees it takes the same route twice. Most of what an agent does is deterministic. Today you pay a model to rediscover it on every run. ## 01 An agent loop is an interpreter There is a precise name for what an agent loop does. It reads a program - the task plus the tools available - and executes it one instruction at a time, deciding at each step what to do next. That is an interpreter in the exact sense a computer scientist means it. Write the loop as a function and the rest of this post follows: `int(p, x) = y` the loop: program p (task + tools), input x, result y `mix(int, p) = p*` Futamura I (1971): specialize the interpreter to one program `p*(x) = int(p, x) for all x` same answers `cost(p*(x)) << cost(int(p, x))` no planning left to pay for `mix` is a partial evaluator. Feed it an interpreter and a fixed program and what falls out is a **compiled program**: the first Futamura projection. Freezing an agent against one task is exactly this operation. The planning collapses into a fixed artifact, and the interpreter's job shrinks to running it. The *judgment* the task genuinely needs - reading a document, choosing between two candidates - stays as residual model calls. Everything else is compiled away. So we built a compiler for agents. It has been in production since April 2026. This is what it measured. ## 02 Two tiers, like a tiered JIT You cannot know which paths are worth compiling until you have watched them run, which is the same shape of problem a tiered JIT solves. So the compiler has the same shape. tier 0 · interpret The ordinary agent loop The model decides every step. Every agent on the market lives here, and every agent still starts here. **5** model calls, **4** tool calls per run (medians) tier 1 · freeze A typed program One successful run distilled into goal, ordered tool calls with templated arguments, declared input and output schema. Calls with no path to the output are dropped by a def-use rule. The model executes it; it no longer plans it. **3** model calls, **2** tool calls tier 2 · compile Executable code The same trace emitted as a script that calls the tools directly. No model in the loop, no tokens, deterministic by construction. **0** model calls Of the 346 programs compiled so far, 303 are tier 1 and 18 are tier 2. Tier 1 carries production traffic today, so tier 1 is what the numbers below describe. ## 03 Method and results The honest way to measure a compiler is to compare a program against *itself* before compilation - not against some other agent, and not against an average. For every compiled program we took the source chat session it was distilled from and compared it to the median of that program's own frozen runs. 78 programs have enough of both to qualify. Same agent, same task, same tools, before and after. Model calls are counted from context-tracking steps, tool calls and tool errors from the execution step log, on completed production runs between April and September 2026. **Figure 1.** Same program before and after compilation, medians over 78 paired programs. Each pair is drawn against its own baseline, so the bars compare shape, not units. | Metric | Tier 0 (chat) | Tier 1 (compiled) | Change | |---|---|---|---| | Model calls per run (median) | 5 | 3 | -40% | | Tool calls per run (median) | 4 | 2 | -50% | | Tool-call error rate | 5.7% | 2.6% | 2.2x lower | | Run completion rate | 94.9% | 95.4% | +0.5 pt | | Time to result (median) | 125 s | 48 s | 2.6x faster | | Runs per compile (mean / max) | 1 | 5.0 / 393 | - | | Path determinism (run-weighted) | - | 34% | open | **Two caveats we are not going to bury.** First, "time to result" compares an attended chat session, which includes a human reading and typing, against an unattended run nobody is watching. The purely machine-side part of that win is the 40% drop in model calls. Second, compiled paths fail less partly because we only compile paths that already worked. Written out, the observed gain has two terms: `Δobserved = Δcompilation + Δselection` and we cannot yet separate the two terms That is selection as much as it is compilation, and it is worth saying out loud. Separating the two terms needs compiled programs whose source runs were *not* successful, which is a study we have not run. ## 04 The economics, in closed form Compilation is not free: it costs one model pass over a trace we already have. Because that trace is longer than a normal step we count it as two calls' worth. With the measured medians the cumulative cost of running one task *n* times is linear in every tier, and the question of when compilation pays back has a one-line answer: `C0(n) = m0 · n` interpreted: m 0 = 5 model calls per run `C1(n) = c + m1 · n` tier 1: c = 2 for the one compile pass, m 1 = 3 `C2(n) = c` tier 2: m 2 = 0, the compile pass is the whole cost `n* = c / (m0 - m1) = 2 / 2 = 1` tier 1 is cheaper than interpreting from run 2 onward `lim 1 - C1/C0 = 1 - m1/m0 = 40%` the saving converges to the drop in per-run model calls **Figure 2.** Cumulative model calls for *n* repeated runs of one task. tier 0 grows at 5 per run, tier 1 at 3 after a 2-call compile, tier 2 is flat. The curves cross between run 1 and run 2. | Runs of the same task | Tier 0 | Tier 1 | Tier 1 saving | Tier 2 | Tier 2 saving | |---|---|---|---|---|---| | 1 | 5 | 5 | 0% | 2 | 60% | | 2 | 10 | 8 | 20% | 2 | 80% | | 10 | 50 | 32 | 36% | 2 | 96% | | 100 | 500 | 302 | 40% | 2 | 99.6% | | 393 our most-reused program | 1,965 | 1,181 | 40% | 2 | 99.9% | **Tier 1 breaks even on the second run.** That is the number worth holding on to, and it is a direct consequence of where the compiler gets its input: an execution trace the platform already recorded, rather than a workflow synthesized from a specification. Published systems that generate a workflow from scratch report break-even around seventeen transactions. We start from evidence, so we start from one. Tier 2 is flat because a compiled script calls no model at all. That row is a projection from the mechanism, not a measurement: tier 2 has 18 compiled programs and six executions so far, three of them clean. We are not going to dress six runs up as a benchmark. ## 05 Reliability is the real product The interesting result is not that compiled agents are cheaper. It is that they are *steadier*, and steadiness is what actually blocks agents from getting deployed. On [τ-bench](https://arxiv.org/abs/2406.12045), the benchmark that introduced the pass^k metric, the best agents clear about 60% of tasks on one attempt and fall below 25% when you require all eight attempts to succeed. Nobody puts a business process into production on those odds. A process that works differently every Tuesday is not a process. Compilation attacks the mechanism directly. Treat each tool call as an independent trial with error rate *e*; a run of *t* calls then survives with probability: `S(t) = (1 - e)t` probability that a run of t tool calls sees no tool error `S0(4) = 0.9434 = 0.79` interpreted: 4 calls at e 0 = 5.7% `S1(2) = 0.9742 = 0.95` compiled: 2 calls at e 1 = 2.6% `pass^k = pk` the τ-bench statistic: 0.60 8 = 0.017 for an agent that passes 60% of single attempts **Figure 3.** Independent-error model of run survival. e = 5.7% (interactive), e = 2.6% (compiled). The compiled program wins twice: a lower error rate per call, and fewer calls to expose to it. In production the picture matches the model: a compiled program fails **2.6%** of its tool calls against **5.7%** in an interactive session, and it does that while the runs worth compiling are the substantial ones. Completion holds at 95.4% with nobody watching the chat to nudge it back on track. The mechanism is unglamorous and that is the point: a compiled program can only use argument shapes that already returned a valid result once. The model never gets another chance to guess a field name. ## 06 The number that is not flattering Here is the result we could have left out. We measured **path determinism**: across repeated runs of one compiled program, how often does it take the exact same sequence of tool calls? Define it per program as the share of runs on that program's most common path, and weight by runs to get a fleet-level figure: `D(p) = rmode(p) / r(p)` share of program p's runs on its modal tool path `D̄runs = Σ r(p) · D(p) / Σ r(p) = 0.34` run-weighted over 26 programs with 5+ runs, 1,534 runs in total `medianp D(p) = 0.43` and 23% of programs sit at D = 1 **Figure 4.** Only the measured figures are plotted. The worst program took 170 distinct paths across 385 runs; the best took exactly one across 168. Both are called "frozen". Both of those are called "frozen". That spread is the entire argument for tier 2. The reason is structural, not a bug. A tier-1 program is still executed by a language model, and a language model reading an instruction can improvise: skip a step it judges unnecessary, retry one it judges failed, reorder two it judges independent. Sometimes that improvisation is why the run succeeds. Often it is just variance. Tier 2 removes it by construction, because a Python script cannot take 170 paths. That is why the roadmap is what it is: - **Guards and deoptimization.** Preconditions and postconditions on every compiled tool call. When a provider renames a field, the guard trips and the run deoptimizes back into the interpreter mid-flight instead of failing, exactly how a JIT handles a failed speculation. - **Compile-then-verify.** A compiled script becomes the live version only after it has replayed its own source trace successfully. - **Binding classification.** Every argument tagged at compile time as a constant, a user input, a copied output, a transform, or a residual model decision, so we can predict a program's determinism before its first run instead of measuring it after 385. ## 07 Where this sits in the literature This idea arrived from several directions at once during 2026, which is usually a sign it is the right one. [TraceCompiler](https://arxiv.org/abs/2608.02680) mines noisy agent traces into mostly deterministic workflows and classifies every argument binding with auditable evidence. [Compiled AI](https://arxiv.org/abs/2604.05150) compiles once and then executes with zero runtime tokens. [Agentic Plan Caching](https://arxiv.org/abs/2506.14852) (NeurIPS 2025) keeps the model but caches the plan, for roughly half the cost. [Agent JIT Compilation](https://arxiv.org/abs/2605.21470) from Stanford borrows the same vocabulary we do and adds the invariant-enforcing tool protocol that makes guards safe, and [LLMCompiler](https://arxiv.org/abs/2312.04511) (ICML 2024) shows what you get once the program is a DAG you can parallelize. What none of them has is a fleet. A compiler needs execution traces across many tools, many users and many failures, and that is the part you cannot download. We have 1500+ connectors, 500+ models and five months of production traces, and every run adds to them. ## Compile your own agent Build it in chat until it works once, then freeze it. The frozen program gets a schema, an API endpoint and a schedule, and it stops paying a model to rediscover what it already knows. [Try FlyMy.AI](https://app.flymy.ai/) [The research stack](https://flymy.ai/Research/#layer-l5) --- For agents: the FlyMyAI operating guide (sign up, connect, run models, tools and agents) is https://flymy.ai/skill.md, and https://flymy.ai/llms.txt lists the other pages and the documentation.