DSPy Flex: Let the AI write your agent's code

Direct answer: dspy.Flex is a new DSPy module that gives an optimizer like GEPA access to your program's code, not just its instructions. On an entity resolution benchmark, it goes from 90.4% accuracy to 95.0% while costing 28% less and running 40% faster than the starting program. Fewer LLM calls also means less data sent to third-party APIs, so a smaller GDPR surface for companies.

What exactly is DSPy Flex?

DSPy is built on one idea: define a task once, in a way that lets it be re-implemented as the AI ecosystem advances. The history of these re-implementations follows the history of the models. In 2022, models needed to see what a task looked like, so optimizers like BootstrapFewShot automated the picking of few-shot examples. Models then became good prompt writers, so MIPROv2 and GEPA improved programs by rewriting their instructions. Today, models have become excellent programmers. Flex taps this ability to rewrite the code itself, not just the prompt.

Concretely, dspy.Flex(YourSignature) drops in where dspy.Predict goes. Here is the transformation:

my_signature = "question -> answer"
my_program = dspy.Predict(my_signature)
 
my_program = dspy.Flex(my_signature)

Before optimization, Flex behaves exactly like a Predict module (or an RLM if you provide tools). The difference appears when you hand it to an optimizer: Flex exposes its code, in addition to its instructions. The reflection model can then decompose the program, write helper functions, implement routing logic, and rewrite the prompts.

How do you optimize a Flex program with GEPA?

The official example optimizes a signature with GEPA. A cheap model serves during inference, a big model writes the code:

program = dspy.Flex(SamePlace)        # was: dspy.Predict(SamePlace)
 
dspy.configure(lm=dspy.LM("anthropic/claude-haiku-4-5"))
 
big_lm = dspy.LM("anthropic/claude-opus-5")
 
optimized = dspy.GEPA(
    metric=make_metric(penalty=0.2),
    reflection_lm=big_lm,
    max_metric_calls=400,
).compile(program, trainset=train, valset=val)

After optimization, optimized.save("program.json") persists the source, and dspy.Flex(SamePlace).load(...) restores it. What you get back is a file you can open, read, diff, and reason about.

Two things tend to follow. Sometimes the optimized program makes no model call at all, because it found a case it could settle in code. And when it does call, the call is better aimed, because the module has already done the parsing and comparison. Fewer calls, better calls, and a program that outperforms the one you started with.

Is Flex safe for production?

Code written by a model is still untrusted code. By default, it never runs in your process. Flex executes the generated source inside a sandboxed interpreter. Only predictor calls and the tools you explicitly provided bridge back to the host process, and a max_predictor_calls cap bounds how many times per forward that bridge can be crossed.

This isolation has a double benefit for companies: it limits the risk of malicious or buggy code, and it keeps execution inside your environment. Combined with open source models hosted in Europe, it fits an AI integration consultant approach geared toward compliance.

The entity resolution benchmark

The featured use case is a geospatial conflation task: given two place listings, decide whether they are the same physical place. It is deceptively hard in the tail. KIN CAFE and KIN at the same address are the same place. CONCESSION #2 KEN MERCER SPORTS PARK and KEN MERCER SPORTS PARK at the same address are not.

The protocol: 1,029 labeled pairs, evaluated on 240 held-out records (class-balanced, so 50% is chance). Caches were disabled, so the figures reflect cold production traffic. The execution model is claude-haiku-4-5, the reflection model is claude-opus-5.

Two takeaways. Even with calls free (λ=0), the optimizer wrote code: it routed 75% of records through deterministic Python and reached 95.0%, more accurate than calling the model every time (90.4%, McNemar p=0.019). At λ=0.4, the program called the model once across 240 records, holding 92.1% accuracy statistically indistinguishable from the baseline, at roughly a hundredth of the cost and a thirtieth of the latency.

Why does prompt-only optimization cost more?

Prompt optimization has a single lever: the instruction. GEPA therefore wrote a much longer one, and every record pays for those extra tokens at inference. Result: 92.5% accuracy, but $2.88 per thousand records, 2.9x the baseline cost, and 48% more latency. Flex gives the optimizer a second lever: the module code. By optimizing the prompt and the code together, Flex produced a program that is 28% cheaper and 40% faster than the baseline, while being more accurate.

How does the metric drive the trade-off?

A GEPA metric returns a score plus natural-language feedback. With Flex, it can also see how many LLM calls the generated program made on each record. You can use that value to penalize the feedback:

score = max(0.0, correct - PENALTY * n_llm_calls)

At PENALTY = 0, calls are free and the optimizer chases accuracy alone. As the penalty λ rises, every LLM call has to buy back more accuracy than it costs, and the optimizer is pushed to settle cases in Python and reserve the model for genuine ambiguity. Past λ = 1.0, a call can never pay for itself, which amounts to never calling the model.

The penalty is also a compliance lever. Every LLM call is a potential data transfer to a third-party API. Penalizing calls mechanically reduces the data sent outside, which aligns with the minimization that GDPR requires. For a company with strict data-location constraints, this is a design parameter as much as a cost parameter.

What does the AI-written code actually do?

At λ=0.4, the program holds about two hundred lines of Python written by the reflection model. The architecture has three stages. Normalize: names are uppercased, stripped of franchise numbers, legal suffixes (LLC, INC) and a handful of generic words (CAFE, RESTAURANT, MARKET, GRILL), and addresses are parsed into a house number and a street core. Compare: the distinctive tokens are scored zero to one with fuzzy similarity and binned into three categories. Decide: each category gets its own rules combining name, address, and distance. Only the case where no rule fires goes to the model, carrying the already-computed analysis as extra input.

The comment at the top, written by the optimizer about its own architecture, sums it up: "the LLM is a LAST-RESORT fallback."

Can Flex go beyond a simple benchmark?

A pilot on SWE-bench Pro, a coding benchmark built from GitHub issues, gives a glimpse. With claude-haiku-4-5, the starting program resolves 0 of 12 sampled issues. After GEPA optimization on the Flex program, with max_metric_calls capped at 60, it resolves 4 of 12, after designing a workflow that mixes Python and LLM calls to research, draft, evaluate, repair, and submit an answer. It is a pilot, but it shows a harness evolving in a handful of turns to 4 of 12, against the 39% Haiku is reported to reach inside a mature, hand-built harness.

Four moves keep recurring when GEPA rewrites programs: decomposition (giving each step its own implementation), method selection (deterministic code or model call per step), routing (clear cases down the cheap path, ambiguous ones to the judge), and evolution (refining signatures, instructions, and code once the structure settles).

What this means for an enterprise project

A hand-written harness does not adapt to new models, datasets, or tactics unless you go back and rewrite it. With Flex, you compile the harness continuously. Concretely, for an AI automation project, you can sweep models and tune your metric to find the optimal cost, latency, and accuracy balance, without rewriting the logic on every model change.

The same instinct runs through coding agents, which increasingly write a Python script to do a job rather than doing it token by token. Flex industrializes that reflex by handing it to an optimizer.

TL;DR

DSPy Flex lets the model write your program's code, not just its instructions. On entity resolution, it reaches 95.0% accuracy while costing 28% less and running 40% faster than the baseline, with 75% fewer LLM calls. The generated code runs in a sandboxed interpreter, and a penalized metric lets you fine-tune the cost, latency, and accuracy trade-off. Fewer calls to third-party APIs also mean less data leaving your environment, a useful point for GDPR and sovereignty constraints.

To fine-tune the real cost of your agent deployments, the RAG cost calculator estimates your token and embedding budget. And if you are weighing architectures, the AI agents vs LLM pipelines comparison can guide you.

Building AI agents and want to cut costs without sacrificing quality, or keep your data in Europe? Let's talk.