How it works

One high-level spec goes in: a function, its constraints and what to optimise. The agent's job is to take that spec down a ladder of ever more faithful, ever more expensive models of the hardware, keeping only the candidates worth the next level's cost, and checking each level against the golden model before trusting it.

The ladder

Each level offers its own power, performance and area trade-offs, and each costs more per design than the one above it. Exploring cheaply at the top is what makes it affordable to be thorough at the bottom.

At the vendored commit all four milestones are done and every level of the ladder is live: spec intake (L0), analytical exploration (L1), system-level simulation in SimPy and the cycle-accurate model (together the repository's L2, milestone 3), RTL simulation (L3), synthesis (L4), gate-level simulation of the netlist, synthesis on a second target, an ASIC on the open sky130 PDK (milestone 4), and back-annotation (L5). The ASIC rung is post-synthesis only: there is no place and route, and its numbers are an open-PDK estimate, not silicon. Exploration itself still happens at L1, where a design costs microseconds to score; the lower levels check, measure and correct what L1 chose.

The design principle: the LLM never produces a number

The LLM chooses which architecture families to explore, over which parameter ranges, and what to do after each round. Code computes every figure. That rule is in the system prompt every recorded call was sent:

Division of labour (strict): - YOU choose architecture families and parameter ranges from the registry below, read compact summaries of results, and decide what to do next. - CODE does all the numbers: a bit-accurate model gives exact accuracy, an analytical cost model gives LUT/FF/Fmax estimates, Optuna NSGA-II searches inside the ranges you choose, and Pareto/hypervolume maths ranks the results. - Never estimate, predict or invent LUTs, FFs, Fmax, throughput, power or error figures. Reason only from the numbers in the summaries you are given.

It is also in the types. Each LLM call names a Pydantic schema (ExplorationPlan, AnalysisDecision), and those schemas have no field for a LUT count, an Fmax or an error figure: the model can name families, ranges, a budget split, a decision and its reasons, and nothing else. Every number is labelled with where it came from:

exact

Computed by the bit-accurate golden model (accuracy, error) or read off the schedule (latency in cycles). The golden model is bit-exact against the reference RTL.

estimate

From the analytical Artix-7 cost model calibrated to two Vivado results (LUTs, flip-flops, Fmax, throughput, power index), or a refit of it that names its calibration. The eval scores every milestone on the same model, so milestones stay comparable.

measured

Recorded from real runs and real tools: the provider-reported tokens and cost of each LLM call, the wall-clock the eval harness timed, and (since M2) LUTs, flip-flops and Fmax from Yosys + nextpnr and Vivado, named with the tool and its version.

simulated

From the L2 SimPy model of the design's system (M3): throughput under back-pressure, p50/p99 and batch latency, queue depth, clocked at the design's estimated Fmax. During exploration the same metrics are screened by an optimistic analytic bound, an estimate.

The graph, replayed

The agent is a LangGraph StateGraph. intake turns the spec into a validated Spec; confirm_spec stops for a human with interrupt(); propose asks the LLM for a plan; explore_family runs once per family, fanned out in parallel with Send, each branch an Optuna NSGA-II study inside the box the LLM chose; analyse merges the results, computes the Pareto front and its hypervolume, shows the LLM a compact summary and asks what next. Code then applies hard rules the LLM cannot override: the round cap, the budget, and no "infeasible" verdict while feasible designs exist. After select, milestone 2's graph adds back_annotate: the selected design's estimates against any measured synthesis data, and a flag if the winner would change. Milestone 3 adds two more nodes, both code only: l2_simulate before it, which simulates every L1-feasible design of the run inside the spec's system and re-selects on the simulated numbers, and l5_reexplore after it, which re-explores around the measured designs under the refit cost model when a measurement flags a winner change. Milestone 4 adds a third, l2_feedback, also code only: when L2 rejects every L1-feasible design, it corrects the L1 bound from the simulations and re-explores (below). It also lets a spec name its target: the same graph explores an ASIC spec with an ASIC cost model and an ASIC system prompt, and the FPGA prompt stays byte for byte what it was.

Milestone 2 also changed how the agent spends its budget, to map the whole trade-off curve and not only the corner the spec's selection rule cares about. The LLM gained a map_front decision, which asks code for a front-mapping round: NSGA-II over the ranges of the families on the front so far. And 40% of the budget is held in reserve: before a run stops, code spends whatever is left on one final front-mapping round, with no LLM call.

The replay below walks one recorded run through that graph. Pick any of the 60 M2 runs or the 48 M1 runs. The quoted decisions are the LLM's own words from the trace, and the counters add up the usage OpenRouter reported for each call measured. Failed calls are shown too: they return no valid answer, and most report no cost.

Loading the animation…

Three things to look for. On the infeasible spec, the LLM itself returns infeasible and names throughput as the binding constraint; code only checks that no feasible design exists before accepting it. In M1 the last word was often code's: the LLM asked to refine, the hypervolume gained less than 1% that round, and that rule stopped the run early, which is why the M1 agents mapped less of the front than NSGA-II. In M2, watch for map_front and for the final round planned by code: that is how the agents closed the gap, as the results show.

L2: the design inside its system

A throughput figure says how fast a design is on its own. A system asks something harder: 32 requests arriving at once on every control tick, bursts of 8 from a sensor front end, a DDS that stalls when its FIFO fills. Milestone 3 adds that rung. Every family obeys one interface contract (ready, valid, a latency and an initiation interval), which the generated RTL implements and the formal checks pin down; a cycle-accurate model implements the same contract, and SimPy models put it inside three systems: a DDS feeding a mixer, a control loop and a bursty request stream. Each design runs on its own clock at its estimated Fmax estimate, so every system number is simulated.

During exploration, L1 screens a system constraint with an analytic bound that is optimistic by construction (each burst or tick served alone, starting on a clock edge), so L1 never discards a design L2 would accept. After select, L2 simulates every L1-feasible design the run evaluated and re-selects. That is cheap: the 635,040 designs share only 1,092 distinct combinations of latency, initiation interval and Fmax, and the simulation is cached on them, which is also how the repository computes an exhaustive simulated ground truth for each system spec.

Here is the spec where it matters. multiaxis_control asks for all 32 results of every 1 µs tick within 0.44 µs. Read in milestone-2 terms (throughput at the average rate, 32 MSPS) the best design is pipelined_m m=7 (953 LUT+FF); the L1 bound picks m=5 (1,013); the simulation picks m=4 (1,072). The replay below runs the repository's SimPy model for all three.

Loading the animation…

m=4 and m=5 have the same numerics, and m=5 is smaller, so m=4 is dominated in the L1 objectives: L2 has to look at every L1-feasible design, not only the L1 front. The first version of the l2_simulate node shortlisted from the front and could never reach m=4; the planner's review found it before the merge. Even sizing for the peak rate (32 requests in 0.44 µs, 72.7 MSPS) stops at m=5, so only the simulation gets to m=4, by a margin of 1.5%. The model aligns each request to the next CORDIC clock edge but has no synchroniser: a real clock-domain crossing would add about two destination cycles, which against that margin could move the winner again. The repository lists it as a limitation.

The cycle model, checked against the RTL

A system model is only as good as its contract, so the cycle model is checked against the generated RTL itself, cycle for cycle. The repository's L3 harness gained a bursty mode (idle gaps before each angle) and a per-edge log; 48 traces over 8 designs, covering every family and both rounding modes, compare ready, valid_out and the output codes on every edge in Icarus Verilog and Verilator: 48 of 48 identical, 28,424 edges. The waveform below is the cycle model driven by the stimulus of one of those traces; this site checks that it takes exactly as many edges, accepts and results as the RTL log recorded.

Loading the animation…

A second target: an ASIC on sky130

Milestone 4 takes the same generated RTL to standard cells. The flow (yosys 0.33 + OpenSTA 3.1.0) runs it against the open sky130_fd_sc_hd library at its typical corner (25 °C, 1.80 V) with a 10 ns clock: cell area, flip-flops, the register-to-register path with the library's wire-load model, and vectorless power measured. There is no placement and no routing: these are post-synthesis numbers, the counterpart of the Vivado anchors, not of a routed chip. OpenROAD place and route was left out (no package for this Ubuntu, a commercial install channel, and a floorplan and power grid per design); it is on the roadmap, from its own pinned release binaries.

57 synthesised designs across all four families calibrate an ASIC cost model, the FPGA model's structural counts re-weighted for standard cells. Leave-one-out it is off by 8.7% on area, 4.6% on flip-flops and 10.6% on Fmax estimate. The fitted adder delay is about 0.29 ns per bit: Yosys's default adder mapping ripples, so every rotation stage costs more clock on sky130 than on the FPGA's carry chains.

Three ASIC specs each get an exhaustive ground truth over the same 635,040 designs, and an FPGA twin: the same constraints with area in LUTs + FFs, scored on the FPGA ground truth. Does the target change the answer? Not the family ranking: it is the same on all three, though on two of them only the two pipelined families can meet the rate, so that is one pairwise comparison each. What changes is the depth inside a family:

Loading the animation…

The held-out check

9 more designs were synthesised and never fitted: each spec's per-family optima, the FPGA twins' winners and the L1 bound's pick. Area holds (within 5% on 9 of 9). Fmax does not: from -14.9% to +39.6%, and the model is optimistic on 5 of the nine.

Loading the animation…

The deadline miss

That optimism matters where a spec's margin is thinner than the model's error. asic_multiaxis_control is milestone 3's control loop on sky130, with a 0.5 µs deadline. Its ground-truth winner, pipelined_m m=2, meets it at the estimated 82.5 MHz; synthesised, it runs at 72.2 MHz, and at that clock it misses the deadline (p99 0.567 µs). The same SimPy replay as milestone 3's, at both clocks:

Loading the animation…

The repository states the miss plainly and does not refit the model on the held-out points, which would end their role as a test. The ASIC back-annotation flags this case, but there is no ASIC re-exploration loop yet; a measure-and-re-explore loop is on the roadmap.

L2 → L1 feedback

L1's system bound is optimistic by construction. If L2 rejects every L1-feasible design of a run, the run spent its budget where the bound was wrong. The l2_feedback node then does for the bound what L5 does for the cost model: it measures the bound's error on every simulated design, fits a monotone correction in the design's utilisation per contract class (never more optimistic than any simulation it has seen, so labelled an empirical estimate), and re-explores with a separate budget of 3 × 60 evaluations under the corrected bound.

On the committed specs it never fires: 0 of 92 recorded searches and 0 of 60 live runs found an L1-feasible set that L2 rejected whole. So the repository constructed a spec where it does (bursts of 32 requests at 130 MS/s, a 130 MS/s throughput floor, p99 within 0.4 µs). It fired in 3 of 15 searches, all random search, and recovered a passing design in 3 of 3; the same extra budget with the uncorrected bound recovered 1. Evidence for the mechanism, on three cases; the results have every search.

Verification at every level

A level is only useful if its numbers can be trusted, so every level is checked against the one golden model before the agent relies on it.

The golden model is the anchor: it produces exactly the reference RTL's output codes on all 65,536 input angles (Icarus Verilog, in the repository's CI). Milestone 2's generator turns any explored design into SystemVerilog, and every generated design is checked against that model, output code by output code, in two simulators, then again as a synthesised netlist. Here is one design, the exhaustive grid's winner for low_area_control, taken all the way down:

Loading the animation…

Across the whole generator, the repository's sweep covers every family:

Every eval run is also graded against the exhaustive ground truth of all 635,040 designs, and this site adds its own check: its TypeScript ports of the golden model and the cost model match the Python exactly, in its own CI.

An optional outer loop: the campaign agent (experimental)

Milestone 3 also tried a different kind of agent on top. A campaign agent, built on LangChain's Deep Agents harness (planning to-dos, a virtual file system, one sub-agent for the ladder, cross-run memory in a LangGraph Store), is meant for long-horizon work: take specs down the whole ladder, re-plan when a measurement changes the winner, carry lessons from one spec to the next. The structured graph stays the inner loop and is one of its tools (run_dse); the others are explore_family, simulate_system, verify_rtl, synthesize, back_annotate, reexplore, finalize, recall and remember. Code enforces the same 400 L1 evaluations per spec, refuses to finalise a system spec on L1 bounds or a flagged winner change that has not been re-explored, and stops a campaign after 40 model calls or four identical tool calls in a row.

It is experimental: optional, off by default, and used by nothing in the structured graph or the command line. The repository measured it in an A/B against the structured graph and found no gain at a higher cost, which the results show in full.

Many agents at once

There are three kinds of parallelism. Inside a run, Send fans explore_family out across the families the LLM chose, so the studies run side by side and a reducer merges their evaluations. Across runs, every run is an independent graph with its own SQLite checkpoint, so runs can be queued and run side by side, and a run interrupted for human input, or killed, resumes from its last completed step. And since milestone 4, several graphs can work on one spec as a fleet live (M4).

The fleet: orchestrated by code

Milestone 3's lesson was that an LLM in the outer loop adds cost and loops without adding search. So the fleet puts code between the graphs. Two members, one per contract class (the FSM families and the pipelined families), each an unchanged structured graph restricted to its families, with its own LLM architect. Phase 1 splits half the budget equally; every evaluation goes to a shared LangGraph Store; code computes per-family lessons from the Store (counts and best values, nothing an LLM wrote), gives the other half to the members in proportion to their share of the merged Pareto front, and seeds each phase-2 study with the Store's front designs of its family. Code merges, selects and runs L2 once.

Why no LLM coordinator? Its decisions here are a budget split and a front merge: arithmetic over numbers code already has. Milestone 3's A/B showed what an LLM in that seat costs (1.44–2.25× as much per spec, for no gain), and its free-text notes into the inner architect narrowed the first box in its worst run. In the fleet there is no channel for free text at all. Code also enforces the equal budget: the fleet never spends more than the 400 L1 evaluations a single graph gets, so the two can be compared.

The replay below is one real fleet run, after the review fix that added the warm start; then the A/B against the single graph, pair by pair.

Loading the animation…

The fleet did not map the front better, and the warm start did not change that; with Qwen it picked better designs. The results have the whole A/B.

The eval runs as a fleet

Each eval was itself a small fleet of independent runs:

Sonnet 5.5 (20 runs, 9.23 min of wall-clock)

DeepSeek V4.1 Flash (20 runs, 29.7 min of wall-clock)

Qwen3.8-27B, reasoning off (20 runs, 15.8 min of wall-clock)

0 min14.8 min29.7 min

Sonnet 5.5 (12 runs, 5.54 min of wall-clock)

DeepSeek V4.1 Flash (12 runs, 20.4 min of wall-clock)

Qwen3.8-27B (12 runs, 144 min of wall-clock)

Qwen3.8-27B, reasoning off (12 runs, 9.63 min of wall-clock)

0 min52.3 min105 min

Those timelines are measured, not projections: M2 ran 60 runs with 232 LLM calls, up to 4 at once; M1 ran 48 runs with 171 calls, up to 6 at once. Milestone 4 ran its A/B one run at a time, so its wall-clock is each run's own: a single Qwen graph evaluated 25,212 designs per hour, the fixed fleet 16,094 (measured). What a fleet running 24/7 would cost is on the results page, worked out from the measured runs.