Multi-agent RL for the placer
No published work shows multi-agent reinforcement learning beating a tuned simulated annealer on routability judged by a real router, on designs it was not trained on — and this corpus's own record says the annealer is not what limits the pass rate. This is a literature review of 24 Sep 2026, parked. No experiment has been run and nothing was built; every number from a paper below is that paper's own, and every number of ours is from CHANGELOG.md Unreleased with its date.
The question
Several learned agents — one a part, or one a cluster (a regulator and its caps, an MCU and its crystal) — each seeing its neighbourhood and moving itself, trained on a shared reward, ideally whether the board routes and passes. The only way this helps is as a generator: placements the anneal never reaches. It cannot help as a ranker; ordering one board's own placements has come back near chance four times (ISOnet, the fold, the mould, the GNN), and the routers agree on 52% of a mixed board's placements.
What the literature shows
Four papers were read in full (RL_PCB, Cheng/Kahng, PCBAgent, Maji et al.); the rest from abstracts or search snippets, marked.
| Work | Agents | Judge | Unseen designs | What it shows |
|---|---|---|---|---|
| RL_PCB — Vassallo & Bajada, DATE 2024 (pdf, code) | Every part an agent, one shared policy (TD3/SAC); 23-value local observation; reward = own wirelength + shared HPWL − overlap | OpenROAD PcbRouter, single layer; routed wirelength only, no completion or DRC | 3 boards of 3–12 parts, trained on 6 | −17% (TD3) / −21% (SAC) routed wirelength against SA-PCB — but SA ran 500 iterations at default settings, the winning reward was picked on the test boards, 10% overlap was tolerated, and 4 of 10 reward settings lost to SA |
| PCBAgent — ASP-DAC 2025 (pdf) | Single agent, not MARL; an LLM re-weights from user requests | HPWL and surface-wire count; no router | Per-design fine-tuning | ~90% of manual quality on 17 industrial tasks, by proxy |
| Maji et al. — DAC 2025 LBR (arXiv 2503.22958) | One tabular Q-agent per analog device group, one over groups | Post-layout simulation (Virtuoso, Calibre, TSMC 40nm) | Per-design search | Better than SA on 3 analog circuits; no numbers in the text |
| AlphaChip — Mirhoseini et al., Nature 2021 (not re-read) | Single agent, macros one at a time | Proxy: wirelength + density + congestion | Claimed via pre-training | ≥ human and SA, claimed |
| Cheng, Kahng et al. — IEEE TCAD 2026 (pdf) | Re-trained AlphaChip from scratch and from Google's 2024 checkpoint | Cadence Innovus, post-route PPA | Ariane, BlackParrot, MemPool | Tuned SA and humans beat AlphaChip, significantly, with fewer resources; pre-training did not reliably help; no outside reproduction as of Nov 2025. Google's rebuttal (arXiv 2411.10053) disputes the setup and gives no public numbers |
| WireMask-BBO — NeurIPS 2023 (arXiv 2306.16844), abstract | Black-box search, no RL | HPWL | — | Beats the RL placers on 6 of 7 benchmarks in ~8 min |
| ChiPBench — 2024 (arXiv 2407.15026), abstract | Six AI placers | Full OpenROAD flow | 20 circuits | A placer that wins on intermediate metrics can lose on final PPA: the proxy misranks |
| DeepPR — NeurIPS 2021 (arXiv 2111.00234), abstract | RL placer + RL router, cooperative | Routed wirelength | Per-design | Nearest thing to a placer/router pair; weak baselines |
| DeepPCB (InstaDeep), vendor material | RL place + route, "self-play" | Own simulator | — | Case studies only; no peer-reviewed benchmark, no comparison against SA or people |
| Zhang & Lin, J. Intell. Manuf. 2024 (link), paywalled | One agent a facility (Wolf-PHC) | Real 3D A* router | — | Closest analogue in shape; numbers not read |
| Wagner et al., Procedia CIRP 2026 (link), snippet only | Single vs multi-agent, factory layout | — | — | Snippet says the multi-agent advantages do not overcome partial observability and competition; unconfirmed |
Nothing substantive was found on Nash-equilibrium placement or on an adversarial placer-against-router setup. "Multi-agent" in this literature mostly means one policy shared across parts, which is a way to scale to any part count rather than a negotiation between players. No credible MAPPO, QMIX or mean-field placement result was found.
What our own record says
- The pool barely grows. Any-clean among k anneals on 92 boards: 76 / 80 / 82 / 84% for k = 1 / 3 / 6 / 20. At ~20 parts, 20k iterations is near-exhaustive.
- The last learned generator lost. FlowPlace, clean@1 65% against the anneal's 82% on 100 held-out boards (−17 pp). 84% of its training data was the annealer's own output.
- The anneal already matches people. 705 untouched boards: human 74.5%, one anneal 74.8%, best of six 80.4%.
- Failures are mostly not placement. Unaided replay of 21 Sep, 40/59: of 19 failures, 12 front-half only, 3 copper only, 4 both. Claude over MCP, 17 Sep, 54/60: all six copper — QFN-56 escape and 2–3 A rails. Of 11 copper failures, 9 close under no router and no seed.
- It was decided once. The Quorra note of 4 Sep: do not build RL, diffusion or GNN placers.
If it is reopened
- The cheap decisive test. RL_PCB's open code against
placer.anneal()on held-out corpus boards at an equal time budget, judged by both routers and kicad-cli DRC. Nobody has published it, so either answer is a finding, and a null is about a day. - The one regime with an argument: large boards. On the 245-part PCBGolf board 20k iterations is not exhaustive, and a shared per-part policy scales with part count where the anneal's budget does not. The test is whether it beats the anneal at an equal budget on boards over ~100 parts. The corpus is thin there, and that would be the first thing to fix.
- What would not help: another within-board judge, or any result on a proxy (HPWL, congestion) alone — ChiPBench and our own ISOnet history both say the proxy misranks.