Multi-agent RL for the placer
No published work shows multi-agent reinforcement learning beating a tuned simulated annealer on routability judged by a real router, on designs it was not trained on — and this corpus's own record says the annealer is not what limits the pass rate. This is a literature review of 24 Sep 2026, parked, with the evidence logged since it added at the end of "What our own record says". No MARL experiment has been run and nothing was built; every number from a paper below is that paper's own, and every number of ours is from CHANGELOG.md Unreleased with its date.
The question
Several learned agents — one a part, or one a cluster (a regulator and its caps, an MCU and its crystal) — each seeing its neighbourhood and moving itself, trained on a shared reward, ideally whether the board routes and passes. The only way this helps is as a generator: placements the anneal never reaches. It is not needed as a ranker: ordering one board's own placements has come back near chance four times (ISOnet, the fold, the mould, the GNN), and in a build Lightcycle now routes all six anneals and ranks them by what it did (tandem.py).
What the literature shows
Four papers were read in full (RL_PCB, Cheng/Kahng, PCBAgent, Maji et al.); the rest from abstracts or search snippets, marked.
| Work | Agents | Judge | Unseen designs | What it shows |
|---|---|---|---|---|
| RL_PCB — Vassallo & Bajada, DATE 2024 (pdf, code) | Every part an agent, one shared policy (TD3/SAC); 23-value local observation; reward = own wirelength + shared HPWL − overlap | OpenROAD PcbRouter, single layer; routed wirelength only, no completion or DRC | 3 boards of 3–12 parts, trained on 6 | −17% (TD3) / −21% (SAC) routed wirelength against SA-PCB — but SA ran 500 iterations at default settings, the winning reward was picked on the test boards, 10% overlap was tolerated, and 4 of 10 reward settings lost to SA |
| PCBAgent — ASP-DAC 2025 (pdf) | Single agent, not MARL; an LLM re-weights from user requests | HPWL and surface-wire count; no router | Per-design fine-tuning | ~90% of manual quality on 17 industrial tasks, by proxy |
| Maji et al. — DAC 2025 LBR (arXiv 2503.22958) | One tabular Q-agent per analog device group, one over groups | Post-layout simulation (Virtuoso, Calibre, TSMC 40nm) | Per-design search | Better than SA on 3 analog circuits; no numbers in the text |
| AlphaChip — Mirhoseini et al., Nature 2021 (not re-read) | Single agent, macros one at a time | Proxy: wirelength + density + congestion | Claimed via pre-training | ≥ human and SA, claimed |
| Cheng, Kahng et al. — IEEE TCAD 2026 (pdf) | Re-trained AlphaChip from scratch and from Google's 2024 checkpoint | Cadence Innovus, post-route PPA | Ariane, BlackParrot, MemPool | Tuned SA and humans beat AlphaChip, significantly, with fewer resources; pre-training did not reliably help; no outside reproduction as of Nov 2025. Google's rebuttal (arXiv 2411.10053) disputes the setup and gives no public numbers |
| WireMask-BBO — NeurIPS 2023 (arXiv 2306.16844), abstract | Black-box search, no RL | HPWL | — | Beats the RL placers on 6 of 7 benchmarks in ~8 min |
| ChiPBench — 2024 (arXiv 2407.15026), abstract | Six AI placers | Full OpenROAD flow | 20 circuits | A placer that wins on intermediate metrics can lose on final PPA: the proxy misranks |
| DeepPR — NeurIPS 2021 (arXiv 2111.00234), abstract | RL placer + RL router, cooperative | Routed wirelength | Per-design | Nearest thing to a placer/router pair; weak baselines |
| DeepPCB (InstaDeep), vendor material | RL place + route, "self-play" | Own simulator | — | Case studies only; no peer-reviewed benchmark, no comparison against SA or people |
| Zhang & Lin, J. Intell. Manuf. 2024 (link), paywalled | One agent a facility (Wolf-PHC) | Real 3D A* router | — | Closest analogue in shape; numbers not read |
| Wagner et al., Procedia CIRP 2026 (link), snippet only | Single vs multi-agent, factory layout | — | — | Snippet says the multi-agent advantages do not overcome partial observability and competition; unconfirmed |
Nothing substantive was found on Nash-equilibrium placement or on an adversarial placer-against-router setup. "Multi-agent" in this literature mostly means one policy shared across parts, which is a way to scale to any part count rather than a negotiation between players. No credible MAPPO, QMIX or mean-field placement result was found.
What our own record says
- The pool barely grows. Any-clean among k anneals on 92 boards: 76 / 80 / 82 / 84% for k = 1 / 3 / 6 / 20. At ~20 parts, 20k iterations is near-exhaustive.
- The last learned generator lost. FlowPlace, clean@1 65% against the anneal's 82% on 100 held-out boards (−17 pp). 84% of its training data was the annealer's own output.
- The anneal already matches people. 705 untouched boards: human 74.5%, one anneal 74.8%, best of six 80.4%.
- Failures are mostly not placement. Unaided replay of 21 Sep, 40/59: of 19 failures, 12 front-half only, 3 copper only, 4 both. Claude over MCP, 17 Sep, 54/60: all six copper — QFN-56 escape and 2–3 A rails. Of 11 copper failures, 9 close under no router and no seed.
- It was decided once. The Quorra note of 4 Sep: do not build RL, diffusion or GNN placers.
Since this review
- A published placer lost to the anneal (8 Nov 2026). Cypress (NVIDIA and Cornell, ISPD 2025), on 60 held-out boards, six layouts a board, the same router: ours routes 48 and Cypress 31; Cypress routes no board ours fails, and ours routes 17 Cypress fails (exact McNemar p = 1.5e-5).
- The anneal is worse than people on order (2 Oct 2026). On 72 paired boards the author turns same-footprint parts the same way far more often (0.73 against 0.53 by the rubric) and puts more on shared rows and columns (0.89 against 0.81). Nothing in the placer's cost rewards order.
- On large multilayer boards people are far ahead (15-16 Nov 2026). The layout gym re-lays 26 open-source KiCad boards of 4 to 10 layers under each board's own rules: the authors pass 14. On the authors' placements soldermask passed 1, then 4 after a night's fixes; on a placement of its own it passes 1. "The anneal already matches people" above is a two-layer, router-only result.
If it is reopened
- The cheap decisive test. RL_PCB's open code against
placer.anneal()on held-out corpus boards at an equal time budget, judged by both routers and kicad-cli DRC. Nobody has published it, so either answer is a finding, and a null is about a day. - The one regime with an argument: large boards. On the 245-part PCBGolf board 20k iterations is not exhaustive, and a shared per-part policy scales with part count where the anneal's budget does not. The test is whether it beats the anneal at an equal budget on boards over ~100 parts. The corpus is thin there; the bench is not any more. The layout gym's 26 dev boards (
tools/layout_gym.py, 15 Nov 2026) have a median of 180 footprints, 16 of them over 100, each judged under its own rules against the author's own layout. - What would not help: another within-board judge, or any result on a proxy (HPWL, congestion) alone — ChiPBench and our own ISOnet history both say the proxy misranks.