soldermaskdocs

Multi-agent RL for the placer

No published work shows multi-agent reinforcement learning beating a tuned simulated annealer on routability judged by a real router, on designs it was not trained on — and this corpus's own record says the annealer is not what limits the pass rate. This is a literature review of 24 Sep 2026, parked, with the evidence logged since it added at the end of "What our own record says". No MARL experiment has been run and nothing was built; every number from a paper below is that paper's own, and every number of ours is from CHANGELOG.md Unreleased with its date.

The question

Several learned agents — one a part, or one a cluster (a regulator and its caps, an MCU and its crystal) — each seeing its neighbourhood and moving itself, trained on a shared reward, ideally whether the board routes and passes. The only way this helps is as a generator: placements the anneal never reaches. It is not needed as a ranker: ordering one board's own placements has come back near chance four times (ISOnet, the fold, the mould, the GNN), and in a build Lightcycle now routes all six anneals and ranks them by what it did (tandem.py).

What the literature shows

Four papers were read in full (RL_PCB, Cheng/Kahng, PCBAgent, Maji et al.); the rest from abstracts or search snippets, marked.

WorkAgentsJudgeUnseen designsWhat it shows
RL_PCB — Vassallo & Bajada, DATE 2024 (pdf, code)Every part an agent, one shared policy (TD3/SAC); 23-value local observation; reward = own wirelength + shared HPWL − overlapOpenROAD PcbRouter, single layer; routed wirelength only, no completion or DRC3 boards of 3–12 parts, trained on 6−17% (TD3) / −21% (SAC) routed wirelength against SA-PCB — but SA ran 500 iterations at default settings, the winning reward was picked on the test boards, 10% overlap was tolerated, and 4 of 10 reward settings lost to SA
PCBAgent — ASP-DAC 2025 (pdf)Single agent, not MARL; an LLM re-weights from user requestsHPWL and surface-wire count; no routerPer-design fine-tuning~90% of manual quality on 17 industrial tasks, by proxy
Maji et al. — DAC 2025 LBR (arXiv 2503.22958)One tabular Q-agent per analog device group, one over groupsPost-layout simulation (Virtuoso, Calibre, TSMC 40nm)Per-design searchBetter than SA on 3 analog circuits; no numbers in the text
AlphaChip — Mirhoseini et al., Nature 2021 (not re-read)Single agent, macros one at a timeProxy: wirelength + density + congestionClaimed via pre-training≥ human and SA, claimed
Cheng, Kahng et al. — IEEE TCAD 2026 (pdf)Re-trained AlphaChip from scratch and from Google's 2024 checkpointCadence Innovus, post-route PPAAriane, BlackParrot, MemPoolTuned SA and humans beat AlphaChip, significantly, with fewer resources; pre-training did not reliably help; no outside reproduction as of Nov 2025. Google's rebuttal (arXiv 2411.10053) disputes the setup and gives no public numbers
WireMask-BBO — NeurIPS 2023 (arXiv 2306.16844), abstractBlack-box search, no RLHPWL—Beats the RL placers on 6 of 7 benchmarks in ~8 min
ChiPBench — 2024 (arXiv 2407.15026), abstractSix AI placersFull OpenROAD flow20 circuitsA placer that wins on intermediate metrics can lose on final PPA: the proxy misranks
DeepPR — NeurIPS 2021 (arXiv 2111.00234), abstractRL placer + RL router, cooperativeRouted wirelengthPer-designNearest thing to a placer/router pair; weak baselines
DeepPCB (InstaDeep), vendor materialRL place + route, "self-play"Own simulator—Case studies only; no peer-reviewed benchmark, no comparison against SA or people
Zhang & Lin, J. Intell. Manuf. 2024 (link), paywalledOne agent a facility (Wolf-PHC)Real 3D A* router—Closest analogue in shape; numbers not read
Wagner et al., Procedia CIRP 2026 (link), snippet onlySingle vs multi-agent, factory layout——Snippet says the multi-agent advantages do not overcome partial observability and competition; unconfirmed

Nothing substantive was found on Nash-equilibrium placement or on an adversarial placer-against-router setup. "Multi-agent" in this literature mostly means one policy shared across parts, which is a way to scale to any part count rather than a negotiation between players. No credible MAPPO, QMIX or mean-field placement result was found.

What our own record says

  • The pool barely grows. Any-clean among k anneals on 92 boards: 76 / 80 / 82 / 84% for k = 1 / 3 / 6 / 20. At ~20 parts, 20k iterations is near-exhaustive.
  • The last learned generator lost. FlowPlace, clean@1 65% against the anneal's 82% on 100 held-out boards (−17 pp). 84% of its training data was the annealer's own output.
  • The anneal already matches people. 705 untouched boards: human 74.5%, one anneal 74.8%, best of six 80.4%.
  • Failures are mostly not placement. Unaided replay of 21 Sep, 40/59: of 19 failures, 12 front-half only, 3 copper only, 4 both. Claude over MCP, 17 Sep, 54/60: all six copper — QFN-56 escape and 2–3 A rails. Of 11 copper failures, 9 close under no router and no seed.
  • It was decided once. The Quorra note of 4 Sep: do not build RL, diffusion or GNN placers.

Since this review

  • A published placer lost to the anneal (8 Nov 2026). Cypress (NVIDIA and Cornell, ISPD 2025), on 60 held-out boards, six layouts a board, the same router: ours routes 48 and Cypress 31; Cypress routes no board ours fails, and ours routes 17 Cypress fails (exact McNemar p = 1.5e-5).
  • The anneal is worse than people on order (2 Oct 2026). On 72 paired boards the author turns same-footprint parts the same way far more often (0.73 against 0.53 by the rubric) and puts more on shared rows and columns (0.89 against 0.81). Nothing in the placer's cost rewards order.
  • On large multilayer boards people are far ahead (15-16 Nov 2026). The layout gym re-lays 26 open-source KiCad boards of 4 to 10 layers under each board's own rules: the authors pass 14. On the authors' placements soldermask passed 1, then 4 after a night's fixes; on a placement of its own it passes 1. "The anneal already matches people" above is a two-layer, router-only result.

If it is reopened

  1. The cheap decisive test. RL_PCB's open code against placer.anneal() on held-out corpus boards at an equal time budget, judged by both routers and kicad-cli DRC. Nobody has published it, so either answer is a finding, and a null is about a day.
  2. The one regime with an argument: large boards. On the 245-part PCBGolf board 20k iterations is not exhaustive, and a shared per-part policy scales with part count where the anneal's budget does not. The test is whether it beats the anneal at an equal budget on boards over ~100 parts. The corpus is thin there; the bench is not any more. The layout gym's 26 dev boards (tools/layout_gym.py, 15 Nov 2026) have a median of 180 footprints, 16 of them over 100, each judged under its own rules against the author's own layout.
  3. What would not help: another within-board judge, or any result on a proxy (HPWL, congestion) alone — ChiPBench and our own ISOnet history both say the proxy misranks.

On this page