The rubric

A board either routes or it does not, and every way one placement is better than another that both route is invisible to that. The rubric asks the other half of the question: given two placements a router would both accept, which one would a reviewer prefer, and why?

Where the judgement comes fromCopy Anchor Link

The corpus has twelve thousand boards and no commentary on any of them. human_gap.py can therefore only ask the coarse question — does a person's placement route clean where the placer's does not, paired per board, McNemar's on the discordant pairs.

John Teel's Design Battle is the exception the rubric is built on. He hands three engineers the same schematic, races them to a layout, reviews each on camera, and publishes the files. So these boards arrive with the thing the corpus has none of: a professional's written judgement of what is good about them and what is not.

The boards are somebody else's to publish, so none are vendored. The default location is $SOLDERMASK_HOME/battle, and a missing directory prints where to put them rather than failing.

.venv/bin/python tools/rubric.py                 the report, from boards on disk
.venv/bin/python tools/rubric.py --write         ...and write rubric/summary.json
.venv/bin/python tools/rubric.py --boards DIR    somewhere other than the default
.venv/bin/python tools/rubric.py --calibrate     score the verdicts against the reviewer

What it measures, and what it refuses toCopy Anchor Link

Three of the reviewer's criteria are measured here in millimetres:

The report names the four it does not measure rather than implying a complete scorecard: antenna over the edge, stitching vias under the hot part, pours rather than traces for power, and silkscreen over pads.

placer.wirelength(), decap_distances() and pad_overlap() are public and unweighted for this reason. _cost() discounts ground and carries the terms the annealer needs; a placement that came from a person was never annealed against any of that, so holding one to those weights would be scoring it in a currency it never traded in.

The reader's rule is human_gap's, unchanged: a board whose bottom-side parts were brought to the front is not the author's placement any more and is set aside — which is what happens to the four-layer entry and its twelve back-side parts.

The resultCopy Anchor Link

Measured on battle 2, five seeds a board: the placer is at least as good on wirelength on one board of two, on median decoupling distance on two of two, and on the worst-placed capacitor on neither.

A single seed had said parity on wirelength on the board where five seeds say 1.39×. That is the whole reason the tool takes a median over seeds, and the reason the line above says what it says.

From a measure to a gradeCopy Anchor Link

A measure is not a grade until something says where the line is.

battle/rules.json is what a reviewer grades a layout on: 82 rules, begun as Teel's eleven-rule checklist and the layout half of his 21 mistakes plus every remark he made on camera across the Design Battles, and now carrying, beside each rule, what the manufacturers' layout guidelines say with numbers — Espressif for the module and its antenna, TI for the switcher, the USB pair and EMI. Each rule is tagged with what it could be computed from: geometry (the placement and footprints alone — what the placer decides), layout (the routed board), drc, netlist (before there is a layout at all), or no (not a property of the layout files).

soldermask/assets/rubric.json says, for every rule the checker measures, the scalar it is judged on, which way is bad, and the line — a number with its source where a source gave one, or a percentile of what professionally made boards do where none did. (Until 11 Sep 2026 these were battle/thresholds.json and battle/reference.json; they moved into the package, as one file, when the pipeline started grading its own boards against them, because an install has no battle/ directory.)

The reference half is built by tools/rubric_reference.py from the corpus's own boards by owner: 505 from SparkFun, Adafruit, Olimex, Soldered, Winterbloom, Great Scott Gadgets, Seeed, M5Stack, Tinkerforge, Watterott, Particle, ProtoCentral, 1BitSquared, Bitcraze, Pimoroni and OpenBCI, against 223 of everyone else's. It was 256 against 286 from 6 Sep 2026, 326 against 223 on the morning of 13 Sep when the crystal scalar was added, and 505 that evening when the vendor sweep landed; the calibration below read 34 of 40 under all three. Percentiles only — and the split shows:

Calibration: 32 of 38Copy Anchor Link

rubric.verdicts() resolves a grade against the lines, and tools/rubric.py --calibrate scores it against the reviewers' own labels: 32 of 38 decided labels agree.

Every miss has a cause that is not a line. One reviewer called the same board both the most compact and too compact. One antenna was graded on its keepout, and the keepout is not measured yet. One inner pour is the file disagreeing with its own Gerbers.

Three lines cut against the reviewer and are drawn from his words anyway, with the reference's number beside them so the disagreement is on record: professionally made two-layer boards mostly route power as traces (pro p75 of the poured fraction is 0), are mostly two-layer (p75 is 2), and usually carry vias on the USB pair (p50 is 3).

What designers change: the revision pairsCopy Anchor Link

The Design Battle labels say whether one board passes a rule. They cannot say whether one placement is better than another, and that is the question a placer needs answered. tools/revision_pairs.py goes back into the git history of the boards the bank already holds for a label nobody had to be asked for: a later commit by the same designer that kept the parts and moved some. It is weak -- a fix for the router counts as much as a change of taste -- and it is signed, and there are three thousand of them where there were forty labels.

Measured 2026-09-13, 5,318 of 5,487 repositories read: 4,294 distinct pairs, 1,273 set aside because the nets changed too, 3,021 measured from 1,368 repositories. The rubric registers a change on 78% of them, wirelength aside. By the exact sign test, later revision against earlier: decoupling distance improves (322 better / 243 worse, p 0.001, and in the re-layouts rather than the nudges), connectors end up seated on an edge more often (179 / 139, p 0.03), wirelength gets shorter by a hair (1,351 / 1,220, p 0.01, median change 0.0%), orientation consistency gets worse (357 / 466, p 0.0002), and functional grouping (605 / 626) and occupancy (815 / 845) do not move. Orientation and decoupling survive a Bonferroni correction at ten scalars; the other two do not.

So the two placement rules the placer already pays for, the decoupling term and the edge pull, are the two things designers demonstrably fix when they revise; the one measure people win against the annealer corpus-wide, orientation, is one they trade away when re-laying a board, so a placement objective fitted to it would be fitting to something engineers give up on purpose; and grouping and occupancy, two of this rubric's headline placement rules, are invisible across three thousand revisions -- either designers do not act on them or these measures cannot see the grouping designers do. The CHANGELOG carries the full accounting, and withdraws the interim reading at 301 pairs that nothing the rubric measures is what designers move parts for.

Where the moved parts went (revision_pairs.py moves, same pairs, 29 tests so the line is p 0.0017): a capacitor that moves ends nearer the part it serves (296 pairs against 199, p 1.5e-5), and a crystal that moves ends nearer its part (64 against 35, p 0.005) -- the crystal rule is in the catalogue with no measure behind it, and this is the first evidence designers act on it. Under the line but consistent: parts close up rather than spread out (700 / 612), drift toward the edge (764 / 678), and leave rows more than they join them (187 / 150). Null: flightline crossings (916 / 873) and displaced chords, the gap to like parts, the distance to a regulator. A revision is not an untangling and not a grouping; it is capacitors and crystals going to their pins, and the placer already pays for the first.

The crystal is now measured (crystal-placement, the far clock pin's distance to its IC, line at the professional p75 of 9.34 mm), and on the pairs it falls on 76 against 37 (p 0.0003). What the pairs cannot do is set a weight. revision_pairs.py fit, a pairwise logistic fit held out by repository, predicts the later revision at 0.521 -- wirelength alone at 0.525, crystal alone at 0.673 on the pairs where it moved -- and every ratio's 90% bootstrap band includes zero. Almost every pair moves wirelength and little else, and wirelength barely predicts what a designer prefers; a weight is a rate of exchange and only a trade-off can measure one. The git history fixes the sign of a placement term. Its size needs a person choosing between two placements of one board.

Within a board (tools/rubric_within.py, 5,009 corpus boards with four or more stored anneals): decoupling and crystal distance differ between a board's own anneals on every board that has them, with a within-board range most of the between-board spread; orientation and grouping on nine in ten; wirelength differs everywhere but its range is 11% of the spread across boards, which is board size. Against the router, every scalar's within-board AUC sits at 0.48 to 0.53, orientation consistency the one below half. Of 60,748 pairs of a board's anneals, 37% are trade-offs. So the rubric can order a board's anneals where the routability judges could not, its order costs nothing on routing, and the pairs a person should be asked to choose between already exist.

The crystal term, in the placer and measured (placer.CRYSTAL_W, off in the tree; tools/crystal_ab.py, 80 held-out crystal boards, two seeds, weights 0, 2, 4, both routers under KiCad's DRC): at weight 4 the far clock pin comes from 14.8 to 8.6 mm at the median, wirelength and decoupling do not move (paired medians +0.04% and 0.00 mm), and routing is null under both judges -- freerouting 4 gains against 5 losses, A* 11 against 5, p 0.21. The cost the within-board test feared, a crystal crowding its pins, does not show on 80 boards; neither does anything inside ten points. Switching it on is a placer version and is not done here.

Where it shows upCopy Anchor Link

On every board the pipeline builds, since 11 Sep 2026. soldermask/rubric.py carries the measures — the geometry rules off the placement, the layout rules off the router's copper and KiCad's fill — and workspace.realise() runs them after the pour and before the board is written. The grades go on the board (board["rubric"]: every measure, every verdict, the reference's date) and a rule the board is bad on is one line in the notes, beside the gaps: the reviewer's rule decap-distance: median 9.35 mm, worst 9.35 mm, 1 caps — a professionally made board's p75 is 3.3 mm. Good and neutral grades are not notes, because a note is a caveat. A rule another stage already reports as a gap (a connector short of the edge, from placer.placement_notes()) is not said twice, and a failure that is the pipeline's rather than the board's says so — no mounting holes, because nothing draws one yet; power as tracks, because ground is what gets poured. The coverage line closes it: graded on 11 of the reviewer's 82 rules, the ones this pipeline can measure … the other 71 are not checked, so the count is never read as a scorecard. The command line prints the same as a rubric section, one rule a line with its grade.

It is reporting and nothing more. Nothing in it moves a part or fails the verdict: repair.passes() is the bar and this is not a second one.

Fifteen rules are measurable on a built board today, fourteen of them with a line to be held to (the hot loop is measured and not yet graded), and a board is graded on the ones its parts and nets give occasion for — eleven on the fixture LDO: compactness, decap-distance, connector-on-edge, antenna-over-edge, component-orientation, functional-grouping and hot-loop off the placement; layer-count, mounting-holes, stitching-vias, stitching-under-hot-part, ground-plane-strategy, pour-for-power, diff-pair and diff-pair-vias off the copper. Silkscreen over pads stays gerber.silk_report()'s and the DRC-backed rules stay KiCad's; a crystal's guard, a trace under a QFN and a plane split under a signal are not computed anywhere yet.

The rubric run over the reviewers' own boards has a card on /benchmarks, read off rubric/summary.json the way the eval runs are read off eval*/summary.json — and doubly gitignored for the same reason twice over, since most checkouts will never have the boards to score.

tests/test_rubric.py asserts what the anneal rests on, which is not a rubric number at all: that placer._Index.local() moves exactly as _cost() does, over four hundred random moves. local() exists so the inner loop does not add the whole board up twice a step, and it is correct only while it charges what _cost() charges — the anneal minimises one and the candidates are ranked by the other, so a drift between them would leave the placer optimising one thing and reporting another with nothing to catch it. Adding a cost term put that at risk, so it is a test rather than a habit.