Routing on a bigger machine
Label the corpus, or build hundreds of boards, on rented Azure machines. Each tool here sets up the machines, runs the job, and brings the results home.
Which tool
| Job | Tool | What runs on the machine |
|---|---|---|
| Label the corpus the way the shipped model's labels were made | tools/azure_astar.py | The A* router, judged by kicad-cli, on several machines |
| Build boards as a laptop does, for speed work or a replay of saved designs | tools/azure_build.py | The build, with Lightcycle at the pinned version, on one large machine |
| Relabel the older freerouting corpus | tools/remote_route.sh | freerouting under Java, on one VM. See The freerouting relabel. |
tools/azure_route.py, tools/azure_ab.py and tools/azure_bench.py are
research harnesses: the board bank, the router comparison and the placer bench.
Each file's opening comment says how to run it.
Every tool needs the Azure command line, logged in once in a browser:
az loginLabel the corpus with the A* router
tools/azure_astar.py routes the corpus's stored placements again with the
A* router, and judges each with kicad-cli. The shipped
routability model was fitted on labels made this
way.
-
See what it would rent, and what it would cost:
.venv/bin/python tools/azure_astar.py plan --vms 2 -
Pack the shards, create the machines and start them.
packalone builds the shards and spends nothing..venv/bin/python tools/azure_astar.py up -
Watch them, then bring the labels home:
.venv/bin/python tools/azure_astar.py status .venv/bin/python tools/azure_astar.py collect -
Stop paying. This deletes everything:
.venv/bin/python tools/azure_astar.py down
| Option | Default | Sets |
|---|---|---|
--vms | 2 | Machines rented |
--size | Standard_D32als_v7 | Machine size |
--corpus | .cache/corpus | The corpus whose placements are labelled again |
--out | .cache/corpus-astar | Where the labels land |
--hours-guard | 8 | Each machine shuts itself down this many hours after the start, plus three |
The judge is pinned to kicad-cli 10.0.6, from KiCad's 10.0 PPA. A machine whose kicad-cli is any other version refuses to start the work. A corpus labelled by two judges would be two corpora.
Build boards on one big machine
tools/azure_build.py rents one machine that builds boards as this laptop does.
-
Rent the machine:
.venv/bin/python tools/azure_build.py up --hours 10 -
Ship the tree and its caches, and set the machine up:
.venv/bin/python tools/azure_build.py push -
Run a command there. It runs in
~/run/soldermask:.venv/bin/python tools/azure_build.py ssh "command" -
Bring a result home:
.venv/bin/python tools/azure_build.py get REMOTE_PATH LOCAL_DIR -
Stop paying. This deletes the resource group:
.venv/bin/python tools/azure_build.py down
Other verbs: sync ships the working tree's code alone between jobs, and
status shows the machine.
| Option | Default | Sets |
|---|---|---|
--size | Standard_D160als_v7 | Machine size |
--hours | 10 | The machine shuts itself down after this many hours |
--group | soldermask-build-eastus | Resource group |
--location | eastus | Region |
push ships the working tree, tracked and untracked, without landing/,
paper/, pitch/ and .cache/. It adds the caches a --design build opens:
the catalogue, the symbols, the footprints, the board bank and the datasheet
records. Lightcycle is built from the tag router.LIGHTCYCLE pins, taken from
a Lightcycle checkout beside this one at ../lightcycle. KiCad comes from its
10.0 PPA, and ngspice from Ubuntu.
The machine's state is kept in .cache/azure-build/state.json.
The freerouting relabel
The router here is freerouting, which labels the older corpus for
learn relabel. It is retired from the build, so only
this pass needs Java. Each route is one Java process using about two cores, for
five seconds to three minutes. learn relabel --rules on 1,423 examples is
about 17 core-hours.
| Machine | Routes at once | Time |
|---|---|---|
| This laptop | 6 | An evening |
| 64 vCPUs | 30 | About half an hour |
Nothing in the code changes between the two. Every harness already takes
--workers.
Run a pass in Azure
tools/remote_route.sh sets up the machine, runs the command and copies the
corpus back.
-
Log in, once, in a browser:
az login -
Check what the subscription allows in the region:
tools/remote_route.sh quota -
Create the resource group and a spot VM with Java 25, Python and the package:
tools/remote_route.sh up -
Upload the code, the jar, every DSN and the corpus. The corpus alone is about 560 MB:
tools/remote_route.sh push -
Run the pass, and watch it:
tools/remote_route.sh run learn relabel --rules --workers 30 --hours 2 tools/remote_route.sh tail -
Bring the corpus back. Only files newer on the VM are copied:
tools/remote_route.sh pull -
Stop paying. The disk stays, and the meter stops:
tools/remote_route.sh down
Other verbs: start wakes a stopped machine, ssh opens a shell, and destroy
deletes the resource group.
The VM gets Java and freerouting. The script does not build the A* router there.
Settings
| Variable | Default |
|---|---|
AZ_RG | soldermask-route |
AZ_LOC | uksouth |
AZ_VM | soldermask-router |
AZ_SIZE | Standard_F64als_v6 |
AZ_USER | soldermask |
AZ_KEY | ~/.ssh/id_ed25519.pub |
AZ_SPOT | 1 (a spot VM) |
What goes up
soldermask/,tools/,tests/,vendor/(the jar),pyproject.tomlandpriors_blocks.json.- Every
*.dsnin the repository, at its own path. The corpus names its sources by path, so a relabel needs them where it expects them. .cache/corpus, and each board bank'sboards.jsonl.
The 6 GB of .kicad_pcb sources under .cache/boards/*/src stay home, as do
the projects and the evaluation trees apart from their DSNs.
What comes back
.cache/corpus comes back file by file, only where the VM's copy is newer.
Each example is one .npz file, written whole. So a partial run merges cleanly,
whether it stopped on a spot eviction or a budget. Running the same command
again picks up where the first run stopped.
One writer at a time
Nothing local may write the same examples while the VM runs.
learn generate writes new ids, so it is safe beside a remote relabel. Two
relabels of the same corpus are not.
Quota
Free-trial and startup subscriptions cap a region at 10 or 20 vCPUs. quota
prints the F-family line.
If it is below the size you asked for, request an increase in the Azure portal;
it usually takes hours. Meanwhile, set AZ_SIZE=Standard_F16s_v2. Sixteen vCPUs
still route eight at a time and leave the laptop free.
Cost
A 64-vCPU F-series machine costs about $3 an hour on demand, and a third of that as a spot VM. A relabel of the size above costs about a dollar.
A GPU buys nothing here. The router is Java on the CPU, and the model the corpus trains is small enough to train on the CPU too.
Java
The VM gets Temurin 25 from Adoptium's apt repository. cli.find_java() tries
Homebrew's installs first, then java on the path, and runs each one before
trusting it. So the same code finds the right Java on both machines.