Tasks/Physical Sciences & Engineering

Robot Routing and Macro-Program Optimization

Jointly plan pickup routes and reusable macro recordings to minimize controller button presses

Physical Sciences & Engineeringrobot planningmacro programs
Background

Macro-operator planning, assembling a robot's plan from reusable subroutines instead of single moves, lacks a settled way to minimize the emitted command program's length. The starting point is a greedy controller that drives a robot to each object and its matching bin along shortest paths, one instruction per step. The work is to redesign that planner: pickup order, routing, and one re-recordable macro chosen together. Replays still spend the execution budget, macro and route determine each other, and one program must serve unseen floor plans.

instruction.mdthis is what the agent is given

The office floor is a mess. Balls of every colour are scattered across an N x N warehouse grid criss-crossed by shelving walls, and each colour has one matching basket somewhere on the floor. You have exactly one robot and one controller. The robot understands four buttons — forward, turn right, turn left, swap — and the controller has one more trick: a macro you can record once and replay as many times as you like. Every button you press costs you, including each macro replay, so a clumsy plan that walks the robot back and forth racks up a huge tab. Your job is to press the fewest buttons that still lands every ball in its own basket before the move budget runs out.

You control a machine through a macro language; the objective is defined below. The entire difficulty is macro compression: a naive plan is a long string of F/R/L/S, but the tours the robot repeats (walk-to-cell, drop, walk-back) are full of structure, and a well-designed macro turns hundreds of moves into a handful of pressed buttons. You are scored relative to a reproduced contest rank-1 solver, so matching a strong human is the bar.

The machine

There is an N x N grid. Cell (0,0) is the top-left; (i,j) is i cells down and j cells right. The outer boundary is walled, and there may be walls between adjacent interior cells. Every cell is reachable from every other cell without crossing a wall.

On the grid are M balls and M baskets. For each type k (0 <= k < M) there is exactly one ball of type k and one basket of type k. Initially each cell holds at most one ball or basket.

The robot starts at (0,0) facing right, holding nothing. You control it with a sequence of buttons:

Basic buttons

  • F (forward): move one cell in the current facing. If a wall blocks the destination, the robot stays put (the button is still spent).
  • R (turn right): rotate 90° clockwise in place.
  • L (turn left): rotate 90° counter-clockwise in place.
  • S (swap): exchange the ball in hand with the ball on the current cell.
  • empty hand + ball here -> pick it up (cell becomes empty);
  • ball in hand + empty cell -> drop it (hand becomes empty);
  • ball in hand + ball here -> swap the two;
  • empty hand + empty cell -> nothing happens.
  • A ball sitting on a basket cell is swapped just like any other ball.

Controller buttons

  • M (macro): if not currently recording, start recording. If currently recording, stop and register the recorded sequence as the macro (replacing any previous one).
  • P (play): replay the most recently registered macro. If none is registered yet, nothing happens.

While recording, any basic F/R/L/S you press is both executed and appended to the macro being recorded. A P pressed while recording replays the previously registered macro, and the basic operations it expands to are executed and appended to the macro being recorded (you cannot replay the macro you are currently recording — only the last completed one).

Example: with RFF already registered, running MFPM records a new macro. M starts recording; F executes+records; P replays the registered RFF (executed and appended); M stops. The basic ops executed are FRFF, and the newly registered macro is FRFF.

Initially no macro is registered and nothing is being recorded.

Budget and scoring

You are given a basic-operation cap T. After macro expansion, at most T basic operations are executed; the T+1-th basic operation is not executed and the run is cut off there. So P is cheap to press but its expansion is charged against T.

Let A be the length of the button sequence you output — M and P each count as one button. Let V be the number of balls sitting on their matching basket at the end of the simulation. The absolute score for a case is:

  • A if V == M (all balls delivered) — lower is better;
  • T * (M - V) if V < M (a big penalty).

You are scored on the absolute score of each case — lower is better. A reproduced contest rank-1 solver's per-case scores ship with the task so you can gauge how strong your solution is, but how the raw scores map to the final reward is deliberately not disclosed. Optimise the raw score itself.

Closing the gap to a top contest solver is almost entirely about better macro synthesis.

Input format (stdin, one instance)

N M T
v_0
...
v_{N-1}        # N lines: v_i is a length-(N-1) 01 string; v_i[j]=1 <=> wall between (i,j) and (i,j+1)
h_0
...
h_{N-2}        # N-1 lines: h_i is a length-N 01 string; h_i[j]=1 <=> wall between (i,j) and (i+1,j)
b_0 c_0 d_0 e_0
...
b_{M-1} c_{M-1} d_{M-1} e_{M-1}   # ball k starts at (b_k,c_k); basket k is at (d_k,e_k)

Constraints: 10 <= N <= 20, N/2 <= M <= 2N, 1 <= T <= 2 N^2 M. All ball and basket cells are distinct.

Output format (stdout)

Print the button sequence, one character per line or all on lines with no spaces — any whitespace layout is fine; only the characters F R L S M P are read, in order. The output length A must be <= T.

What you submit

Your solver lives in methods/main/ (the graded directory), containing:

  • run.sh (required): run once per test case as bash run.sh < instance.txt > out.txt. It must read one instance on stdin and write the button sequence on stdout. Any language.
  • build.sh (optional): if present, the grader runs it once before grading (e.g. to compile a C++ solver). Do your compilation here and have run.sh exec the built binary.

The starter methods/main/run.sh runs the shipped greedy baseline (methods/main/solution.py). Replace it with your solver.

Local dev bench

  • tools/in/ — the 100 visible instances (seeds 0-99).
  • tools/gen — the official generator. Make more instances with ./tools/gen seeds.txt --dir=OUTDIR where seeds.txt is one unsigned-64-bit seed per line. For local testing use seeds in 0..10000 only — the sealed grading seeds live far outside that range, so staying inside it keeps your practice set from colliding with the hidden set.
  • tools/vis — the official visualiser/judge: ./tools/vis in.txt out.txt prints Score = <absolute score> and writes a vis.html you can open.
  • python3 selfcheck.py [N] — runs your run.sh on the first N visible cases (default 100), prints each case's raw score alongside the reference solver's raw score on the same case. Free and unlimited.

The generator, the distribution, and the visualiser are exactly the contest's. The sealed evaluation reruns your run.sh on 200 fresh sealed instances from the same generator and scores them with the same visualiser; there is no feedback loop — whatever sits in methods/main/ at the end is what is graded.

Notes

  • CPU only, no network. Python 3 and g++ are available in both the workbench and the grader.
  • Each sealed case runs your run.sh under a wall-clock cap (20 s/case); a crash, timeout, malformed output, or a case that fails to deliver all balls scores that case's rel at ~0. Complete every case first, then optimise length.
  • The reference contest was a 2-second-per-case time limit; you have more slack here, but the aggregate is dominated by macro quality, not raw search time.

Rollouts

352 minWall clock
$115.51Spend
200.1MTokens
75Versions, 48 kept

On the visible set

0 100 200 300 400 500 600 700 0 300 600 900 1,200 Agent step Visible mean button count ↓ v0 v1 v2 v3 v5 v6 v7 v8 v10 v17 v40 v72
keptrolled backsubmitted
  1. v0Inherited the index-order BFS baseline without macros.All-100 mean: 651.42 buttons (100/100 complete)1 min · $0.36
  2. v1Added stochastic ordering, facing-aware paths, and one repeated macro.Tune mean: 296.20 buttons (80/80 complete)5 min · $0.88
  3. v2Planned directly over 31 fixed macro motion primitives.Tune mean: 235.03 buttons (80/80 complete)11 min · $1.71
  4. v3Expanded dogleg macros and boundary-reset candidates.Tune mean: 225.86 buttons (80/80 complete)23 min · $3.49
  5. v4Tested waypoint-level re-registration among eight macros.First-20 mean: 213.15 buttons (20/20 complete)26 min · $4.02
  6. v5Ported the planner to C++ and widened dogleg lengths.Tune mean: 225.12 buttons (80/80 complete)29 min · $4.75
  7. v6Added simulated-annealing refinement for leading macro-order candidates.Tune mean: 221.41 buttons (80/80 complete)32 min · $5.52
  8. v7Added mirrored, stair, stride, and wall-following macros.Tune mean: 217.86 buttons (80/80 complete)36 min · $6.41
  9. v8Evolved instance-specific macros for three generations.Tune mean: 205.75 buttons (80/80 complete)40 min · $7.24
  10. v9Deepened macro evolution and broadened mutation operators.First-20 mean: 189.70 buttons (20/20 complete)42 min · $7.82
  11. v10Portfolied shallow and deep macro-evolution beams.Tune mean: 201.04; validation mean: 168.95 (100/100 complete)47 min · $9.16
  12. v11Added a decorrelated second deep evolutionary restart.First-20 mean: 188.55 buttons (20/20 complete)51 min · $10.31
  13. v12Tested suffix-S macros that replay pickup and drop actions.First 5 + 8 targeted checks: all complete; 0 beat v1157 min · $11.33
  14. v13Raised both deep-beam macro length caps to 60.First-20 mean: 188.80 buttons (20/20 complete)59 min · $12.20
  15. v14Extended one preserved deep beam with longer macros.First-20 mean: 187.75 buttons (20/20 complete)62 min · $13.17
  16. v15Added exhaustive monotonic route-order polishing.First-20 mean: 185.50 buttons (20/20 complete)64 min · $13.99
  17. v16Tested exact subset DP for small job sets.First-20 mean: 185.50 buttons (20/20 complete)66 min · $14.67
  18. v17Allowed up to ten route-polish rounds.Tune mean: 196.79 buttons (80/80 complete)68 min · $15.29
  19. v18Tried endpoint-preserving simplification of the winning macro.First-20 mean: 185.25 buttons (20/20 complete)71 min · $16.04
  20. v19Refined route orders inside evolutionary fitness selection.First-20 mean: 185.70 buttons (20/20 complete)73 min · $16.81
  21. v20Added extra macro restarts on open boards.First-20 mean: 185.15 buttons (20/20 complete)83 min · $20.19
  22. v21Allocated bonus beams by instance complexity.First-20 mean: 183.35 buttons (20/20 complete)86 min · $21.26
  23. v22Tested multi-macro re-registration on open boards.7/7 targeted cases complete; all matched v21 with 2 M buttons89 min · $22.03
  24. v23Seeded every bonus beam with procedural random-walk macros.First-20 mean: 183.00 buttons (20/20 complete)93 min · $23.63
  25. v24Confined procedural seeds to one bonus restart.First-20 mean: 183.05; generated mean: 206.15 (140/140 complete)100 min · $26.10
  26. v25Enumerated low-complexity two-turn open-board macros.10/10 open cases complete; case 0: 283→281, case 45: 186→187109 min · $29.09
  27. v26Added one-edit descent around the seeded macro leader.First-20 mean: 182.75 buttons (20/20 complete)114 min · $31.09
  28. v27Added short block edits and relocations to macro descent.First-20 mean: 182.45 buttons (20/20 complete)119 min · $33.05
  29. v28Reduced bonus beams on timeout-prone large instances.5/5 stress cases complete in 9.04–14.66 s; case 68: 279→283123 min · $34.85
  30. v29Spent safe small-instance headroom on extra restarts.First-20 mean: 182.15 buttons (20/20 complete)126 min · $36.25
  31. v30Extended block descent from two rounds to five.9/9 targeted cases matched v29 exactly128 min · $37.11
  32. v31Applied one-edit descent to unseeded bonus leaders.First-20 mean: 182.05 buttons (20/20 complete)132 min · $38.62
  33. v32Polished the top ten route finalists instead of six.First-20 mean: 181.80 buttons (20/20 complete)136 min · $40.14
  34. v33Polished all fifteen route finalists.First-20 mean: 181.80 buttons (20/20 complete)139 min · $41.56
  35. v34Extended route block relocations through length six.First-20 mean: 181.50 buttons (20/20 complete)143 min · $43.23
  36. v35Portfolied short-, long-, and chained-block route polishing.First-20 mean: 181.35 buttons (20/20 complete)150 min · $46.26
  37. v36Added a reversed short-block route-polish basin.First-20 mean: 181.30 buttons (20/20 complete)154 min · $47.98
  38. v37Descended from the second- and third-ranked seeded finalists.12/12 targeted cases matched v36 exactly159 min · $49.78
  39. v38Searched again around the actual post-polish winner.First-20 mean: 181.05; cases 2/18: 277→273 and 120→119162 min · $51.03
  40. v39Extended post-final macro search to medium-complexity cases.9 completed targets matched v38; case 32 exceeded 20 s169 min · $54.11
  41. v40Tightened expensive search gates using an M-cubic runtime model.First-20 mean: 181.05; unsafe cases fell to 6.6/7.0/5.6 s181 min · $59.55
  42. v41Inserted swap actions into open-board motion macros.Open-case mean: 151.90 buttons (10/10 complete)211 min · $66.93
  43. v42Re-tested exact ordering on the actual winning macro.34/34 eligible cases matched v40 exactly214 min · $67.39
  44. v43Tested macros encoding an entire pickup-delivery job.10/10 open cases matched v40; alternatives scored 113/68 and 295/184217 min · $68.09
  45. v44Added hierarchical macro redefinition from squared base words.Case 3: 112→111; 10/10 open cases complete223 min · $69.33
  46. v45Annealed route order under the hierarchical macro cost.Case 3 reached 109; 1/20 generated cases improved 160→159230 min · $70.96
  47. v46Tightened the high-M open-board safety threshold.Unsafe case: 20.40→6.04 s with score 264→270232 min · $71.39
  48. v47Narrowed hierarchical search and tested fourth-power macros.22 eligible checks retained 112→109 and 160→159; W⁴ gained 0234 min · $71.92
  49. v48Removed the ineffective fourth-power macro candidates.Target scores held at 109/159/270; unsafe runtime stayed 6.0 s235 min · $72.27
  50. v49Added one-turn descendants to hierarchical macro bases.8 visible ties; 1/14 generated improved 105→104239 min · $73.52
  51. v50Lengthened annealing over the compact hierarchical state set.Wins held at 109/104/159; case 61 runtime reached 19.29 s243 min · $74.33
  52. v51Shortened and rescaled hierarchical route annealing.3/22 cases improved by 6 buttons; 22/22 complete245 min · $75.17
  53. v52Sampled eight independent full-search RNG trajectories.8 salts × 4 cases: 0 improvements; 3 salts regressed case 65248 min · $75.77
  54. v53Added exhaustive canonical macros up to length ten.1/10 generated improved 68→66; 4/4 visible targets tied253 min · $77.30
  55. v54Pruned exhaustive search to exactly length ten.Score 66 retained; worst measured runtime fell 16.5→13.6 s254 min · $77.75
  56. v55Tested exhaustive canonical macros of length eleven.10/10 eligible cases matched v54 exactly256 min · $78.40
  57. v56Extended length-ten exhaustive search to small walled boards.15 visible ties; 1/44 generated improved 100→99260 min · $80.28
  58. v57Extended exhaustive search to job counts 11 through 13.15/15 eligible cases matched v56 exactly262 min · $80.99
  59. v58Tested longer sparse macros with at most two turns.66/66 eligible cases matched v56 exactly267 min · $82.75
  60. v59Fully refined fifty exhaustive candidates instead of fifteen.73/73 eligible cases matched v56 exactly272 min · $84.29
  61. v60Added board-length exhaustive macros on N=11 and N=12.5 visible ties; 1/19 generated improved 72→71277 min · $85.88
  62. v61Restricted board-length enumeration to safe N=12 cases.Winner stayed 71; retained runtimes were at most 15.6 s278 min · $86.54
  63. v62Extended hierarchical macros to small walled instances.3/19 visible and 19/65 generated improved; 53 buttons saved286 min · $89.04
  64. v63Widened walled hierarchical search through 16 jobs.2/9 visible and 14/36 generated improved; 34 buttons saved292 min · $91.43
  65. v64Widened walled hierarchical search through 18 jobs.1/8 visible and 3/15 generated improved; 8 buttons saved297 min · $93.26
  66. v65Widened walled hierarchical search through 20 jobs.5 visible ties; 2/18 generated improved by 3 buttons302 min · $95.30
  67. v66Added monotonic route refinement after hierarchical annealing.37 visible ties; 5/33 generated improved by 7 buttons308 min · $97.28
  68. v67Added a fourth hierarchical base on very small instances.22 visible ties; 0 gains and 2 generated regressions totaling 3 buttons311 min · $98.59
  69. v68Added a preserved second hierarchical annealing run.1/22 visible improved by 1 button; 0/72 generated improved321 min · $102.07
  70. v69Promoted the post-final macro winner into hierarchy bases.Corrected first-20 slice matched v66 exactly328 min · $104.59
  71. v70Added cubed base words to hierarchical candidates.0/49 eligible visible cases improved; case 3 regressed 109→110333 min · $106.61
  72. v71Raised hierarchical route descent from two passes to five.5/5 generated beneficiaries matched v66 exactly334 min · $107.36
  73. v72Added short block relocation to hierarchical route descent.2 visible cases saved 3 buttons; 3/33 generated saved 5 buttons341 min · $110.23
  74. v73Added reversed short-block relocation on smallest instances.44/44 affected visible/generated cases matched v72 exactly344 min · $111.68
  75. v74Extended hierarchical block relocations through length five.44/44 affected visible/generated cases matched v72 exactly347 min · $113.26

On the hidden set

Original metricNormalised score
Starter0.30220.0
Frontier-calibrated reference1.00.6
Upper1.51.0
This run (GPT-5.6-sol)0.9462170.5723
133 minWall clock
$22.95Spend
26.0MTokens
8Versions, 7 kept

On the visible set

0 100 200 300 400 500 600 700 0 2 3 4 6 Agent step Visible mean button count ↓ v0 v1 v2 v2b v2c v3 v4 v5
keptrolled backsubmitted
  1. v0The agent inherited the shipped greedy baseline that used no macros651.429 min · $1.25
  2. v1The agent rewrote the solver in C++ with an ATSP tour and macro DP233.5714 min · $2.54
  3. v2The agent optimized the tour under an exact token-distance metric by annealing221.5222 min · $3.82
  4. v2bThe agent added net-rotation-zero translation macros that jump a fixed vector205.34$6.97
  5. v2cThe agent searched free-form macro strings with doubling build plans201.83$10.12
  6. v3The agent introduced S-carrying macros that absorb the explicit swap tokens200.8580 min · $13.27
  7. v4The agent fixed the swap-parity and clock-polling bugs that spoiled large cases197.7997 min · $16.82
  8. v5The agent scaled the hard deadline with the time limit and tuned ZMAX196.16125 min · $21.59

On the hidden set

Original metricNormalised score
Starter0.30220.0
Frontier-calibrated reference1.00.6
Upper1.51.0
This run (Opus 5)0.9011610.5478
720 minWall clock
$7.42Spend
35.6MTokens
8Versions, 7 kept

On the visible set

0 100 200 300 400 500 600 700 0 2 3 4 6 Agent step Visible mean button count ↓ v0 v1 v2 v3 v4 v5 v6 v7
keptrolled backsubmitted
  1. v0The agent inherited the greedy baseline that carried balls in index order651.42
  2. v1The agent rewrote it in C++ with an annealed tour and macro compression282.97
  3. v2The agent planned paths directly in controller-button space over candidate macros268.45
  4. v3The agent added two-step leg lookahead and multi-macro DP post-compression261.3
  5. v4The agent ran multi-start annealing with Or-opt moves and a leg cache258.82
  6. v5The agent mined macro patterns from the uncompressed tour and searched deeper256.4
  7. v6The agent partitioned the tour by DP while tracking the active macro255.19
  8. v7The agent rerouted legs with a lexicographic Dijkstra that also minimized turns252.42

On the hidden set

Original metricNormalised score
Starter0.30220.0
Frontier-calibrated reference1.00.6
Upper1.51.0
This run (Gemini 3.7 Flash)0.7962620.4858
352 minWall clock
$24.63Spend
57.7MTokens
8Versions, 7 kept

On the visible set

0 100 200 300 400 500 600 700 0 50 100 150 200 Agent step Visible mean button count ↓ v0 v1 v2 v3 v4 v5 v6 v7
keptrolled backsubmitted
  1. v0The agent inherited the greedy BFS baseline that used no macros651.4$0.35
  2. v1The agent recorded one long slide macro and searched the delivery order313.2$0.79
  3. v2The agent tuned the macro length per case and sped up the solver267.1$2.83
  4. v3The agent strengthened the ordering with or-opt and ILS over wider K254.3$3.64
  5. v4The agent added direction-aware DP path reconstruction, deadlines and a safe fallback256.6$10.02
  6. v5The agent retuned the K sweep, the deadline and the ILS restarts253.2$12.02
  7. v6The agent hardened robustness with per-K fallbacks and a tighter time budget253.18$15.12
  8. v7The agent picked the macro's slide direction per case252.4$23.09

On the hidden set

Original metricNormalised score
Starter0.30220.0
Frontier-calibrated reference1.00.6
Upper1.51.0
This run (Kimi K3)0.7417230.4502
67 minWall clock
$14.25Spend
22.0MTokens
4Versions, 3 kept

On the visible set

0 100 200 300 400 500 600 700 0 25 50 75 100 Agent step Visible mean button count ↓ v1 v2 v3 v4
keptrolled backsubmitted
  1. v1The agent inherited the greedy index-order baseline without macros651.42
  2. v2The agent added paired TSP ordering, facing-aware paths and macro compressors265.75
  3. v3The agent ported the solver to C++ and added Held-Karp plus annealing271.44
  4. v4The agent added 3-opt on the paired TSP and deepened the macro DP269

On the hidden set

Original metricNormalised score
Starter0.30220.0
Frontier-calibrated reference1.00.6
Upper1.51.0
This run (Grok 4.6)0.6995050.4208
448 minWall clock
$7.02Spend
48.1MTokens
37Versions, 36 kept

On the visible set

0 100 200 300 400 500 600 700 0 8 15 22 30 Agent step Visible mean button count ↓ v1 v2 v2b v3 v4 v4_selfcheck v5 v5_selfcheck v6 v6_selfcheck v7 v7_selfcheck v8 v8_selfcheck v8c v8d v8f v8g v8h v8i v8j v8k v8l v9 v9_selfcheck v10 v10_selfcheck v11 v11_selfcheck v12 v12_selfcheck v13 v13_selfcheck v14 v14_selfcheck v15 v15_selfcheck
keptrolled backsubmitted
  1. v1The agent rewrote the solver in C++ with Dijkstra legs and substring macros292.3619 min · $0.49
  2. v2The agent switched to min-turn legs and an exact macro-language DP compressor279.95216 min · $3.46
  3. v2bThe agent kept alternate delivery orders and climbed on a compressibility estimate277.58$3.56
  4. v3The agent deduped DP states by macro content and peeled multi-macro programs276.32$3.66
  5. v4The agent added shaped leg variants that end in a free straight run277.45$3.76
  6. v4_selfcheckThe agent ran the DP ladder on the three best alternate orders275.26$3.86
  7. v5The agent deduped DP states by macro content and peeled multi-macro programs276.32$3.97
  8. v5_selfcheckThe agent covered every seen order and added a second independent climb274.86$4.07
  9. v6The agent extended the DP cap ladder now that dedupe made it cheap276.26$4.17
  10. v6_selfcheckThe agent ranked candidate walks by real DP cost before escalating caps274.82$4.27
  11. v7The agent ran the DP ladder on the three best alternate orders275.26$4.37
  12. v7_selfcheckThe agent built walks with a beam over per-leg variants274.48$4.47
  13. v8The agent covered every seen order and added a second independent climb274.86$4.58
  14. v8_selfcheckThe agent widened the beams and escalated the top eight candidate walks273.98$4.68
  15. v8cThe agent ranked candidate walks by real DP cost before escalating caps274.82$4.78
  16. v8dThe agent built walks with a beam over per-leg variants274.48$4.88
  17. v8fThe agent widened the beams and escalated the top eight candidate walks273.98$4.98
  18. v8gThe agent deferred the deep DP caps so candidates got the budget first273.98$5.09
  19. v8hThe agent added a hill climb on raw walk length273.91$5.19
  20. v8iThe agent reallocated budget from the second order's ladder to more candidates273.91$5.29
  21. v8jThe agent ran the beam as an extra stage on small-M corridor cases273.73$5.39
  22. v8kThe agent widened the extra beam stage from M<=10 to M<=12273.73$5.49
  23. v8lThe agent added one deeper DP cap on the best walk273.73$5.59
  24. v9The agent deferred the deep DP caps so candidates got the budget first273.98$5.70
  25. v9_selfcheckThe agent deferred the deep DP caps so candidates got the budget first273.98$5.80
  26. v10The agent added a hill climb on raw walk length273.91$5.90
  27. v10_selfcheckThe agent added a hill climb on raw walk length273.91$6.00
  28. v11The agent reallocated budget from the second order's ladder to more candidates273.91$6.10
  29. v11_selfcheckThe agent reallocated budget from the second order's ladder to more candidates273.91$6.21
  30. v12The agent ran the beam as an extra stage on small-M corridor cases273.73$6.31
  31. v12_selfcheckThe agent ran the beam as an extra stage on small-M corridor cases273.73$6.41
  32. v13The agent widened the extra beam stage from M<=10 to M<=12273.73$6.51
  33. v13_selfcheckThe agent widened the extra beam stage from M<=10 to M<=12273.73$6.61
  34. v14The agent added one deeper DP cap on the best walk273.73$6.72
  35. v14_selfcheckThe agent added one deeper DP cap on the best walk273.73$6.82
  36. v15The agent appended a final stage with extended shaped variants and a new climb274.26441 min · $6.92
  37. v15_selfcheckThe agent appended a final stage with extended shaped variants and a new climb274.26$6.97

On the hidden set

Original metricNormalised score
Starter0.30220.0
Frontier-calibrated reference1.00.6
Upper1.51.0
This run (DeepSeek V4 Pro)0.6969690.4190
192 minWall clock
$18.72Spend
65.5MTokens
10Versions, 7 kept

On the visible set

0 100 200 300 400 500 600 700 0 2 4 6 8 Agent step Visible mean button count ↓ v1 v2 v3 v4 v5 v6 v8 v9
keptrolled backsubmitted
  1. v1The agent inherited the index-order greedy tour with substring macro compression416.930 min · $1.67
  2. v2The agent routed each ball by BFS over cells and facings using macros264.639 min · $2.20
  3. v3The agent optimized the ball order and enlarged the macro shape library232.847 min · $2.89
  4. v4The agent added facing-aware DP chaining and a two-phase macro, then reverted234$4.35
  5. v4-v6The agent deployed the facing-DP variant for a full tuning runno number logged$5.81
  6. v5The agent added slide-combo shapes and an annealing polish that found nothing235.1$7.26
  7. v6The agent added exact subset-DP ordering and per-ball macro switching23198 min · $8.72
  8. v7The agent made the parallel exploration anytime after fork storms wrecked runtimestune mean spiked 288152 min · $15.37
  9. v8The agent pruned worker shapes and added early-exit BFS routing222.3176 min · $17.69
  10. v9The agent cross-pollinated orders across shapes when the machine was idle222.3187 min · $18.51

On the hidden set

Original metricNormalised score
Starter0.30220.0
Frontier-calibrated reference1.00.6
Upper1.51.0
This run (Qwen3.8 Max)0.8176230.4990
83 minWall clock
$4.25Spend
11.1MTokens
3Versions, 2 kept

On the visible set

0 100 200 300 400 500 600 700 0 15 30 45 60 Agent step Visible mean button count ↓ v0 v1 v2
keptrolled backsubmitted
  1. v0The agent inherited the shipped greedy baseline517.556 min · $2.90
  2. v1The agent added a TSP-style tour, facing DP and a beam macro synthesizer245.957 min · $3.04
  3. v2The agent fixed the compressor crash and selected candidates per case276.882 min · $4.20

On the hidden set

Original metricNormalised score
Starter0.30220.0
Frontier-calibrated reference1.00.6
Upper1.51.0
This run (GLM 5.3)0.6834610.4092
97 minWall clock
$10.18Spend
14.1MTokens
19Versions, 14 kept

On the visible set

0 100 200 300 400 500 600 700 0 40 80 120 160 Agent step Visible mean button count ↓ v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18
keptrolled backsubmitted
  1. v0The agent inherited the greedy index-order baseline without macros651.422 min · $0.27
  2. v1The agent added orientation-aware shortest paths and local search on job order453.834 min · $0.69
  3. v2The agent compressed the routed plan with the best repeated substring macro290.117 min · $0.97
  4. v3The agent ran a bounded DP allowing several macro redefinitions286.4511 min · $1.43
  5. v4The agent let a new macro recording append the previous macro281.9215 min · $2.06
  6. v5The agent added an insertion neighborhood to the job-order local search278.816 min · $2.35
  7. v6The agent added an exact job-order DP for small instances277.2818 min · $2.65
  8. v7The agent raised the exact job-order DP threshold to M<=15276.9721 min · $3.17
  9. v8The agent raised the exact job-order DP threshold to M<=16276.9523 min · $3.56
  10. v9The agent flipped the Dijkstra tie-break to prefer R before L267.124 min · $3.77
  11. v10The agent also compressed the top raw greedy orders and kept the best276.7128 min · $4.37
  12. v11The agent doubled the compressed candidate orders and paid for it in runtime263.1832 min · $4.89
  13. v12The agent capped the compressed candidate orders back at five276.6832 min · $5.00
  14. v13The agent tried row-major and snake spatial order candidates263.1834 min · $5.25
  15. v14The agent reran the route local search from the three best greedy seeds275.1240 min · $6.05
  16. v15The agent widened the local-search seeds from three to five274.2245 min · $6.57
  17. v16The agent widened the local-search seeds from five to ten273.461 min · $7.91
  18. v17The agent widened the seeds to fifteen but judged the runtime margin too thin272.9280 min · $8.92
  19. v18The agent kept fifteen search seeds but only ten direct compression candidates272.9296 min · $9.86

On the hidden set

Original metricNormalised score
Starter0.30220.0
Frontier-calibrated reference1.00.6
Upper1.51.0
This run (GPT-5.5)0.6965130.4187

Leaderboard

Where each run landed on the sealed held-out set, on the same normalised-score scale as the anchors above.

0 0.3 0.6 1.0 1 GPT-5.6-sol codex · max 0.572 2 Opus 5 claude code · max 0.548 3 Qwen3.8 Max qwen coder · xhigh 0.499 4 Gemini 3.7 Flash antigravity · high 0.486 5 Kimi K3 kimi cli · max 0.450 6 Grok 4.6 grok · xhigh 0.421 7 DeepSeek V4 Pro claude code · max 0.419 8 GPT-5.5 codex · xhigh 0.419 9 GLM 5.3 claude code · max 0.409