Tasks/Optimization, Planning & Control

Query-Budgeted Stochastic Inventory Policy Optimization

Minimize simulated inventory cost by tuning four replenishment controls within 48 noisy queries

Optimization, Planning & Controlsimulation optimizationinventory control
Background

Simulation optimization designs search algorithms for systems whose objective can only be sampled, noisily, from a stochastic simulator, where the standing difficulty is doing well within a few dozen costly runs. The starting point is a uniform-random search routine that proposes four normalized controls of a replenishment policy and receives noisy cost feedback. The work is to redesign that routine into a query-efficient search. The result must hold across unseen demand and cost regimes, where noise alone can make a mediocre setting look best.

instruction.mdthis is what the agent is given

Design a reproducible optimizer for four normalized controls of a replenishment policy under stochastic demand and lead times. Minimize simulated operating cost within 48 queries over [0, 1]^4. You inherit a uniform-random starter and receive noisy objective feedback through a bounded ask/tell interface.

Hard Constraints

  • Edit only /app/methods/main/; solver.py must define Optimizer.
  • Use only the Python standard library and NumPy 2.2.6.
  • The constructor is Optimizer(dim, lower, upper, budget, seed, rng).
  • ask(n) must return a finite matrix with 1 through n rows, exactly dim columns, and every coordinate inside the supplied bounds.
  • The trusted evaluator owns the simulator, independent rescoring, query counter, and timeout. Extra returned points do not increase the query budget.
  • Import failures, crashes, malformed output, non-finite values, and out-of-bounds proposals invalidate the submission. If the aggregate runtime expires, completed valid work is retained under the published partial-work rule.
  • The submitted process cannot read or modify trusted evaluator assets and has no verifier network access.

The sealed evaluator runs many short optimizer sessions under one aggregate runtime boundary. Keep each ask and tell bounded and vectorized; expensive dense refits or very large candidate scans at every query can exhaust that shared budget.

What You Have

  • /app/data/visible.json contains public development cases spanning the same inventory-policy regimes as evaluation.
  • /app/data/visible_anchors.json contains public calibration traces used by the matched visible scorer.
  • /app/methods/main/solver.py is the uniform-random weak starter.
  • /app/selfcheck.py runs the same query, independent-rescoring, aggregation, and mapping semantics on public cases.
  • The four coordinates encode reorder level, order-up-to level, expedite threshold, and smoothing, each normalized to [0, 1].
  • The supplied rng is np.random.default_rng(seed) and should drive optimizer randomness. Simulator noise is controlled by the trusted evaluator.
  • The evaluator calls ask(n), evaluates the returned controls, and calls tell(X, y) or tell(X, y, metadata) when accepted. Values in y are costs, so lower is better.
  • A positive integer self.batch may request a preferred batch size; the evaluator caps it to the remaining budget.

What You Submit

Submit general optimizer code, not a final control vector or precomputed case answers. The submission must be self-contained in /app/methods/main/solver.py; sibling modules are not copied to the trusted verifier.

How It Is Judged

The trusted parent re-runs the optimizer on distinct cases with the same 48-query interface used publicly. When an observed incumbent improves, it is independently re-simulated with a candidate-keyed random stream. Best-so-far independent traces are summarized across repeated runs, with both sustained progress and final quality contributing to the metric. Better cost reduction across the full case mixture is better. Calibration assets and the exact leaderboard transformation remain verifier-only.

Rollouts

83 minWall clock
$28.16Spend
40.5MTokens
61Versions, 11 kept

On the visible set

0 0.075 0.150 0.225 0.300 0 75 150 225 300 Agent step Visible mapped reward ↑ v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v32 v33 v34 v37 v38 v39 v41 v42 v43 v44 v45 v47 v48 v49 v50 v51 v52 v53 v54 v55 v57 v58 v59 v60 v61
keptrolled backsubmitted
  1. v0The agent started from the shipped uniform-random baseline01 min · $0.24
  2. v1The agent swapped random sampling for an even space-filling sweep0.0869275 min · $0.65
  3. v2The agent opened with hand-built policies spanning the four regimes0.2981026 min · $0.90
  4. v3The agent tried mutating its best observations, and one case got all the gain0.2981768 min · $1.18
  5. v4The agent tried a Gaussian process, and it bought little for double the runtime0.29814510 min · $1.48
  6. v5The agent tried a systematic axis sweep, and one lean case collapsed0.28403114 min · $2.13
  7. v6The agent read early costs to tell lean cases from service cases0.30336416 min · $2.42
  8. v7The agent tried better-looking probes, and noise kept them off the record0.29710418 min · $2.78
  9. v8The agent shortened its probing and spent the rest refining near its best0.30355120 min · $3.10
  10. v9The agent tried jittering only the expedite knob, and all-axis moves won0.30354422 min · $3.46
  11. v10The agent tried refining only its single best point, and reserved cases fell0.3039124 min · $3.90
  12. v11The agent tried halving its step size, and the moves became redundant0.30377424 min · $4.06
  13. v12The agent doubled its step size and the gain repeated across noise panels0.30392825 min · $4.33
  14. v13The agent tried stepping even wider, and it degraded its own best policies0.30373126 min · $4.53
  15. v14The agent tried cutting the lean branch short, and one case turned unstable0.30367427 min · $4.76
  16. v15The agent tried probing longer before refining, and the balance got worse0.30370927 min · $4.98
  17. v16The agent routed on the whole cost trend instead of just its endpoints0.30399230 min · $5.48
  18. v17The agent tried keeping two parents, and the third proved a useful hedge0.30392930 min · $5.71
  19. v18The agent tried flattening its parent weights, and the gain did not repeat0.30383631 min · $6.01
  20. v19The agent tried steering moves per regime, and one panel collapsed0.30399633 min · $6.33
  21. v21The agent tried searching in policy coordinates, and lost local diversity0.30383334 min · $6.86
  22. v22The agent tried adding a stronger opening policy, and record timing broke0.29568336 min · $7.22
  23. v23The agent tried replacing its first probe, and that probe was essential0.30297936 min · $7.48
  24. v24The agent tried averaging with neighbours to fight the winner's curse0.30383338 min · $8.25
  25. v25The agent tried forcing its parents apart, and one basin can deserve several0.30390338 min · $8.51
  26. v26The agent tried mixing several step scales, and one fixed scale was steadier0.30368739 min · $8.78
  27. v27The agent stopped shrinking its step late and kept refining to the end0.30400540 min · $9.12
  28. v28The agent tried a smaller fixed step, and it clearly lost0.3037541 min · $9.38
  29. v29The agent tried a larger fixed step, and it also lost0.30390341 min · $9.65
  30. v30The agent added a second lean policy near the holding optimum0.3040342 min · $10.10
  31. v31The agent tried reordering the lean probes, and the old order hedged betteron a subset: 0.02410644 min · $10.79
  32. v32The agent tried a simpler routing rule, and it lost narrowly but always0.30410746 min · $11.40
  33. v33The agent tried zeroing the expedite probe, and lost a needed contrast0.29726946 min · $11.63
  34. v34The agent retested single-parent refinement, and hedging still won0.30385647 min · $11.95
  35. v35The agent tried a fourth parent, and three kept the better balanceon a subset: 0.0243448 min · $12.33
  36. v36The agent tried broad parents early and narrow late, and validation disagreedon a subset: 0.02440349 min · $12.81
  37. v37The agent tried mirrored pairs of moves, and updating each query was better0.30397150 min · $13.13
  38. v38The agent tried moving one control at a time, and joint moves were stabler0.30292951 min · $13.53
  39. v39The agent tried cautious routing, and lost more lean cases than it saved0.30386253 min · $14.12
  40. v40The agent tried reordering only for confident lean cases, and gradual wonon a subset: 0.02417555 min · $14.68
  41. v41The agent tried interleaving near-duplicate resamples, and lost refinements0.30386555 min · $15.02
  42. v42The agent tried exploiting its best parent harder, and the old mix was safer0.30405757 min · $15.54
  43. v43The agent tried steering along elite differences, and random moves held up0.30396357 min · $15.89
  44. v44The agent tried a fast path for expensive-looking cases, and it fired too often0.30330860 min · $16.73
  45. v45The agent tightened that fast path, and the reserved cases still lost0.30404561 min · $17.96
  46. v46The agent tightened the fast path once more, then dropped the branchon a subset: 0.02432762 min · $18.42
  47. v47The agent tried a third branch for service-like cases, and routing noise hurt0.30376863 min · $19.06
  48. v48The agent tried tiny jitter to farm extra scoring draws, and quality collapsed0.29438365 min · $20.16
  49. v49The agent tried refining the centroid of its best three, and blurred them0.30389367 min · $20.70
  50. v50The agent tried swapping probes for costly-looking sessions, and it did not last0.30392468 min · $21.19
  51. v51The agent tried adding service variants mid-run, and they crowded out refining0.303969 min · $21.60
  52. v52The agent tried pinning proposals to the box edges, and inward moves mattered0.30385670 min · $22.01
  53. v53The agent tried uniform instead of Gaussian moves, and missed the tails0.30394970 min · $22.41
  54. v54The agent tried a smaller step for lean cases, and they liked the shared one0.30392471 min · $22.82
  55. v55The agent tried a larger step for lean cases, and stayed below the incumbent0.30397772 min · $23.22
  56. v56The agent tuned the lean-only step once more, and the shared step still wonon a subset: 0.02426573 min · $23.74
  57. v57The agent tried a larger step for non-lean cases, and it also lost0.30384173 min · $24.15
  58. v58The agent swept its routing thresholds, and the winner failed fresh panels0.30407677 min · $25.14
  59. v59The agent tried following the direction of past wins, and noise gave none0.30384778 min · $25.62
  60. v60The agent asked for its fixed probes in one batch to cut round trips0.30366779 min · $26.51
  61. v61The agent batched the branch probes too and fuzz-tested the whole interface0.30366780 min · $26.96

On the hidden set

CaseStarter · 0.0Reference · 0.6Upper · 1.0This run (GPT-5.6-sol)
Case 010.0000000.0400521.0000000.046930
Case 020.0000000.0414451.0000000.050016
Case 030.0000000.0089481.0000000.008686
Case 040.0000000.0092601.0000000.028073
Case 050.0000000.0260321.0000000.046464
Case 060.0000000.0200901.0000000.029066
Case 070.0000000.0311891.0000000.028998
Case 080.0000000.0202161.0000000.016136
Case 090.0000000.0185091.0000000.014298
Case 100.0000000.0269021.0000000.029309
Case 110.0000000.0161851.0000000.024458
Case 120.0000000.0165101.0000000.015960
Case 130.0000000.0290491.0000000.034305
Case 140.0000000.0173051.0000000.025874
Case 150.0000000.0071591.0000000.015928
Case 160.0000000.0112581.0000000.007920
Normalised score0.5484
376 minWall clock
$41.46Spend
67.0MTokens
8Versions, 7 kept

On the visible set

0 0.075 0.150 0.225 0.300 0 2 3 4 6 Agent step Visible mapped score ↑ v0 v1 v2 v3 v4 v5 v6 v7
keptrolled backsubmitted
  1. v0The agent inherited the shipped uniform-random search baseline034 min · $3.25
  2. v1The agent added a kernel-ridge surrogate with trust region and informed design0.2788235 min · $3.43
  3. v2The agent shortened the informed design to six best-prior-first points0.28934 min · $3.25
  4. v3The agent cut the design to three points and retuned the trust region0.30356 min · $5.79
  5. v4The agent rewrote the surrogate as a low-rank family basis0.2944141 min · $16.93
  6. v5The agent shortened screening and switched to a lower-confidence-bound proposal0.2998200 min · $21.71
  7. v6The agent replaced the Gaussian posterior with a nonparametric particle cloud0.3022253 min · $27.11
  8. v7The agent rebuilt the basis and cloud from 4200 broader instances0.3025375 min · $41.23

On the hidden set

CaseStarter · 0.0Reference · 0.6Upper · 1.0This run (Opus 5)
Case 010.0000000.0400521.0000000.051208
Case 020.0000000.0414451.0000000.032326
Case 030.0000000.0089481.0000000.011652
Case 040.0000000.0092601.0000000.028254
Case 050.0000000.0260321.0000000.040151
Case 060.0000000.0200901.0000000.027992
Case 070.0000000.0311891.0000000.031041
Case 080.0000000.0202161.0000000.012016
Case 090.0000000.0185091.0000000.024271
Case 100.0000000.0269021.0000000.023772
Case 110.0000000.0161851.0000000.021966
Case 120.0000000.0165101.0000000.015480
Case 130.0000000.0290491.0000000.034983
Case 140.0000000.0173051.0000000.030843
Case 150.0000000.0071591.0000000.023293
Case 160.0000000.0112581.0000000.016949
Normalised score0.5544
15 minWall clock
$1.46Spend
6.1MTokens
5Versions, 4 kept

On the visible set

0 0.2 0.4 0.6 0.8 1.0 0 1 2 3 4 Agent step Visible mapped reward ↑ v0 v1 v2 v3 v4
keptrolled backsubmitted
  1. v0The agent inherited the shipped uniform-random search baseline0
  2. v1The agent fitted a basic GP with fixed lengthscales and LCB0.125176
  3. v2The agent switched to TuRBO-GP with rank warping and simplex polishing0.902636
  4. v3The agent added multi-band trust regions and quadratic refinement0.90656
  5. v4The agent added a dual incumbent and batched ask support0.907474

On the hidden set

CaseStarter · 0.0Reference · 0.6Upper · 1.0This run (Gemini 3.7 Flash)
Case 010.0000000.0400521.0000000.045184
Case 020.0000000.0414451.0000000.022160
Case 030.0000000.0089481.0000000.012565
Case 040.0000000.0092601.0000000.002686
Case 050.0000000.0260321.0000000.006561
Case 060.0000000.0200901.0000000.003574
Case 070.0000000.0311891.0000000.017922
Case 080.0000000.0202161.0000000.000000
Case 090.0000000.0185091.0000000.000866
Case 100.0000000.0269021.0000000.011020
Case 110.0000000.0161851.0000000.017712
Case 120.0000000.0165101.0000000.000046
Case 130.0000000.0290491.0000000.012422
Case 140.0000000.0173051.0000000.006438
Case 150.0000000.0071591.0000000.000000
Case 160.0000000.0112581.0000000.000000
Normalised score0.1883
351 minWall clock
$21.91Spend
54.6MTokens
11Versions, 10 kept

On the visible set

0 0.075 0.150 0.225 0.300 0 2 5 8 10 Agent step Selfcheck visible mapped reward ↑ v0 v1 v2 v3 v4 v5 v6 v7 v8 v9
keptrolled backsubmitted
  1. v0The agent inherited the shipped uniform-random search baseline0$0.54
  2. v1The agent built a GP-BO with Latin-hypercube init and expected improvement0.199$1.08
  3. v2The agent dropped output standardization so the GP interpolated raw costs0.234$2.61
  4. v3The agent added an inhibition radius and multi-scale local candidates0.244$3.58
  5. v4The agent replaced the init with six regime-spanning warm-start points0.279$4.16
  6. v4_tmpThe agent snapshotted the wrong code as v4_tmp and deleted itmis-snapshot, deleted$4.69
  7. v5The agent made candidates local-heavy and lowered the assumed noise0.287$5.22
  8. v6The agent widened the random candidate pool and raised the lengthscale0.2972$6.91
  9. v7The agent rewrote the solver cleanly on a Matern-3/2 kernel0.2966$8.65
  10. v8The agent lowered inhibition and lengthened the final re-evaluation tail0.2945$9.81
  11. v9The agent added a bounds-safety clip to its proposals0.2945$16.61

On the hidden set

CaseStarter · 0.0Reference · 0.6Upper · 1.0This run (Kimi K3)
Case 010.0000000.0400521.0000000.062841
Case 020.0000000.0414451.0000000.030342
Case 030.0000000.0089481.0000000.008403
Case 040.0000000.0092601.0000000.027588
Case 050.0000000.0260321.0000000.040941
Case 060.0000000.0200901.0000000.021597
Case 070.0000000.0311891.0000000.030523
Case 080.0000000.0202161.0000000.013728
Case 090.0000000.0185091.0000000.018959
Case 100.0000000.0269021.0000000.026911
Case 110.0000000.0161851.0000000.021426
Case 120.0000000.0165101.0000000.017365
Case 130.0000000.0290491.0000000.035927
Case 140.0000000.0173051.0000000.023651
Case 150.0000000.0071591.0000000.015185
Case 160.0000000.0112581.0000000.014535
Normalised score0.5618
38 minWall clock
$10.33Spend
15.4MTokens
20Versions, 8 kept

On the visible set

0 0.075 0.150 0.225 0.300 0 4 8 12 16 Agent step Visible mapped reward (early ver ↑ v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19
keptrolled backsubmitted
  1. v0The agent inherited the shipped uniform-random search baseline0
  2. v1The agent added a designed (s,S) prototype start with compass and GP-LCB0.02336
  3. v2The agent lengthened the design and added confirmation re-evals and line searches0.0229
  4. v3The agent added mean-best repeats and a frozen-axis ridge refinement0.02339
  5. v4The agent widened the design to twenty anytime ridge-cover points0.02394
  6. v5The agent moved holding and service prototypes earlier and searched on-face0.02572
  7. v6The agent replaced the lucky first point with a true-mean holding policy0.292
  8. v7The agent probed more early main-face points and drew unlucky scores0.287
  9. v8The agent swapped in new prototypes and lost the holding complement0.295
  10. v9The agent kept the v5 prefix and added four complements plus face fill0.30471
  11. v10The agent capped the plan at 24 points and made the grid adaptive0.30468
  12. v11The agent mixed all three faces into the adaptive grid0.30465
  13. v12The agent re-queried the top five mean-best unique points0.30463
  14. v13The agent swapped one holding plan point for a truly better one0.30463
  15. v14The agent swapped a service plan point so a lucky miss cannot block0.30503
  16. v15The agent also swapped a second plan slot for no gain0.30503
  17. v16The agent replaced a duplicate extra point without touching the prefix0.30506
  18. v17The agent replaced the golden face fill with GP-LCB candidates0.30505
  19. v18The agent swapped two late plan slots for no gain0.30506
  20. v19The agent jittered the late face fill0.30499

On the hidden set

CaseStarter · 0.0Reference · 0.6Upper · 1.0This run (Grok 4.6)
Case 010.0000000.0400521.0000000.046317
Case 020.0000000.0414451.0000000.051758
Case 030.0000000.0089481.0000000.016195
Case 040.0000000.0092601.0000000.020208
Case 050.0000000.0260321.0000000.029560
Case 060.0000000.0200901.0000000.028154
Case 070.0000000.0311891.0000000.030137
Case 080.0000000.0202161.0000000.016462
Case 090.0000000.0185091.0000000.024936
Case 100.0000000.0269021.0000000.027918
Case 110.0000000.0161851.0000000.016650
Case 120.0000000.0165101.0000000.023480
Case 130.0000000.0290491.0000000.030933
Case 140.0000000.0173051.0000000.016726
Case 150.0000000.0071591.0000000.025000
Case 160.0000000.0112581.0000000.002537
Normalised score0.5524
69 minWall clock
$1.32Spend
11.9MTokens
15Versions, 9 kept

On the visible set

0 0.075 0.150 0.225 0.300 0 3 6 9 12 Agent step Visible mapped score ↑ v0 v1 v2 v3 v4 v5 v6 v7 v7_aniso v7_threec v7_twoc v7_twoc sweep v7_twocw v8 v8-v10
keptrolled backsubmitted
  1. v0The agent inherited the shipped uniform-random search baseline0$0.33
  2. v1The agent added an informed design with a quadratic surrogate0.193540 min · $0.66
  3. v2The agent switched to a two-basin design with Gaussian sampling0.2692$0.72
  4. v3The agent learned the design greedily from observation means over synthetic cases0.2655$0.78
  5. v4The agent added a quadratic-surrogate center and a final re-evaluation phase0.2672$0.83
  6. v5The agent centered sampling on exponentially weighted means with shrinking sigma0.2851$0.89
  7. v6The agent embedded a ridge predictor trained on synthetic instances0.2829$0.94
  8. v7The agent split sampling into two basin centers with softmax allocation0.29758 min · $1.00
  9. v7_anisoThe agent widened one axis around a single noisy center0.2787$1.04
  10. v7_threecThe agent split the search into three basin centers0.2965$1.08
  11. v7_twocThe agent introduced per-basin argmin centers with anisotropic sigma0.2928$1.12
  12. v7_twoc sweepThe agent swept the basin allocation temperature and picked 1000.2961$1.16
  13. v7_twocwThe agent weighted the basin centers by region instead of argmin0.2874$1.20
  14. v8The agent added a tight final sampling phase and robustness guards0.29700867 min · $1.25
  15. v8-v10The agent tried merges, re-evals and pooled centers, all plateauing0.277$1.28

On the hidden set

CaseStarter · 0.0Reference · 0.6Upper · 1.0This run (DeepSeek V4 Pro)
Case 010.0000000.0400521.0000000.049147
Case 020.0000000.0414451.0000000.050940
Case 030.0000000.0089481.0000000.017324
Case 040.0000000.0092601.0000000.029638
Case 050.0000000.0260321.0000000.042477
Case 060.0000000.0200901.0000000.025403
Case 070.0000000.0311891.0000000.025441
Case 080.0000000.0202161.0000000.017100
Case 090.0000000.0185091.0000000.024357
Case 100.0000000.0269021.0000000.043293
Case 110.0000000.0161851.0000000.017453
Case 120.0000000.0165101.0000000.013942
Case 130.0000000.0290491.0000000.029090
Case 140.0000000.0173051.0000000.028130
Case 150.0000000.0071591.0000000.017049
Case 160.0000000.0112581.0000000.012194
Normalised score0.5704
107 minWall clock
$7.57Spend
24.1MTokens
25Versions, 9 kept

On the visible set

0 0.075 0.150 0.225 0.300 0 5 10 15 20 Agent step Selfcheck visible mapped score ↑ v1 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 A1 ARD-lite B1 C1 M1 T1 T2 T4
keptrolled backsubmitted
  1. v1The agent inherited the shipped uniform-random search baseline03 min · $0.15
  2. v2-v4The agent iterated GP-BO builds on five-case subsets without snapshotting5-case subsets only$1.04
  3. v5The agent consolidated the GP-BO with a fixed eight-point designsubset 0.0143-0.049435 min · $1.92
  4. v6The agent redesigned toward interior expedite optima and added epsilon revisitssubset 0.0098-0.0541$2.27
  5. v7The agent added farthest-point basin spreading and an axis grid0.285352$2.61
  6. v8The agent rewrote the design around basins and dropped epsilon revisits0.29309249 min · $2.96
  7. v9The agent band-limited expected improvement and widened the spreading masks0.30260454 min · $3.27
  8. v10The agent weighted spreading by entry probability with wider masks0.29828$3.72
  9. v11The agent densified the axis grid and added pairwise face candidates0.29733$4.17
  10. v12The agent spent twelve percent of budget revisiting never-entered points0.29752$4.62
  11. v13The agent added descent chains of tight queries after each entry0.29518$5.06
  12. v14The agent gave late queries unbounded expected improvement to find new basins0.2999780 min · $5.51
  13. v15The agent hardened the solver against degenerate and adversarial inputs0.2992782 min · $5.64
  14. v16The agent gave promising unentered design points a second chance0.3041290 min · $6.30
  15. v17The agent raised the design retry cap from two to three0.2970192 min · $6.37
  16. v18The agent raised the retry cap to four0.29813$6.97
  17. v19The agent halved the retry threshold0.023571$6.97
  18. A1The agent raised the spreading mask minimum count0.024083$6.97
  19. ARD-liteThe agent gave the smoothing axis its own lengthscale0.023985$6.97
  20. B1The agent lowered the spreading probability0.023961$6.97
  21. C1The agent made retries conditional on the observation gate being passable0.023354$6.97
  22. M1The agent widened the spreading masks0.023813$6.97
  23. T1The agent trimmed the design from ten points to eight0.29686$6.97
  24. T2The agent raised the spreading probability0.29469$6.97
  25. T4The agent refined the hyperparameter grid0.29473$6.97

On the hidden set

CaseStarter · 0.0Reference · 0.6Upper · 1.0This run (Qwen3.8 Max)
Case 010.0000000.0400521.0000000.047457
Case 020.0000000.0414451.0000000.039128
Case 030.0000000.0089481.0000000.016236
Case 040.0000000.0092601.0000000.020981
Case 050.0000000.0260321.0000000.022309
Case 060.0000000.0200901.0000000.020986
Case 070.0000000.0311891.0000000.031157
Case 080.0000000.0202161.0000000.018189
Case 090.0000000.0185091.0000000.022717
Case 100.0000000.0269021.0000000.026314
Case 110.0000000.0161851.0000000.022180
Case 120.0000000.0165101.0000000.019577
Case 130.0000000.0290491.0000000.022713
Case 140.0000000.0173051.0000000.040348
Case 150.0000000.0071591.0000000.014009
Case 160.0000000.0112581.0000000.010312
Normalised score0.5593
175 minWall clock
$8.29Spend
25.0MTokens
14Versions, 5 kept

On the visible set

0 0.075 0.150 0.225 0.300 0 3 6 9 12 Agent step Dev-set raw quality ↑ v1 v2 v3 v4 v6 v7/v6 variants v8 v9 v10 v11a/b/c v12 v13/v14/v15 v17
keptrolled backsubmitted
  1. v1The agent built an ARD-RBF GP with regime probes and decaying LCB0.2905236 min · $1.44
  2. v2The agent designed an eight-probe set offline and moved the GP to log-cost0.28693$2.13
  3. v3The agent added structured boundary-sweep candidates0.0229$2.82
  4. v4The agent fixed the GP hyperparameters and used a leave-one-out nugget0.0223$3.51
  5. v6The agent rewrote the kernels as matmuls and cached the posterior0.02308$4.19
  6. v7/v6 variantsThe agent swept lengthscale, probe count, kappa and trust-region scales0.0227$4.88
  7. v8The agent hardened the solver for odd dimensions and non-finite observations0.29133$5.57
  8. v9The agent re-optimised the probe set for the anytime objective0.02362$6.26
  9. v10The agent re-optimised the probe order for the anytime objective0.28016147 min · $6.95
  10. v11a/b/cThe agent tried alternative probe orderings0.288$7.62
  11. v12The agent forced late-phase spatial spread0.29174$7.62
  12. v13/v14/v15The agent traded expedite probes for denser order-up-to coverage0.298$7.62
  13. v17The agent cut candidates and perturbations for speed0.29195$7.62
  14. kappa/ELL/NPROBE sweepsThe agent swept kappa, lengthscale and probe count within noiseall within noise$7.62

On the hidden set

CaseStarter · 0.0Reference · 0.6Upper · 1.0This run (GLM 5.3)
Case 010.0000000.0400521.0000000.046003
Case 020.0000000.0414451.0000000.035698
Case 030.0000000.0089481.0000000.010724
Case 040.0000000.0092601.0000000.029638
Case 050.0000000.0260321.0000000.042307
Case 060.0000000.0200901.0000000.026435
Case 070.0000000.0311891.0000000.031051
Case 080.0000000.0202161.0000000.012487
Case 090.0000000.0185091.0000000.024357
Case 100.0000000.0269021.0000000.035939
Case 110.0000000.0161851.0000000.022180
Case 120.0000000.0165101.0000000.009223
Case 130.0000000.0290491.0000000.035606
Case 140.0000000.0173051.0000000.024414
Case 150.0000000.0071591.0000000.022879
Case 160.0000000.0112581.0000000.014173
Normalised score0.5494
28 minWall clock
$9.45Spend
13.1MTokens
34Versions, 9 kept

On the visible set

0 0.075 0.150 0.225 0.300 0 30 60 90 120 Agent step Public selfcheck visible score ↑ v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31 v32 v33
keptrolled backsubmitted
  1. v0The agent inherited the shipped uniform-random search baseline01 min · $0.17
  2. v1The agent added a domain-informed warm start with quadratic surrogate local search0.2821015 min · $0.58
  3. v2The agent moved service-mode warm-start candidates earlier0.2822756 min · $0.79
  4. v3The agent replaced the holding warm-start points with stronger canonical candidates0.291657 min · $1.03
  5. v4The agent reordered service warm starts so robust candidates precede noisier ones0.30468 min · $1.27
  6. v5The agent added broad service and long-lead warm-start candidates0.3049489 min · $1.58
  7. v6The agent added three long-lead probes after the early service block0.30467810 min · $1.79
  8. v7The agent moved a long-lead probe ahead of the broad service probes0.30466811 min · $1.97
  9. v8The agent swapped the order of the first two holding probes0.30523311 min · $2.13
  10. v9The agent replaced two late no-update probes with long-lead ones0.30505712 min · $2.36
  11. v10The agent branched on the first four observations to advance the holding tail0.30523313 min · $2.65
  12. v11The agent doubled the surrogate random scan and local samples0.30499114 min · $2.92
  13. v12The agent removed the surrogate distance exploration bonus0.30506114 min · $3.09
  14. v13The agent set the distance exploration bonus to 0.020.30504214 min · $3.26
  15. v14The agent set the distance exploration bonus to 0.080.3051915 min · $3.42
  16. v15The agent swapped the opening probe0.30516915 min · $3.60
  17. v16The agent pruned the warm-start tail so the surrogate starts earlier0.30520515 min · $3.85
  18. v17The agent moved a specialized service probe later to reduce blocking0.30509316 min · $4.05
  19. v18The agent replaced a late probe with a more service-oriented one0.30526817 min · $4.36
  20. v19The agent replaced another late probe0.30507317 min · $4.55
  21. v20The agent nudged the final late service candidate0.30526117 min · $4.74
  22. v21The agent replaced a late long-lead probe0.30514918 min · $5.03
  23. v22The agent moved the late service candidate into slot twelve0.30528819 min · $5.30
  24. v23The agent moved that candidate up to slot nine0.30492219 min · $5.51
  25. v24The agent moved that candidate one slot earlier0.30512419 min · $5.72
  26. v25The agent shrank the local and surrogate perturbation radii0.30519321 min · $6.24
  27. v26The agent weighted the quadratic surrogate fit by observed rank0.3053621 min · $6.48
  28. v27The agent strengthened the surrogate rank weighting0.30533222 min · $6.70
  29. v28The agent weakened the surrogate rank weighting0.30527222 min · $6.91
  30. v29The agent raised the distance bonus on the weighted surrogate0.30525223 min · $7.60
  31. v30The agent lowered the distance bonus on the weighted surrogate0.30540424 min · $7.83
  32. v31The agent removed the distance bonus entirely0.30543924 min · $8.07
  33. v32The agent doubled the scan and local samples again0.30527625 min · $8.31
  34. v33The agent shrank the scan and local samples0.30527425 min · $8.54

On the hidden set

CaseStarter · 0.0Reference · 0.6Upper · 1.0This run (GPT-5.5)
Case 010.0000000.0400521.0000000.051359
Case 020.0000000.0414451.0000000.050159
Case 030.0000000.0089481.0000000.009966
Case 040.0000000.0092601.0000000.020756
Case 050.0000000.0260321.0000000.042613
Case 060.0000000.0200901.0000000.026872
Case 070.0000000.0311891.0000000.031712
Case 080.0000000.0202161.0000000.010445
Case 090.0000000.0185091.0000000.017012
Case 100.0000000.0269021.0000000.015366
Case 110.0000000.0161851.0000000.017709
Case 120.0000000.0165101.0000000.018997
Case 130.0000000.0290491.0000000.024639
Case 140.0000000.0173051.0000000.024333
Case 150.0000000.0071591.0000000.022876
Case 160.0000000.0112581.0000000.016387
Normalised score0.5396

Leaderboard

Where each run landed on the sealed held-out set, on the same normalised-score scale as the anchors above.

0 0.3 0.6 1.0 1 DeepSeek V4 Pro claude code · max 0.570 2 Kimi K3 kimi cli · max 0.562 3 Qwen3.8 Max qwen coder · xhigh 0.559 4 Opus 5 claude code · max 0.554 5 Grok 4.6 grok · xhigh 0.552 6 GLM 5.3 claude code · max 0.549 7 GPT-5.6-sol codex · max 0.548 8 GPT-5.5 codex · xhigh 0.540 9 Gemini 3.7 Flash antigravity · high 0.188