Tasks/Systems & Hardware

Multi-Stream Gated FlashFFTConv Kernel Optimization

Accelerate gated causal long convolutions across independent streams and unseen shapes

Systems & HardwareGPU kernelslong convolution
Background

Inference-kernel engineering for long-convolution sequence models — architectures that swap attention for a fast Fourier transform convolution — is limited less by arithmetic than by data movement. The starting point is a plain PyTorch routine that promotes half-precision activations to single precision, transforms them, multiplies, inverts, and applies input and output gates in separate passes. The work is to redesign that forward path for many independent gated streams at once. Gains must hold on sealed shapes, gate distributions, and lengths without loosening numerical agreement.

instruction.mdthis is what the agent is given

Optimize multiple independent gated causal convolutions across the published long-convolution range. The final implementation is re-run on sealed stream counts, shapes, gate distributions, kernel scales, lengths, and seeds.

Hard Constraints

  • Edit only /app/methods/main/solver.py.
  • Keep flashfftconv_multistream_gated_forward(x, k, in_gate, out_gate, fft_size) unchanged.
  • Return one contiguous tensor with the same shape and dtype as x.
  • For every stream and head compute out_gate * causal_conv(x * in_gate, k), retain the first length positions, and use zero-padded linear convolution with fft_size = 2 * length.
  • Do not call external FFT-convolution packages, mutate inputs, inspect verifier paths, replace timers, cache answers by call order, or specialize on hidden identities.
  • The convolution is zero-padded causal, not circular.
  • Streams are independent: a kernel may not be shared or mixed across them.
  • Each gate applies on its documented side of the convolution.
  • The result keeps the input dtype; FP32 output is incorrect.

What You Have

  • x, in_gate, and out_gate are contiguous FP16 tensors [batch, streams, heads, length].
  • k is contiguous FP32 [streams, heads, length].
  • Visible cases span four batch sizes and multiple factorizations of the published FP16 768-channel, length-2048 regime; sealed cases use disjoint stream/head factorizations, gate distributions, and seeds from the same family.
  • Streams are independent channels, so reshaping [streams, heads] into a combined channel axis is mathematically valid.
  • The inherited baseline is a human-written PyTorch FP32 RFFT/IRFFT implementation with separate gates.

What You Submit

Submit /app/methods/main/ containing solver.py and local PyTorch or Triton helpers. Every invocation must compute the result for fresh inputs.

How It Is Judged

Each correct sealed case scores frozen eager PyTorch FFT latency / candidate median latency; reward is the geometric mean raw speedup.

Rollouts

≥152 minobserved wall
74Versions, 34 kept

On the visible set

0.75 1.50 2.25 3.00 3.75 0 20 40 60 Agent step Visible raw speedup ↑ v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31 v32 v33 v34 v35 v36 v37 v39 v40 v41 v42 v43 v44 v45 v46 v47 v48 v49 v50 v51 v52 v54 v55 v56 v57 v58 v59 v61 v65 v66 v67 v71 v72 v73 v76
keptrolled backsubmitted
  1. v0The agent started from the reference FP32 FFT convolution0.998x
  2. v1The agent moved the whole convolution to FP16 for cheaper transforms2.144x
  3. v2The agent let the compiler fuse the gates and casts around the FFT calls2.4374x
  4. v3The agent folded the inverse FFT normalization into the output gate2.7763x
  5. v4The agent tried real arithmetic on the Hermitian spectrum, and it lost ground1.9711x
  6. v5The agent tried reusing the input spectrum's memory in place, and gained nothing2.7769x
  7. v6The agent tried CUDA graphs, and the static input copies ate the gain2.2789x
  8. v7The agent tried plain real/imag math instead of half-complex, and it ran slower2.721x
  9. v8The agent tried a half-length real inverse, and the compiled version was too slow2.1691x
  10. v9The agent wrote one kernel that multiplies and repacks the spectrum in a pass3.0906x
  11. v10The agent tuned the packing kernel's launch shape for better occupancy3.1479x
  12. v11The agent cached the twiddle tables so they stop being rebuilt every call3.2302x
  13. v12The agent flattened the launch so huge batches stay inside the grid limits3.2354x
  14. v13The agent picked a wider launch shape for the mid-length transforms3.25774x
  15. v14The agent split the launch rule into two length regimes for unseen lengths3.2558x
  16. v15The agent tried sharing loads across two signals per program, and registers ran out3.0867x
  17. v16The agent tried transforming the kernel as an extra batch plane, and it did not pay3.2404x
  18. v17The agent tried dropping the widening to FP32 on loads, and it came out slower3.2359x
  19. v18The agent paired conjugate frequencies so each product is computed only once3.3045x
  20. v19The agent tidied the frequency tiling so no nearly empty block is launched3.3184x
  21. v20The agent retuned the pairwise kernel to one launch shape for every length3.41x
  22. v21The agent packed two real signals into one complex FFT to halve transform work3.6072x
  23. v22The agent tried folding the midpoints into one lane, and it only added pressure3.6045x
  24. v23The agent split even and odd batches so the packed path always has a fallback3.616x
  25. v24The agent tried max-autotune compilation, and it bought only compile time3.6077x
  26. v25The agent tried convolving the packed spectrum directly, and aliasing hurt3.4086x
  27. v26The agent tried an out-of-place version of that multiply, and allocation cost more3.3642x
  28. v27The agent tried its own input-preparation kernel, and launch overhead won3.4407x
  29. v28The agent tried a wider packer launch, and the isolated win did not hold3.6028x
  30. v29The agent tried narrowing precision in the recovery kernel, and deltas were mixed3.6095x
  31. v30The agent tried forcing one whole compiled graph, and it made no difference3.5967x
  32. v31The agent tried reordering the multiply to shorten register lifetimes, and it lost3.5967x
  33. v32The agent tried computing only the causal half of the inverse, and rebuilding cost more3.2817x
  34. v33The agent tried collecting the leftover midpoints in one launch, and it cost more3.5571x
  35. v34The agent tried reordering blocks for cache reuse, and strided access undid it3.5571x
  36. v35The agent tried running both transforms in place, and the effect stayed in noise3.6074x
  37. v36The agent tried the in-place trick on the inverse alone, and no gain held3.6099x
  38. v37The agent tried it on the forward alone, and the out-of-place plan stayed faster3.5965x
  39. v39The agent tried giving each block four signals to share loads, and occupancy fell3.5766x
  40. v40The agent tried shrinking that block to restore occupancy, and it still lagged3.5618x
  41. v41The agent tried overlapping the two FFTs on a second stream, and boundaries cost more3.3983x
  42. v42The agent tried capturing the whole overlap in one graph, and contention erased it3.6047x
  43. v43The agent tried storing the twiddle tables in FP16, and conversion cost more3.5917x
  44. v44The agent tried deriving a pair's second output by subtraction, and the chain grew3.6039x
  45. v45The agent tried single-stage scheduling in the packers, and gained nothing3.603x
  46. v46The agent tried two-stage scheduling instead, and the default stayed best3.6054x
  47. v47The agent capped the packer's registers for occupancy, a gain near the noise floor3.6108x
  48. v48The agent tried a tighter register cap, and it helped nowhere3.6068x
  49. v49The agent tried a looser register cap, and it failed to hold across batches3.6122x
  50. v50The agent tried keeping the complex products in FP16, and conversion ate the gain3.5965x
  51. v51The agent tried native packed-half instructions, and the memory pipeline went hungry3.5059x
  52. v52The agent tried rebalancing that packed kernel's warps, and it only tied3.608x
  53. v54The agent tried the 2D launch again where it fits, and it did not pay off3.6019x
  54. v55The agent packed complex halves into single words so every access is aligned3.6865x
  55. v56The agent tried hand-written pack instructions, and the compiler's were as good3.6867x
  56. v57The agent chose the packer launch shape by length, a small win everywhere3.684x
  57. v58The agent gave the odd-batch fallback the same wide aligned memory access3.6907x
  58. v59The agent tuned the odd-batch launch per length, helping all but batch one3.6805x
  59. v60The agent kept the old launch for single-batch calls to undo that regression+0.76% (B1/L2048)
  60. v61The agent replaced the compiled output epilogue with its own wide-access kernel3.6884x
  61. v62The agent tried splitting the output launch by length, and the edge did not hold−0.027% geomean
  62. v63The agent tried bypassing L1 for streaming loads, and the effect stayed mixed+0.067% geomean
  63. v65The agent tried a packed-half output gate, and it changed almost nothing3.6968x
  64. v66The agent tried folding the inverse scale into the spectrum, and bandwidth still ruled3.6936x
  65. v67The agent paired adjacent channels so odd batches also get the packed transform3.6926x
  66. v68The agent retuned that kernel and mapped where channel pairing beats the plain route+0.002% (B33)
  67. v69The agent gated channel pairing to the shapes where it measurably wins+6.49% (L4096/B3)
  68. v70The agent set the channel-pair launch by length, keeping only the short-length part+3.185% (L512/B33)
  69. v71The agent restored the proven long-length launch and kept the short-length gain3.7x
  70. v72The agent tried channel pairing on the even visible cases, and short lengths won3.7149x
  71. v73The agent routed by batch parity and length so every shape takes its best path3.7124x
  72. v74The agent tried channel pairing for small batches at length 4096, and one regressed+0.25% (L4096/B4)
  73. v75The agent narrowed that exception to the batches that really gained+0.25% (L4096/B4)
  74. v76The agent folded the kernel transform into the same batched FFT, its biggest win3.7835x

On the hidden set

Original metricNormalised score
Starter1.00x0.00
Frontier-calibrated reference8.08x0.60
Upper21.36x1.00
This run (GPT-5.6-sol)4.09480.6083
184 minWall clock
$43.08Spend
108.3MTokens
10Versions, 8 kept

On the visible set

0 2.5 5.0 7.5 10.0 0 2 4 6 8 Agent step Visible raw speedup ↑ v0 v1 v2 v3 v4 v5 v6 v7 v8
keptrolled backsubmitted
  1. v0The agent inherited a PyTorch RFFT gated convolution with hand-written autograd1
  2. v1The agent switched the cuFFT transforms from FP32 to FP162.15
  3. v2The agent fused the whole op into one Triton four-step Monarch kernel2.08
  4. v3The agent switched the tensor-core dots to FP16 accumulators, killing register spills6.99
  5. v4The agent prefetched the next sequence and marked streaming loads evict_first7.29
  6. v5The agent tuned maxnreg to the largest value fitting three CTAs per SM8.44
  7. v6The agent built the kernel spectrum in registers under a channel-major grid8.74
  8. v7The agent exploited Hermitian symmetry to halve both dominant matmul stages10.07
  9. v8The agent moved the batch loop inside one block per channel11.05
  10. ideaThe agent tried six further ideas and rejected every oneall six rejected

On the hidden set

Original metricNormalised score
Starter1.00x0.00
Frontier-calibrated reference8.08x0.60
Upper21.36x1.00
This run (Opus 5)12.33770.7819
-Wall clock
$1.56Spend
6.3MTokens
1Versions, 0 kept

Only one comparable self-check score, so no curve — the versions below list what each one changed.

  1. v0The agent inherited a PyTorch RFFT gated convolution with hand-written autograd1

On the hidden set

Original metricNormalised score
Starter1.00x0.00
Frontier-calibrated reference8.08x0.60
Upper21.36x1.00
This run (Gemini 3.7 Flash)4.86120.6245
<240 minWall clock
-Spend
-Tokens
13Versions, 12 kept

On the visible set

1.5 3.0 4.5 6.0 7.5 0 2 5 8 10 Agent step Visible raw speedup ↑ v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12
keptrolled backsubmitted
  1. v0The agent inherited a PyTorch RFFT gated convolution with hand-written autograd1
  2. v1The agent moved the whole eager path to FP16 cuFFT2.147
  3. v2The agent replaced cuFFT with a fused four-step matmul FFT kernel3.719
  4. v3The agent trimmed registers with a half-K first stage and FP16 twiddles5.197
  5. v4The agent switched all twelve dots to FP16 tensor-core accumulators5.766
  6. v5The agent reordered programs channel-major and split the conjugate twiddle tables5.795
  7. v6The agent gave each program a serial eight-row loop reusing register tables6.803
  8. v7The agent computed the kernel spectrum in its own Triton kernel7.487
  9. v8The agent folded per-channel kernel normalization into the spectrum kernel7.544
  10. v9The agent fused the norm in-kernel and split the scale for subnormal safety7.557
  11. v10The agent raised the row loop to sixteen for tall channel counts7.57
  12. v11The agent reordered the scale multiply to stop FP16 overflow7.623
  13. v12The agent dropped the FP32 temporary and kept the gate multiply FP167.685

The visible run reached 7.685×, but the submitted kernel failed correctness on all four hidden cases.

On the hidden set

Original metricNormalised score
Starter1.00x0.00
Frontier-calibrated reference8.08x0.60
Upper21.36x1.00
This run (Kimi K3)00.0000
<240 minWall clock
$0.46Spend
3.7MTokens
4Measured versions

On the visible set

1.0 2.5 4.0 5.5 7.0 0 1 2 3 Agent step Visible raw speedup ↑ v0 v1 v2 v3
keptrolled backsubmitted
  1. v0The agent inherited the PyTorch RFFT gated-convolution baseline1.00000 min · $0.00
  2. v1The agent replaced the forward path with an in-kernel 64×64 Triton FFT6.3513
  3. v2The agent reduced FP16 casts and raised the pipeline depth to four6.4840
  4. v3The agent added final guards and reran the complete visible self-check6.4693<240 min · $0.46

On the hidden set

Original metricNormalised score
Starter1.00x0.00
Frontier-calibrated reference8.08x0.60
Upper21.36x1.00
This run (Grok 4.6)7.0280.6700
145 minWall clock
$61.75Spend
215.6MTokens
1Versions, 0 kept

Only one comparable self-check score, so no curve — the versions below list what each one changed.

  1. v0The agent inherited a PyTorch RFFT gated convolution with hand-written autograd1

On the hidden set

Original metricNormalised score
Starter1.00x0.00
Frontier-calibrated reference8.08x0.60
Upper21.36x1.00
This run (DeepSeek V4 Pro)3.97520.6057
116 minWall clock
$15.98Spend
6.9MTokens
1Versions, 0 kept

Only one comparable self-check score, so no curve — the versions below list what each one changed.

  1. v0The agent inherited a PyTorch RFFT gated convolution with hand-written autograd1

On the hidden set

Original metricNormalised score
Starter1.00x0.00
Frontier-calibrated reference8.08x0.60
Upper21.36x1.00
This run (Qwen3.8 Max)3.81170.6000
40 minWall clock
$4.45Spend
400kTokens
1Versions, 0 kept

Only one comparable self-check score, so no curve — the versions below list what each one changed.

  1. v0The agent inherited a PyTorch RFFT gated convolution with hand-written autograd1

On the hidden set

Original metricNormalised score
Starter1.00x0.00
Frontier-calibrated reference8.08x0.60
Upper21.36x1.00
This run (GLM 5.3)0.9998260.0000
<240 minWall clock
$4.48Spend
4.4MTokens
8Versions, 7 kept

On the visible set

0.9 1.2 1.5 1.8 2.1 2.4 0 2 3 4 6 Agent step Visible raw speedup ↑ v0 v1 v2 v3 v4 v5 v6 v7
keptrolled backsubmitted
  1. v0The agent inherited a PyTorch RFFT gated convolution with hand-written autograd1
  2. v1The agent ran the input and kernel FFTs in FP16 on power-of-two sizes2.1496
  3. v2The agent restricted the fast half-FFT path to FP16 power-of-two cases2.1523
  4. v3The agent added a batch-one path transforming input and kernel together2.1501
  5. v4The agent used a 3D inverse half FFT instead of flattening frequencies2.1502
  6. v5The agent moved the convolution scaling into the forward-norm FFT identity2.3977
  7. v6The agent applied the same normalization identity to the FP32 fallback2.4006
  8. v7The agent flattened the final conv and out-gate multiply to two dimensions2.4014<240 min · $4.48

On the hidden set

Original metricNormalised score
Starter1.00x0.00
Frontier-calibrated reference8.08x0.60
Upper21.36x1.00
This run (GPT-5.5)2.39340.3099

Leaderboard

Where each run landed on the sealed held-out set, on the same normalised-score scale as the anchors above.

0 0.3 0.6 1.0 1 Opus 5 claude code · max 0.782 2 Grok 4.6 grok · xhigh 0.670 3 Gemini 3.7 Flash antigravity · high 0.624 4 GPT-5.6-sol codex · max 0.608 5 DeepSeek V4 Pro claude code · max 0.606 6 Qwen3.8 Max qwen coder · xhigh 0.600 7 GPT-5.5 codex · xhigh 0.310 8 GLM 5.3 claude code · max 0.000 9 Kimi K3 kimi cli · max 0.000