Tasks/Life Sciences & Medicine

Protein–Ligand Co-Folding Inference Optimization

Improve sampling, pose selection, and post-processing under physical-validity and 2 Å RMSD checks

Life Sciences & Medicinestructural biologymolecular docking
Background

Deep-learning co-folding — predicting a protein's structure jointly with a bound small molecule from sequence alone — underpins structure-based drug design, yet poses often miss atomic accuracy or violate basic chemistry on unfamiliar targets. The starting point is a single-shot pipeline that runs one pretrained model at fixed settings and returns its default pose. The work is to redesign how that inference is run — sampling, selection, and post-processing — while weights stay frozen. Gains must hold on sealed complexes under strict physical-validity and 2 Å accuracy checks.

instruction.mdthis is what the agent is given

You inherit a deliberately weak inference pipeline that predicts a protein-ligand complex from protein sequence, ligand SMILES, and a precomputed MSA. Improve its inference-time success rate under the pinned physical-validity checks and a 2 Å ligand-RMSD threshold; your submitted method is re-run on a sealed post-training-cutoff held-out split for scoring.

Hard Constraints

  • Work only in /app/methods/main/. The graded artifact is that source directory.
  • Keep predict_complex(item: dict) -> dict as the entry point. It must predict each case at grading time and may not return precomputed coordinates.
  • Use only the protein sequences, ligand SMILES, and MSAs in the three-field predict_complex input. Visible identities and crystal truth remain in the development workspace so you can analyze self-check failures, but submitted prediction code must not read or branch on them. The sealed grader stages an anonymous input and enforces that hidden identities, crystal truth, hidden paths, and hidden results are inaccessible to the prediction process.
  • This is co-folding, not docking into a supplied receptor. Do not retrieve experimental structures or use a structure database to identify a case.
  • Inference-time changes are allowed; training or fine-tuning model parameters is not.
  • Stay within one GPU. The autonomous research run and each complete sealed evaluation each have a 12-hour compute window. The verifier fails malformed output, missing output, crashes, and timeouts closed.

What You Have

  • /app/methods/main/solver.py — the deterministic weak baseline you edit.
  • /app/methods/main/cofold_utils.py — one deterministic baseline adapter plus complex-to-PDB/SDF conversion helpers. You may edit it or add source files beside it.
  • /app/data/visible/ — a frozen 20-case development suite with sequences, ligand SMILES, precomputed MSAs, and crystal truth. It contains the accepted author's original visible10 plus 10 cases preregistered from the original held-out set using input-only distribution features; no crystal, model prediction, or score was used to choose them.
  • /app/selfcheck.py — runs your current method and the grading metric on visible data. With no flags it evaluates the full frozen visible suite, starting a fresh prediction process per case under the grading time limits; --case-index and --smoke are diagnostic subsets and are not comparable scores.
  • Pinned local inference assets and all required scientific dependencies are installed in the image. Runtime model downloads are neither needed nor allowed.

What You Submit

Leave your best source implementation in /app/methods/main/. For each call, the input contains exactly:

{
    "protein_chains": [{"chain_id": str, "sequence": str}, ...],
    "ligand_smiles": str,
    "msa_dir": str,
}

Return exactly:

{"protein_pdb": str, "ligand_sdf": str}

The PDB must contain the predicted protein and the SDF must contain one predicted ligand pose. Both structures must come from the same prediction and remain in the same coordinate frame. Helper modules added under /app/methods/main/ are included in the submission; generated poses, model weights, binaries, compressed data blobs, and files outside that directory are not.

The source-only artifact may contain at most 32 UTF-8 Python files, 128 KiB per file and 256 KiB total, with nesting depth at most four. Symlinks, hard links, hidden paths, non-Python files, literals above 16 KiB (or above 64 KiB in aggregate), programs above 20,000 AST nodes, dynamic eval/exec/compile/__import__, and direct imports of base64, binascii, bz2, gzip, lzma, marshal, pickle, or zlib are rejected. Names reserved by the harness (metric.py, source_contract.py, selfcheck.py, evaluate.py, grade.py, child_predict.py, and score_pose_worker.py) may not be added under the artifact. These bounds prevent bundling a public structure lookup table; ordinary inference logic and the installed model libraries remain available. selfcheck.py applies this same byte-identical contract before importing your method, and python /app/source_contract.py /app/methods/main runs it without GPU inference.

The predicted PDB must represent every supplied protein chain and provide sequence-matched C-alpha coordinates for at least 95% of each chain. A few unresolved terminal residues are tolerated; returning only a pocket or one domain is not a valid full-complex co-folding prediction.

How It Is Judged

  • The verifier runs the submitted method independently on the 42 complexes that remain sealed after the preregistered visible20/hidden42 repartition of the accepted author's 10+52 cases. The agent runtime contains no held-out files or repository history and reaches only the model API; the separate verifier is offline and exposes only one anonymous inference input at a time. The repartition preserves every original case exactly once and does not change the input fields, co-folding operation, parameter envelope, case weighting, or metric.
  • For each case, the verifier matches predicted and crystal protein chains by sequence, aligns sequence-matched C-alpha atoms with one rigid transform, and applies that transform to the predicted ligand. Absolute translation and rotation therefore do not affect the score.
  • A case succeeds only if the transformed ligand passes every pinned binary redocking check, including symmetry-aware heavy-atom RMSD at or below 2 Å. Missing or non-boolean checks fail closed.
  • The raw metric is the unweighted fraction of successful complexes. Higher is better. The normalized score is a monotonic function of sealed success rate and is not shown to you; optimize raw success and cross-case generalization.
  • The hidden process exposes only the aggregate result. It does not expose case identities, per-case scores, error traces, or checkpoints, and it is not an optimization oracle.

Rollouts

1559 minWall clock
$94.29Spend
158.3MTokens
8Versions, 3 kept

On the visible set

0.2 0.3 0.4 0.5 0.6 0.7 0 3 6 9 12 Agent step Visible PoseBusters success rate ↑ v0 v1 v2 v3 v4 v5 v6 v7
keptrolled backsubmitted
  1. v0The agent started from one deterministic Boltz prediction.8/2038 min · $4.18
  2. v1The agent ranked five Boltz samples by native confidence.7/20107 min · $11.65
  3. v2The agent gated extra samples behind a confidence improvement.9/20202 min · $22.12
  4. v4The agent tried Chai alone and lost most Boltz wins.4/20234 min · $25.54
  5. v5The agent increased recycling and lost a baseline success.7/20278 min · $30.44
  6. v3The agent sent uncertain Boltz cases to a calibrated Chai fallback.10/20857 min · $53.85
  7. v6The agent added Chai diversity but only tied the submission.10/201416 min · $76.13
  8. v7The agent added noisier Boltz samples and again only tied.10/201557 min · $94.02

On the hidden set

Original metricNormalised score
Starter0.40476 (17/42)0.00
Upper1.00000 (42/42)1.00
This run (GPT-5.6-sol)0.4285710.0400
262 minWall clock
$19.75Spend
29.8MTokens
16Versions, 12 kept

On the visible set

0.2 0.3 0.4 0.5 0.6 0.7 0 4 8 12 Agent step Visible PoseBusters success rate ↑ v0 v1 v2 v3 E1 P0 P1 P2 P3 P4 P5 P6 P7 P8
keptrolled backsubmitted
  1. v0The agent inherited a one-shot co-folding baseline with no pose cleanup0.486 min · $7.42
  2. v1The agent sampled ten poses and eased the winner off the protein0.5165 min · $12.93
  3. v2The agent called a second folding model when its own poses disagreed0.55$16.34
  4. v3The agent dropped ligand hydrogens and loosened when clash relief fires0.55$16.34
  5. v4The agent capped retry time by the case's remaining allowanceno number logged$16.34
  6. E1The agent added a second folding model hoping consensus would find missed poses0.5$16.34
  7. E2The agent injected the true cofactor to test its failure hypothesis0/5 correct, reverted$16.34
  8. P0The agent froze an offline pose pool to test ideas without the GPU0.4$16.34
  9. P1The agent relaxed each pose with a force field0.45$16.34
  10. P2The agent began easing the ligand off the protein at 0.85 clearance0.45$16.34
  11. P3The agent widened the clearance margin to 0.95 of van der Waals contact0.5$16.34
  12. P4The agent opened the contact distance to full van der Waals touching0.5$16.34
  13. P5The agent settled on the middle of the safe clearance plateau0.5$16.34
  14. P6The agent pushed the margin further to find where the plateau ends0.5$16.34
  15. P7The agent pushed the poses too far off and started losing them0.45$16.34
  16. P8The agent demanded clearance from every predicted pocket and over-constrained the poses0.45$16.34

On the hidden set

Original metricNormalised score
Starter0.40476 (17/42)0.00
Upper1.00000 (42/42)1.00
This run (Opus 5)0.4047620.0000
257 minWall clock
$14.83Spend
96.1MTokens
2Versions, 1 kept

On the visible set

0.2 0.3 0.4 0.5 0.6 0.7 0 0 0 1 1 Agent step Visible PoseBusters success rate ↑ v0 v1
keptrolled backsubmitted
  1. v0The agent took one deterministic pose per complex as its starting point0.4
  2. v1The agent sampled three poses and refined only the hydrogens0.45

On the hidden set

Original metricNormalised score
Starter0.40476 (17/42)0.00
Upper1.00000 (42/42)1.00
This run (Gemini 3.7 Flash)0.4047620.0000
-Wall clock
-Spend
20.2MTokens
3Versions, 1 kept

On the visible set

0.2 0.3 0.4 0.5 0.6 0.7 0 0 1 2 2 Agent step Visible PoseBusters success rate ↑ v0 v2
keptrolled backsubmitted
  1. v0The agent inherited one deterministic pose and estimated how often it lands0.5
  2. v1The agent switched models on a confidence threshold it never calibratednever calibrated
  3. v2The agent let two folding models vote on where the ligand sits0.45

On the hidden set

Original metricNormalised score
Starter0.40476 (17/42)0.00
Upper1.00000 (42/42)1.00
This run (Kimi K3)0.3809520.0000
284 minWall clock
$25.35Spend
43.1MTokens
8Versions, 4 kept

On the visible set

0.2 0.3 0.4 0.5 0.6 0.7 0 2 3 4 6 Agent step Visible PoseBusters success rate ↑ v0 v5 v7
keptrolled backsubmitted
  1. v0The agent took the inherited deterministic pose as its tuning reference0.5
  2. v1The agent pooled extra samples from both models and picked the least clashing7W6F fail, rank shifts
  3. v2The agent clustered poses across models and over-trusted the newcomer7SSM, 6ZR8 fail
  4. v3The agent gated the second model behind a burial margin7W6F pass, 2 fail
  5. v4The agent restored single-sample Boltz and kept only the burial-gated override7OLT pass, 2 fail
  6. v5The agent ranked poses by crystal-free PoseBusters dock validity0.583
  7. v6The agent gave only large proteins extra diffusion draws7MWN pass, rmsd 0.80
  8. v7The agent raised the size cutoff so mid-sized systems stayed single-sample0.667

On the hidden set

Original metricNormalised score
Starter0.40476 (17/42)0.00
Upper1.00000 (42/42)1.00
This run (Grok 4.6)00.0000
462 minWall clock
$3.58Spend
44.3MTokens
3Versions, 2 kept

On the visible set

0.2 0.3 0.4 0.5 0.6 0.7 0 100 200 300 Agent step Visible PoseBusters success rate ↑ v0 v2
keptrolled backsubmitted
  1. v0The agent measured the inherited one-shot pipeline to find where it fails0.4444 min · $0.13
  2. v1The agent built a multi-sample pipeline with validity checks and pose repairGPU blocked, unscored24 min · $0.84
  3. v2The agent found its repairs never reached the output and bought extra draws0.4293 min · $3.10

On the hidden set

Original metricNormalised score
Starter0.40476 (17/42)0.00
Upper1.00000 (42/42)1.00
This run (DeepSeek V4 Pro)00.0000
275 minWall clock
$26.50Spend
62.3MTokens
5Versions, 4 kept

On the visible set

0.2 0.3 0.4 0.5 0.6 0.7 0 1 2 3 4 Agent step Visible PoseBusters success rate ↑ v1 v2
keptrolled backsubmitted
  1. v1The agent measured the inherited one-sample baseline and sorted its failures0.42 min · $0.18
  2. v2The agent drew eight poses and ranked them by ligand-interface confidence0.4526 min · $3.45
  3. v2.1The agent nudged correct-but-clashing poses free of the receptorcase 11 not rescued$9.08
  4. v3The agent snapshotted the selection-and-refinement pipeline before adding a second modelsnapshot only$14.70
  5. v4The agent added a second folding model and let confidence choose the familyselfcheck still running188 min · $20.32

On the hidden set

Original metricNormalised score
Starter0.40476 (17/42)0.00
Upper1.00000 (42/42)1.00
This run (Qwen3.8 Max)0.3809520.0000
576 minWall clock
$19.12Spend
69.0MTokens
7Versions, 5 kept

On the visible set

0.2 0.3 0.4 0.5 0.6 0.7 0 2 3 4 6 Agent step Visible PoseBusters success rate ↑ v0 v1 v2 v2-selfcheck v3 v4
keptrolled backsubmitted
  1. v0The agent started from a single deterministic pose and charted its failures0.444 min · $1.43
  2. v1The agent drew eight poses and ranked them by ligand-focused confidence0.45212 min · $6.84
  3. v1bThe agent shoved the ligand clear of the receptor and made things worsemargin sweep, no gain$9.79
  4. v2The agent added a second folding model, called only when the first wavered0.5385 min · $12.74
  5. v2-selfcheckThe agent confirmed the two-model portfolio end to end on the visible suite0.5$14.18
  6. v3The agent stripped dead code and made missing inputs degrade instead of crash0.5526 min · $15.62
  7. v4The agent shipped the ligand as heavy atoms only, matching the grader0.5575 min · $18.75

On the hidden set

Original metricNormalised score
Starter0.40476 (17/42)0.00
Upper1.00000 (42/42)1.00
This run (GLM 5.3)0.4047620.0000
288 minWall clock
$45.26Spend
81.4MTokens
3Versions, 2 kept

On the visible set

0.2 0.3 0.4 0.5 0.6 0.7 0 150 300 450 Agent step Visible PoseBusters success rate ↑ v0 v1 v2
keptrolled backsubmitted
  1. v0The agent measured the inherited single-sample baseline as its reference0.448 min · $2.89
  2. v1The agent called a second model only when it was very confident0.45208 min · $29.21
  3. v2The agent trusted the second model sooner on repeated-sequence complexes0.5287 min · $44.71

On the hidden set

Original metricNormalised score
Starter0.40476 (17/42)0.00
Upper1.00000 (42/42)1.00
This run (GPT-5.5)0.4047620.0000

Leaderboard

Where each run landed on the sealed held-out set, on the same normalised-score scale as the anchors above.

0 0.3 0.6 1.0 1 GPT-5.6-sol codex · max 0.040 2 Opus 5 claude code · max 0.000 3 Gemini 3.7 Flash antigravity · high 0.000 4 Kimi K3 kimi cli · max 0.000 5 Grok 4.6 grok · xhigh 0.000 6 DeepSeek V4 Pro claude code · max 0.000 7 Qwen3.8 Max qwen coder · xhigh 0.000 8 GLM 5.3 claude code · max 0.000 9 GPT-5.5 codex · xhigh 0.000