PallasBench
The first benchmark for LLM-generated JAX Pallas kernels on TPU and GPU. Three difficulty levels, correctness-gated speedup scoring against JAX baselines, robustness filters, CI and a Hugging Face dataset.
JAXPallasTPU
JaxBench
KernelBench, except the thing being optimised is JAX itself. Agents rewrite C++/CUDA hot paths and XLA passes, JAX is rebuilt, and the score is the speedup of the real library, gated on correctness.
C++XLAShinkaEvolve
LewyGym
A ProteinGym-compatible benchmark for zero-shot variant effect prediction across the Parkinson's gene panel (SNCA, LRRK2, GBA, PRKN, PINK1…). Finding: ESM-2's signal inverts on SNCA, and larger models make it worse.
protein LMsESM-2Parkinson's
ESM-2 650M inference optimisation
Reproduces the published ProteinGym baseline for ESM-2 650M (ρ = 0.414) on Modal A100s, pinned to exact SHAs. Compares masked-marginal, batched and single-pass scoring, followed by kernel-level work in Mojo, CuteDSL, Pallas and quantisation.
PyTorchModalProteinGym
Erdős Math Emporium
Ten parallel ShinkaEvolve instances attack open Erdős problems, with Lean 4 proof verification and a live constellation UI. Built at the {Tech: Europe} London AI Hackathon.
Lean 4Next.jsGemini
Alxandria
An autonomous research agent that finds related work, writes code and runs experiments on Kaggle. 3rd place (£1,000) at the Perplexity AI Hackathon, London.
Perplexity SonarFastAPIReact