Research interests
Model efficiency and reasoning; context, memory, and expressivity in transformers; diffusion language models; efficient inference and adaptation.
Education
University of Oxford
2016–2020MMath in Mathematics & Statistics
Experience
Independent Researcher
CurrentResearch on model efficiency and reasoning, context and memory in transformers, and diffusion language models.
Esquel Group
2019HR Data Analyst Intern
Developed an R-based model for three-month employee attrition prediction.
Selected publications
* Equal contribution
-
PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attention
ICLR 2026 · Co-first author
-
Rethinking Pathology Image Analysis through Shuffling
npj Biomedical Innovations, 2026 · Co-first author
-
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
ICML 2024 · First author
-
In-Context Learning Behaves as a Greedy Layer-Wise Gradient Descent Algorithm
NeurIPS Workshop, 2024 · First author
Submitted manuscripts
ICLR 2027 submissions
The Value of Memory in Language Models: Joint Limits on Language Model Expressivity
How Far Can Sliding Window Attention See?
Reasoning Under Intervention: Learned Soft Prefixes as Behavioral Probes
Speculative Correction: Draft-Then-Refine Decoding for Diffusion Language Models
Work in progress
Bottlenecks in Hybrid Sliding-Window Attention
The Expressivity of the Indexer Controls the Expressivity of the Language Model
BFLA: Block-Filtered Long-Context Attention Mechanism
Index-Predictor: Does DeepSeek Need as Many Static Heads?
Amortized Memory Systems for Long-Horizon Agentic Harnesses
Skills & languages
Programming: Python, R
Languages: English (native), Mandarin (native)