Preprint 2026
Language Modeling with Hyperspherical Flows
Justin Deschenaux, Caglar Gulcehre
๐-FLM is a hyperspherical flow language model that replaces Gaussian noise on one-hot vectors with rotations on the unit sphere.
N9o8lCf
2026
ICLR 2026 Workshop ReALM-GEN ยท spotlight talk
BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers
Justin Deschenaux, Caglar Gulcehre
BlockGen trains a single denoiser over multiple block sizes, with uniform diffusion and introduces AR-guided diffusion sampling to improve quality.
Preprint 2026
Scaling Beyond Masked Diffusion Language Models
Subham Sekhar Sahoo, Jean-Marie Lemercier, Zhihan Yang, Justin Deschenaux, Jingyu Liu, John Thickstun, Ante Jukic
Scaling law study of discrete diffusion models. Despite worse perplexity, uniform-state diffusion performs best on downstream tasks.
ICLR 2026
The Diffusion Duality, Chapter II: ฮจ-Samplers and Efficient Curriculum
Justin Deschenaux, Caglar Gulcehre, Subham Sekhar Sahoo
ฮจ-samplers unlock test-time scaling for uniform discrete diffusion.
ICLR 2026 ยท oral (Top 1.13%)
Partition Generative Modeling: Masked Modeling Without Masks
Justin Deschenaux, Lan Tran, Caglar Gulcehre
Partition Generative Models (PGMs) save compute by never processing [MASK] tokens at inference.
ICLR 2026
Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wall
Mingyu Jo, Jaesik Yoon, Justin Deschenaux, Caglar Gulcehre, Sungjin Ahn
Loopholing enables forwarding latent representations across sampling steps, for improved sample quality.
2025
ICML 2025
The Diffusion Duality
Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin T Chiu, Volodymyr Kuleshov
Duo connects uniform and Gaussian diffusion, and enables Discrete Consistency Distillation for fast sampling.
ICLR 2025
Beyond Autoregression: Fast LLMs via Self-Distillation Through Time
Justin Deschenaux, Caglar Gulcehre
One of the first distillation methods for discrete diffusion models.
NeurIPS 2025
One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Models
Viacheslav Surkov, Chris Wendler, Antonio Mari, Mikhail Terekhov, Justin Deschenaux, Robert West, Caglar Gulcehre
Sparse Autoencoders enable controllable, few-steps generation of latent diffusion models.
2024
ArXiv 2024
Promises, Outlooks and Challenges of Diffusion Language Modeling
Justin Deschenaux, Caglar Gulcehre
A study of the current shortcomings of discrete diffusion models.
ICML 2024
Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations
Justin Deschenaux*, Igor Krawczuk*, Grigorios Chrysos, Volkan Cevher
In controlled settings, DDPMs can generate samples from unseen regions of the data distribution.
2023
ICLR 2023
Distributed extra-gradient with optimal complexity and communication guarantees
Ali Ramezani-Kebrya, Kimon Antonakopoulos, Igor Krawczuk, Justin Deschenaux, Volkan Cevher
Accelerated distributed training in Variational Inequality problems.