Blog

Your batch size should have a schedule, too

November 10, 2026

TL;DR: Let your learning-rate schedule determine your batch-size schedule.

Demystifying Manifold Constraints in LLM Pre-training

By Kang An · External post · Shared November 4, 2026

An interactive explanation of our MACRO work, exploring how manifold constraints interact with RMSNorm, weight decay, and low-precision LLM training.