Research
For a complete and frequently updated bibliography, please see my Google Scholar. I keep this page as a compact research summary rather than a duplicate publication database.
Last manually curated: May 2026.
Research Threads
- Optimization for machine learning model training. Algorithm design and convergence theory for training a broad range of ML models, from classical statistical learning to modern deep neural networks.
- Optimization for large language models. Optimizer design, memory-efficient pretraining and fine-tuning, and optimization perspectives on alignment.
- Bilevel, minimax, and multi-objective learning. Theory and algorithms for hierarchical learning problems, including constrained and multi-task settings.
- Riemannian and zeroth-order optimization. Stochastic approximation, derivative estimation, ADMM-type methods, and optimization over manifolds.
- Distributed and decentralized optimization. Problem-parameter-free and federated algorithms for large-scale nonconvex learning.
Recent Highlights
- A minimalist optimizer design for LLM pretraining. ICML 2026 [PDF]
- A Tale of Two Problems: Multi-Task Bilevel Learning Meets Equality Constrained Multi-Objective Optimization. ICML 2026
- Muon Outperforms Adam in Tail-End Associative Memory Learning. ICLR 2026 [PDF]
- Joint Demonstration and Preference Learning Improves Policy Alignment with Human Feedback. ICLR 2025 (Spotlight) [PDF]
Selected Publications
- Riemannian Bilevel Optimization. Journal of Machine Learning Research [PDF]
- A Riemannian ADMM. Mathematics of Operations Research [PDF]
- Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms. SIAM Journal on Optimization [PDF]
- Problem-Parameter-Free Decentralized Nonconvex Stochastic Optimization. Pacific Journal of Optimization [PDF]
- Stochastic Zeroth-order Riemannian Derivative Estimation and Optimization. Mathematics of Operations Research [PDF]
- Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment. NeurIPS 2024 [PDF]
- Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark. ICML 2024 [PDF]