Research

For a complete and frequently updated bibliography, please see my Google Scholar. I keep this page as a compact research summary rather than a duplicate publication database.

Last manually curated: May 2026.

Research Threads

• Optimization for machine learning model training. Algorithm design and convergence theory for training a broad range of ML models, from classical statistical learning to modern deep neural networks.
• Optimization for large language models. Optimizer design, memory-efficient pretraining and fine-tuning, and optimization perspectives on alignment.
• Bilevel, minimax, and multi-objective learning. Theory and algorithms for hierarchical learning problems, including constrained and multi-task settings.
• Riemannian and zeroth-order optimization. Stochastic approximation, derivative estimation, ADMM-type methods, and optimization over manifolds.
• Distributed and decentralized optimization. Problem-parameter-free and federated algorithms for large-scale nonconvex learning.

Recent Highlights

• A minimalist optimizer design for LLM pretraining. ICML 2026 [PDF]
• A Tale of Two Problems: Multi-Task Bilevel Learning Meets Equality Constrained Multi-Objective Optimization. ICML 2026
• Muon Outperforms Adam in Tail-End Associative Memory Learning. ICLR 2026 [PDF]
• Joint Demonstration and Preference Learning Improves Policy Alignment with Human Feedback. ICLR 2025 (Spotlight) [PDF]

Selected Publications

• Riemannian Bilevel Optimization. Journal of Machine Learning Research [PDF]
• A Riemannian ADMM. Mathematics of Operations Research [PDF]
• Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms. SIAM Journal on Optimization [PDF]
• Problem-Parameter-Free Decentralized Nonconvex Stochastic Optimization. Pacific Journal of Optimization [PDF]
• Stochastic Zeroth-order Riemannian Derivative Estimation and Optimization. Mathematics of Operations Research [PDF]
• Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment. NeurIPS 2024 [PDF]
• Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark. ICML 2024 [PDF]