▪️ One-Step Gradient Delay is Not a Barrier for Large-Scale Asynchronous Pipeline Parallel LLM Pretraining
▪️From Optimization to Generalization under Heavy-Tailed Data: The Role of Gradient Clipping
▪️Softsign: Smooth Sign in Your Optimizer For Better Parameter Heterogeneity Handling
▪️Gradient-Free Approaches is a Key to an Efficient Interaction with Markovian Stochasticity

1️⃣Workshop on Connecting Low-rank Representations in AI
▪️Tensorion: A Tensor-Aware Generalization of the Muon Optimizer - совместно с ВШЭ

2️⃣Workshop on Weight-Space Symmetries
▪️Hierarchical Mixture-of-Experts with Two-Stage Optimization
▪️Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression
Analyzing Stream Collapse in Hyper-Connections: From Diagnosis to Mitigation