According to reports, NVIDIA and MIT researchers released Lightning OPD (Offline On-Policy Distillation), a new post-training framework for large language models that eliminates the need to keep a teacher model running during training. By precomputing the teacher model’s log-probabilities offline, the framework improves training efficiency by 4x while freeing all GPU resources for student model training.
In testing on 8 NVIDIA H100 GPUs, Lightning OPD successfully distilled Qwen3-30B-A3B-Base (a 30-billion parameter MoE model) and achieved 71.0 on the AIME 2024 benchmark, whereas standard OPD ran out of memory on the same hardware. For the smaller Qwen3-8B model, the framework required only 30 GPU hours to reach 69.9 points.
Disclaimer: The information on this page may come from third parties and does not represent the views or opinions of Gate. The content displayed on this page is for reference only and does not constitute any financial, investment, or legal advice. Gate does not guarantee the accuracy or completeness of the information and shall not be liable for any losses arising from the use of this information. Virtual asset investments carry high risks and are subject to significant price volatility. You may lose all of your invested principal. Please fully understand the relevant risks and make prudent decisions based on your own financial situation and risk tolerance. For details, please refer to
Disclaimer.
Related Articles
B.AI Platform Adds 8,756 Users on May 11, DeepSeek-V4 Drives 60% of Token Consumption
According to B.AI, the platform added 8,756 new users on May 11, while Stripe payment adoption among core paying users reached 69.0%, reflecting improved retention of traditional developers and production-grade users. DeepSeek-V4 series models accounted for nearly 60% of token consumption,
GateNews11m ago
A flood of major macro catalysts is coming this week: a panoramic breakdown from the release of CPI to the consideration of the CLARITY bill
In mid-May 2026, the cryptocurrency market is going through an extremely rare macro-sensitive window. U.S. April CPI data will be released on May 12, the Trump-Xi Beijing summit is scheduled for May 13 to 15, and the Senate Banking Committee will review the CLARITY Act (the Digital Asset Market Clarity Act) on May 14. Three independent but highly interconnected events will land within just four days, creating a phase-by-phase stress test for the pricing mechanism of crypto assets. Event calendar
GateInstantTrends34m ago
AI Voice Startup Vapi Completes $50M Series B Led by Peak XV Partners
According to TechCrunch, Vapi, an AI voice startup, completed a $50 million Series B funding round led by Peak XV Partners, with a post-money valuation of approximately $500 million. Ring, Amazon's subsidiary, deployed Vapi to handle 100% of its inbound calls after evaluating over 40 AI voice
GateNews1h ago
GPT-5.4 Accuracy Drops from 100% to 54% on ARC-AGI After Repeated Memory Summarization
According to Beating, a recent Agent memory study by Dylan Zhang, a PhD student at University of Illinois, found that repeatedly summarizing model experiences can degrade performance rather than improve it. In ARC-AGI tasks, GPT-5.4 achieved 100% accuracy on 19 problems without memory, but after
GateNews1h ago
OpenAI Expands Trusted Access Program to Dozens of European Enterprises on May 12
According to reports on May 12, OpenAI announced plans to expand its Trusted Access Program to dozens of European enterprises.
GateNews1h ago
A real-life Transformer! UBTECH Technology launches the world’s first mass-produced robot vehicle, priced at $570k
Chinese robotics company Unitree Robotics released its latest product, GD01, on May 12 — a manned mecha that can freely switch between bipedal humanoid and quadruped crawling modes, arguably bringing the sci-fi scenes from the blockbuster films Transformers and Pacific Rim into reality. This release is not just a technological spectacle, but also reflects the rapid push forward of China’s robotics industry. Unitree Unveils: GD01, A Manned Transformable Mecha, from $650,000 The world’s first prod
ChainNewsAbmedia2h ago
