Category: LLM Training & Models

Perplexity AI Cuts Inference Latency 5x with New Open-Source Rust Tokenizer Perplexity open-sourced a tokenizer that runs at 63 microseconds with zero memory allocations — compared to 7,295 allocations and…

NVIDIA’s Nemotron-Labs-Diffusion Advances AI Text Generation with New Decoding Methods NVIDIA has introduced Nemotron-Labs-Diffusion, a novel language model family that breaks from solely focusing on parameter count to prioritize inference…

NVIDIA Demos 4-Bit Training for Giant AI Models, Cutting Costs and Boosting Speed The race to build the most capable artificial intelligence models is increasingly prioritizing computational efficiency alongside raw…

New Training Method Slashes AI Model Pre-Training Time by 2.5x The race to build more capable AI models is hitting a speed bump: pre-training takes too long and costs too…

Dense Matrix Multiplication’s Dominance Is Being Challenged — And the Numbers Back It Up Feedforward layers sit at the heart of every large language model, consuming over two-thirds of model…

NVIDIA’s Star Elastic Model Packs Multiple Sizes Into One Checkpoint A single 18.7 GB file can now run what used to require three separate model deployments totaling 126 GB —…

One in Four Words Gone: Why Trusting LLMs With Your Documents Is a Gamble You’re Likely Losing Hand a document to a frontier AI model and ask it to manage…

The race for more capable AI is often framed as a quest for ever-larger models. However, Zyphra’s newly released ZAYA1-8B language model upends this assumption, demonstrating that advanced reasoning abilities,…

The practical deployment of large language models (LLMs) has moved beyond sheer capability to address the engineering hurdle of speed. Google AI has introduced Multi-Token Prediction (MTP) drafters for its…

A surprising number of conversational AI systems have been forced to choose between speaking fast and speaking intelligently. Sakana AI’s new KAME architecture shatters this dichotomy, introducing a system that…