#machine-learning

Articles tagged with #machine-learning

Chunking Is Design, Not Preprocessing: The Hidden Decision Behind Every RAG System
Retrieval-Augmented Generation was meant to solve one simple problem: language models forget. RAG gave them a way to look things up — to ground their words in real knowledge. But grounding is fragile,
Sep 1, 202614 min read4
How KV Cache and Paged Attention Unlock Faster, Cheaper LLM Inference
TL;DR: LLM inference bottlenecks are usually memory, not compute. KV cache eliminates redundant attention computation. Paged attention eliminates memory fragmentation — cutting waste from ~60–80% down
Aug 27, 202613 min read310
Understanding the LLM Inference Workload: From Tokens to Attention
Based on insights from Mark Moyou, Senior Solutions Architect at NVIDIA Why LLM Inference Is Different Send a prompt to a large language model and you're doing something fundamentally different from t
Jul 28, 20265 min read1.2K