https://duoan.github.io/tags/diffusion/ Recent content in Diffusion on Duo's Tech Blog https://duoan.github.io/images/papermod-cover.png https://duoan.github.io/images/papermod-cover.png Hugo -- 0.153.1 en-us Thu, 16 Jul 2026 00:00:00 +0000 https://duoan.github.io/posts/foundation-model-kernel-optimization-2026/ Thu, 16 Jul 2026 00:00:00 +0000 https://duoan.github.io/posts/foundation-model-kernel-optimization-2026/ <h1 id="foundation-model-kernel-optimization-in-2026-a-field-guide-across-dense-moe-multimodal-and-diffusion">Foundation Model Kernel Optimization in 2026: A Field Guide Across Dense, MoE, Multimodal, and Diffusion</h1> <p>Most “kernel optimization” conversations still start with FLOPs. In 2026 that is usually the wrong first question. Foundation-model runtime is dominated by <strong>HBM traffic, KV cache, temporary tensors, collectives, dynamic permutation, and launch overhead</strong>. FlashAttention, fused linear–cross-entropy, paged KV, MoE grouped GEMM, and DeepEP-style dispatch all share one essence: <em>do not materialize intermediates</em>, or <em>make each byte travel once</em>.</p>