Filtered by: LLM × Clear all

Full-Pipeline Inference Optimization for MiMo-V2.5 Series: Pushing Hybrid SWA Efficiency to the Limit

Xiaomi MiMo Team, Anqi Liu, Aoxin Ma, Bo Chen 2026-07-16
Attention Inference LLM × GPU Rdma Serving

The problem is that Hybrid Sliding Window Attention (SWA) reduces compute and KVCache storage compared to Full Attention, but realizing these gains in production requires substantial engineering effort. The method systematically optimizes the KVCache system with layerwise prefetch, SWA-aware prefix cache trees, and specialized placement strategies, and builds GCache, a high-performance distributed cache infrastructure with RDMA-optimized networking. Experimental evidence shows strict O(W) SWA storage and high cache hit rates are achieved, with the system being the first large-scale LLM serving system in production to efficiently cover the Hybrid SWA + MoE + multimodal composite architecture. This matters because it pushes hybrid SWA efficiency to the limit, enabling practical deployment of complex multimodal models with reduced computation and storage overhead.

PDF

HeteroMosaic: Exposing and Exploiting Heterogeneous Execution Opportunities for Energy-Efficient Edge LLM Inference

Gregory Hyegang Jun, Wesley Pang, Eddie Richter, Mehdi Saeedi 2026-07-16
Roofline HPC Compiler Runtime LLM × Inference

HeteroMosaic addresses the problem that existing LLM runtimes underutilize heterogeneous resources in edge SoCs by making coarse device-level decisions or optimizing operators in isolation. The method introduces a heterogeneity-first scheduling framework that uses a heterogeneous roofline model, dependency-preserving micro-batches, and trace-guided co-optimization of scheduling and device allocation. On a balanced AMD Ryzen AI platform, HeteroMosaic achieves up to 1.73X speedup over an iGPU baseline, 1.78X over an NPU baseline, and 2.05X over frameworks like llama.cpp, while reducing energy by up to 45.3%. This matters because it demonstrates significant performance and energy efficiency gains for edge LLM inference by effectively exposing and exploiting cross-accelerator execution opportunities on unified-memory platforms.

PDF

Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction

Huan Zhu 2026-07-16
Language Model LLM × Hardware

The problem is that self-refinement fails to improve few-shot inductive reasoning in large language models because simply prompting explicit rule verbalization is ineffective. The method introduces Hourglass reasoning, which enforces strict context isolation between stages, using a frozen LLM as a meta-constructor to pass only a compressed symbolic state (schema φ and rule T) across stage boundaries. Experimental evidence shows Hourglass raises ARC-AGI-2 best-of-5 accuracy by up to 14 points, nearly doubles GPT-5.5 Verilog synthesis accuracy on ChipBench from 31% to 58%, and reverses the harmful effect of explicit verbalization on BBEH-Linguini with Gemini 3.1 Pro. This matters because it demonstrates that the structural flow of information through isolated reasoning stages, not the language used, drives rigorous inductive reasoning in frozen LLMs.

PDF

Can LLMs Perform Deep Technical Comprehension of Computer Architecture Papers?

Nishant Aggarwal, Ayushi Dubal, Sreeraj Kannakarankodi, Ian McDougall 2026-07-16
Computer Architecture Agentic Microarchitecture Simulation LLM ×

The problem is that existing LLM evaluations focus on summarization rather than deep technical comprehension, which requires structured critique identifying core mechanisms, buried assumptions, and cross-paper contributions. The method introduces Gauntlet, an open-source pipeline using five independent expert-persona reviewers and an adversarial synthesis stage to analyze computer architecture papers. On 20 ISCA 2025 and HPCA 2026 papers, evaluators preferred Gauntlet over human analysis in 15 of 20 comparisons, with significant advantage on Critical Rigor and only vanishing on Calibration, while humans won on trust and usefulness rather than depth. This matters because Gauntlet demonstrates that multi-agent LLM pipelines can outperform humans in deep technical critique, and the released analyses, scores, and rubric provide a community resource for advancing automated paper comprehension.

PDF