Filtered by: Sparsity × Clear all

Non-Parametric Dual-Manifold Mapping via 8-Bit Bounded Transformation Matrices: Challenging FP-centric Hardware Paradigms in Low-Energy AI

Lars Kopp 2026-06-14
Hardware AI

The paper addresses the problem of high energy costs from floating-point arithmetic in deep learning hardware. It proposes a non-parametric, training-free framework using 8-bit signed integer transformation matrices and bitwise logic for dual-manifold mapping. Experimental evidence shows near-perfect reconstruction under 90% truncation sparsity and 20% random node destruction, demonstrating extreme holographic resilience. This matters because it challenges the necessity of dense, floating-point-centric GPU accelerators, enabling a shift toward low-energy neuromorphic edge-computing.

PDF

MiniMax Sparse Attention

Xunhao Lai, Weiqi Xu, Yufeng Yang, Qiaorui Chen 2026-06-14
Sparsity ×

The problem is that quadratic-cost softmax attention makes ultra-long-context LLM inference untenable at deployment scale. The method, MiniMax Sparse Attention (MSA), uses a lightweight Index Branch for blockwise Top-k selection per GQA group and a Main Branch for exact block-sparse attention, co-designed with an exp-free GPU kernel. On a 109B multimodal model, MSA reduces per-token attention compute by 28.4x at 1M context and achieves 14.2x prefill and 7.6x decoding speedups on H800. This matters because it enables practical deployment of frontier LLMs with million-token contexts for agentic workflows and repository-scale reasoning.

PDF