Filtered by: Inference × Clear all

MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel

Lenore Mulin, Gaetan Hains 2026-07-25
Attention Inference × Transformer GPU Hardware Cache

The problem is to derive memory-optimal inference artifacts for transformer attention using the Mathematics of Arrays (MoA), directly from the forward-pass Denotational Normal Form (DNF) with a fixed query-row index. The method produces four artifacts: a single-query decode DNF eliminating the K^T buffer, a C/OpenACC GPU kernel with exact IEEE-754 arithmetic, a multi-step KV-cache with O(d_k+d_v) per-step append, and GQA/MQA via ψ-selection. Experimental evidence shows the DNF achieves DRAM traffic of (d_k+nd_k+nd_v+d_v)×4 B with numerical error ≤2×10⁻⁷, and the GPU kernel is verified to have zero error against PyTorch scaled_dot_product_attention. This matters because it provides proven, memory-optimal attention inference with exact arithmetic, enabling efficient deployment of large transformer models on GPUs.

PDF

A Flexible Sparsity-Aware FPGA Accelerator with Column-Wise Compression for Efficient CNN Inference

Amirhossein Zarei, Shervin Vakili 2026-07-25
Microarchitecture Simulation Sparsity Accelerator Inference ×

Problem: Efficient CNN inference on resource-constrained FPGAs is challenged by the irregularity of sparsity patterns and hardware overhead. Method: SparHiXcel-v2 introduces a column-wise kernel compression technique and a hardware-algorithm co-design framework with ordering optimization and multi-phase structured pruning. Finding: On a cost-effective AMD Kintex UltraScale+ FPGA, SparHiXcel-v2 achieves over 2.5 TOPS and 210 GOP/s/W for VGG16, and over 1.1 TOPS and 72 GOP/s/W for ResNet18 with modest accuracy degradation. Why it matters: This work provides a flexible and efficient FPGA accelerator that balances sparsity flexibility and hardware efficiency for resource-constrained CNN inference.

PDF