Microflow: Microarchitectural Causal Observability for Deep Cross-Layer Analysis and Optimization

Saber Ganjisaffar, Chengyu Song, Nael Abu-Ghazaleh 2026-07-19
Microarchitectural Microarchitecture Simulation

The problem is that existing architectural simulators expose aggregate metrics or raw traces but fail to reveal complex interactions among microarchitectural events and their relationship to program execution. Microflow introduces an observability framework that transforms execution traces into the Microflow Intermediate Representation (MFIR), explicitly capturing dependencies across software semantics, instructions, microarchitectural events, and hardware resources. On two SPEC CPU 2017 benchmarks, Microflow uncovers bottlenecks invisible from aggregate symptoms, such as hidden misprediction costs in leela and cross-loop-iteration contention in mcf. This matters because making causality queryable provides a strong foundation for performance analysis and hardware-software co-design, enabling systematic reasoning about complex interactions opaque to existing tools.

PDF

Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent

Lingyun Yang, Yuxiao Wang, Shenghao Liang, Linfeng Yang 2026-07-19
Inference LLM GPU Hardware Agent

Atrex-Bench addresses the problem that existing GPU kernel benchmarks use synthetic or curated workloads, not production traces. The method samples 30 operators and 440 shapes from full-cluster inference traces, weighting each by GPU time and card-hours. Experimental evidence shows the best vanilla model reaches only ~10% of roofline performance, with much apparent correctness from PyTorch fallbacks. This matters because the co-released Atrex-Kernel-Agent (AKA) converts fallbacks into kernels matching hand-tuned baselines, demonstrating a path to production-ready LLM-generated kernels.

PDF

ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMM

Hyunwoo Oh, Suyeon Jang, Hanning Chen, Sanggeon Yun 2026-07-19
Gem5 Microarchitecture Simulation Inference

The problem is that low-bit GEMM is central to efficient ML inference, but very-low-bit execution is poorly suited for conventional CPUs, and fragmented precision regimes make lightweight CPU support selection a first-class design challenge. ExaGEMM proposes a workload-aware codesign framework for CPU-native low-bit GEMM via register-resident LUT execution, requiring only an in-register select/feed mechanism with explicitly modeled cost. Experimental evidence shows ExaGEMM improves latency by 13.29x over software-only baselines across representative ML models and CPU targets, while pruning the candidate space by 99.2% before simulation. This matters because it demonstrates that workload-aware frontier selection is especially important for mixed-precision LLM workloads, enabling efficient CPU-driven ML inference without major hardware redesign.

PDF

LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget

Changhai Zhou, Kieran Liu, Yuhua Zhou, Qian Qiao 2026-07-19
Attention Inference Training GPU

LongStraw addresses the growing gap between inference context lengths and RL post-training, where inference systems handle million-token contexts but post-training often stays at 256K tokens or below. The method evaluates shared prompts without autograd, retains only model-specific state for later tokens, and replays short response branches one at a time to reduce the live training graph under a fixed GPU budget. On eight H20 GPUs, LongStraw completes grouped Qwen scoring and response backward at 2.1M positions for groups of 2 and 8, with a stress test reaching 4.46M positions, while on 32 H20 GPUs it validates the execution path for a 2.1M-token prompt across all 78 layers of GLM-5.2. This matters because it enables million-token RL post-training for AI agents with long trajectories under fixed GPU budgets, though the abstract does not disclose complete training correctness.

PDF

PolyQ: Codesigning End-to-End Quantization Framework for Scalable Edge CPU LLM Inference

Hyunwoo Oh, Suyeon Jang, Hanning Chen, KyungIn Nam 2026-07-19
Inference LLM Quantization Compiler Low Bit Runtime

PolyQ addresses the problem that existing low-bit quantization for CPU LLM inference offers either coarse operating points or fine-grained mixed precision that is inefficient on CPUs. The method is a compiler/quantization co-design that assigns per-channel bit-widths from {2,3,4,8,16} and uses compile-time permutation and clustering to generate SIMD- and LUT-compatible kernels with layout regularization off the runtime path. On Falcon-H1-3B, Llama2-13B, and Qwen3-32B, PolyQ improves perplexity by 2.4–32.1% over prior methods at a 3b target and reduces activation reorder traffic by up to 70.8% on three representative CPUs. This matters because it demonstrates that fractional-bit CPU deployment is practical, predictable, and energy-efficient for scalable edge inference.

PDF

Valinor: Architectural Support for Fast, Energy-Efficient and Programmable Physical Memory Allocation

Konstantinos Kanellopoulos, Spiros Galanopoulos, Konstantinos Sgouras, Vlad-Petru Nitu 2026-07-19
Workload Runtime Hardware Simulation

The problem is that physical memory allocation in current systems incurs high overhead from minor page faults, which can account for up to 54% of runtime and 40% of system energy in short-lived workloads like serverless functions. The method, Valinor, is a hardware-OS cooperative substrate that introduces a programmable hardware allocation engine executing compact OS-supplied allocation libraries at near fixed-hardware speed. On a BOOM RISC-V soft core running Linux, Valinor accelerates allocation by 17x, improves end-to-end performance by 16%, and reduces energy consumption by up to 8%, with full-system simulation confirming hardware-class performance across six allocation libraries. This matters because Valinor provides the flexibility to support diverse allocation policies and adapt to new hardware conditions while achieving the performance and energy efficiency of dedicated hardware.

PDF

NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference

Jiajun Hu, Ruthwik Reddy Sunketa, Lei Zhao, Archit Gajjar 2026-07-19
Design Space Exploration Agentic Architecture Inference

The problem is that conventional ReRAM-based IMC blocks in FPGAs only support static-weight VMM, limiting efficiency gains for Transformer models that require nonlinear and dynamic matrix-matrix multiplication (DIMM), while ADCs consume over 70% of IMC block area and power. The method proposes a novel FPGA architecture with an ADC-free IMC block using analog content-addressable memories (ACAMs) for native nonlinear operations, along with FPGA-aware design-space exploration and efficient mapping for DIMM. Experimental evidence shows up to 40x and 1.9x higher energy efficiency and 4.1x and 2.5x higher area efficiency on CNN and Transformer benchmarks, respectively. This matters because it significantly improves FPGA DL inference efficiency, especially for Transformer-based workloads, advancing domain-specialized FPGA design.

PDF

StreamingQEC: Streaming Quantum Error Correction in Tightly Integrated Quantum-Classical Systems via Certified Recurrence

Panayiotis Christou, Shuwen Kan, Hao Wang, Ying Mao 2026-07-19
Design Space Exploration Quant Ph Agentic Architecture

StreamingQEC addresses the problem of modeling resource contention in tightly integrated quantum-classical systems during continuous quantum error correction. The method introduces a system-level simulator with explicit discrete-event simulation, an automatic staged-fluid mode for faster exploration, and a certified recurrence mechanism that compresses repeated scheduling states. Experimental evidence shows recurrence achieves a 24.0x host-side speedup while preserving 59,743,936 decoding events for a 16-job anchor workload, and the staged-fluid mode yields a mean makespan error of 2.60%. This matters because it enables system architects to evaluate resource-limited pipeline stalls and saturation under microsecond-scale QEC cycles, which is critical for designing scalable fault-tolerant quantum computers.

PDF