ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMM

https://arxiv.org/abs/2607.14622v1

Core Idea

The problem is that low-bit GEMM is central to efficient ML inference, but very-low-bit execution is poorly suited to conventional CPUs, and the fragmented design space of precision regimes makes lightweight CPU support selection a first-class challenge.

For this daily profile, it is worth opening because it links Gem5, Microarchitecture, and Simulation to a concrete method, not just a broad trend.

What Is New

The novelty signal is concentrated around Gem5, Microarchitecture, Simulation, and Inference. For this profile, the important question is whether the paper changes how architecture ideas are generated, evaluated, or connected to software and hardware constraints.

Methodology

Read this as a loop: define the target system, apply the proposed mechanism, measure against a baseline, then use the measured signal to justify the next design choice. Mechanism: Low-bit GEMM is increasingly central to efficient ML inference, yet very-low-bit execution remains a poor fit for conventional CPUs. Evidence: Across representative ML models and CPU targets, ExaGEMM improves latency by 13.29x over software-only baselines, while showing that workload-aware frontier selection is especially important for mixed-precision LLM.

score(design) = quality_metric(design) - cost_to_evaluate(design) + feedback_gain(design)

Figure To Read First

Read this visual first: focus on the first architecture, workflow, or pipeline figure before the experiments. It should show what is optimized, what feedback signal is used, and where the system boundary sits.

Minimal Mental Model

research artifact
  question      -> what design, runtime, or system boundary changes?
  mechanism     -> model, agent, compiler, simulator, or hardware feedback
  evaluation    -> baseline comparison plus cost / latency / accuracy signal
  reusable idea -> what should carry into the next architecture experiment?

Why It Matters

Paper recommendations matter when they sharpen the research map: what problem is now easier to study, what methodology becomes reusable, and which architecture assumptions should be questioned next.