FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications

Krish Agarwal, Zhuoming Chen, Yanyuan Qin, Zhenyu Gu 2026-07-23
Inference Compiler GPU Hardware Model Parallelism Serving Agent

FlashRT addresses the problem of efficiently deploying real-time multimodal applications by automating the optimization of heterogeneous model pipelines. It introduces a chain-of-program paradigm that guides a coding agent to transform reference implementations into an intermediate representation, validate it, and iteratively optimize deployments. On NVIDIA B200 GPUs, FlashRT achieves up to ~70x latency reduction and 2.8x throughput improvement, while on AMD MI355X GPUs it matches peak latency reduction and increases throughput to 3.6x, outperforming expert implementations like vLLM-Omni. This matters because agent-driven optimization enables scalable, high-performance deployment across diverse hardware platforms without requiring hand-crafted implementations.

PDF

DGNA: Dissecting GPU NUMA Architecture through Microbenchmarking and Data Analysis

Changxi Liu, Yun Chen, Trevor E. Carlson 2026-07-23
Cache Coherence Memory Hierarchy Numa Microarchitecture Simulation HPC

The problem is that modern GPU memory architectures, particularly NUMA mechanisms within L2 and DRAM, remain a black-box, hindering optimization and simulation. DGNA introduces a methodology using microbenchmarking and Gaussian mixture models to measure L2 and DRAM latency without relying on intrinsic instructions. Applied to NVIDIA's A100 and H100 GPUs, it reveals NUMA node architecture, SM-NUMA relationships, and NUMA-aware memory allocation strategies for cache coherence. This matters because it is the first work to detail GPU memory subsystem NUMA architecture, enabling better application optimization and architectural design.

PDF

Fine-grained Computation-Communication Overlap via Tile-level Signaling and Scheduling for Mixture-of-Experts

Minyu Cui, Anna Wingkvist, Morgan Ericsson 2026-07-23
Language Model GPU Communication Scheduling

The problem is that conventional Mixture-of-Experts (MoE) implementations launch the return all-to-all communication after expert compute completes, exposing communication latency on the critical path and reducing GPU utilization. The method is a fine-grained producer-consumer co-design using tile-level signaling and scheduling, combining a persistent per-rank computation kernel that prioritizes remote-critical tiles and a persistent communication kernel on a small dedicated SM partition for segment-granular transfers. Experimental evidence on a 4-A100 GPU platform across three MoE models shows up to 2.64x end-to-end speedup and 2.74x MoE-layer speedup over four state-of-the-art systems, with consistent performance improvements across varying GEMM shapes and router modes. This matters because it provides a practical, non-intrusive approach to overlapping expert compute with the second all-to-all, significantly improving distributed MoE execution efficiency for scaling LLMs to trillion-parameter regimes.

PDF

Mapping Without Graphs: Learning Coherence Traffic for Task Placement

Guochu Xiong, Tianrui Ma, Weichen Liu 2026-07-23
Cache Coherence Microarchitecture Simulation

Existing task mapping approaches rely on predefined task graphs that fail to capture coherence-induced interactions from shared data accesses, leading to suboptimal mappings. CoTM addresses this by constructing task graphs from dynamic coherence behavior and using a lightweight heuristic with a multi-start optimization strategy guided by a coherence-aware penalty function. Experimental results show CoTM reduces average link utilization by up to 47.85% and total energy consumption by up to 10.30% compared to existing approaches. This matters because it demonstrates that incorporating cache coherence into task mapping significantly improves performance and energy efficiency for future many-core NoC systems.

PDF

BaseRT: Advancing Best-in-Class LLM Inference with Apple M5 Neural Accelerators

Fabian Waschkowski, Prabod Rathnayaka, Lukas Wesemann 2026-07-23
Attention Inference Language Model LLM Accelerator GPU Hardware

The paper addresses the problem of maximizing LLM inference throughput on Apple M5 hardware by leveraging its redesigned GPU with dedicated Neural Accelerators. The method, BaseRT, is a native Metal inference runtime that introduces hand-written Metal 4 tensor-core kernels for dense and mixture-of-experts GEMM and flash-attention prefill, routing compute-bound operations through the M5 Neural Accelerators while keeping memory-bound decode on existing kernels. On an M5 Pro across fifteen model configurations from Qwen3, Llama 3.2, and Gemma 4 families, BaseRT achieves up to 6.4× higher prompt-processing throughput than llama.cpp and 3.9× higher than MLX, with decode improvements up to 1.75× and 1.33× respectively. This establishes a new performance ceiling for on-device LLM inference, demonstrating that M5 tensor cores are decisive for prompt processing on Apple Silicon.

PDF

Formal Verification of an Out-of-Order Multiprocessor against an In-Order Weak-Memory ISA

Janggun Lee, Jeehoon Kang 2026-07-23
Microarchitecture Microarchitectural Simulation

The problem is the formal verification of an out-of-order multiprocessor, which must handle both unrestricted inter-core interleaving and intra-core out-of-order execution that produce weak outcomes not explainable by sequential execution, with the microarchitecture entering temporary states forbidden by the ISA. The method introduces a well-designed core specification that captures excess executions in a single instruction list, enabling a two-step proof decomposition: core refinement against this specification and system inclusion that serializes memory executions into the ISA. Experimental evidence shows that all proofs are mechanized in Rocq, with large language model (LLM) agents used to write proofs automatically, though the abstract does not disclose specific experimental results. This matters because it provides the first unbounded formal verification of an out-of-order multiprocessor against an in-order weak-memory ISA, addressing a critical gap in prior work.

PDF

Coherence in Control: Bridging Many-Core Mapping and Routing through Cost Unification

Guochu Xiong, Xiangzhong Luo, Weichen Liu 2026-07-23
Cache Coherence Microarchitecture Simulation

The problem is that existing many-core mapping and routing optimizations overlook cache coherence, causing a mismatch between optimization objectives and actual communication patterns. CoCo proposes a unified cost model integrating communication cost, coherence overhead, and load imbalance to jointly optimize task mapping and routing. Experiments show CoCo reduces link utilization by 88.46%, packet delay by 17.40%, and execution time by 17.58% compared to prior approaches. This matters because it demonstrates that coherence-aware co-optimization is critical for improving performance in data-intensive many-core systems.

PDF

High-Level Synthesis of Efficient Pipelines with Visibility Control

Jungin Rhee, Minseong Jang, Jaewoo Kim, Jeehoon Kang 2026-07-23
Design Space Exploration Agentic Architecture

The problem is that existing high-level synthesis (HLS) tools either lack fine-grained control over pipeline structure and hazard resolution or sacrifice sequential semantics to provide it. The method introduces an HLS tool built on visibility control, a novel programming abstraction that unifies hazard resolution strategies including stalling, bypassing, speculation, deferred commit, and register renaming within a sequential programming model. Experimental evidence shows that on in-order RISC-V cores, histograms, and an AES accelerator, compiled pipelines outperform HLS tools with sequential semantics and achieve power, performance, and area (PPA) comparable to hand-written RTL. This matters because it enables rapid design-space exploration for efficient pipelines without requiring RTL or concurrent programming models.

PDF

HyMCache: A KV Cache Framework for Multi-Turn LLM Serving with CXL-Hybrid Memory

Hakbeom Jang, Inho Song, Sam H. Noh, Jongryool Kim 2026-07-23
Prefetching Microarchitecture Simulation Cache LLM Serving

HyMCache addresses the problem of high memory costs in multi-turn LLM serving by proposing a KV-cache framework that uses CXL-hybrid memory (CXL-HM), combining a small in-device DRAM with large SSD-backed capacity. The method employs request-level prefix prefetching and opportunistic write buffering to stage latency-critical reads in device DRAM, exploiting the read-dominant and predictable access patterns of multi-turn KV-cache. Experimental evidence on a real CXL-HM prototype shows HyMCache outperforms local LMCache by 3.0x in single-node and 1.45x in PD-disaggregated serving under the same DRAM budget, and incurs about 30% lower performance than 1 TB distributed-DRAM Mooncake while using 16x less DRAM. This matters because it enables TB-scale shared KV-cache reuse at SSD-level cost, significantly reducing DRAM requirements for scalable multi-turn LLM serving.

PDF

A Flexible Sparsity-Aware FPGA Accelerator with Column-Wise Compression for Efficient CNN Inference

Amirhossein Zarei, Shervin Vakili 2026-07-23
Microarchitecture Simulation Sparsity Accelerator Inference

SparHiXcel-v2 addresses the problem of balancing sparsity flexibility and hardware efficiency in CNN acceleration on resource-constrained platforms. The method introduces a column-wise kernel compression technique within a scalable two-dimensional MAC array and a hardware-algorithm co-design framework with ordering optimization and multi-phase structured pruning. Experimental results on VGG16 and ResNet18 show SparHiXcel-v2 achieves over 2.5 TOPS and 210 GOP/s/W for VGG16 on an AMD Kintex UltraScale+ FPGA with modest accuracy loss. This matters because it enables efficient, flexible sparsity-aware CNN inference on cost-effective FPGAs, improving throughput and energy efficiency for edge deployments.

PDF

A Reinforcement-Learning-Augmented Liquid-Fueled Reactor Network Model for Predicting Lean Blowout in Gas Turbine Combustors

Philip John, Eloghosa Ikponmwoba, Pinaki Pal, Opeoluwa Owoyele 2026-07-23
Design Space Exploration Agentic Architecture Network

The problem is that existing methods for determining cluster boundaries in reactor network models rely on manual heuristics or distance-based metrics, which are not optimized for target metrics like lean blowout (LBO) prediction accuracy. The method introduces a reinforcement learning (RL) framework that uses a multi-stage clustering-classification strategy, where an actor-critic RL agent merges initial micro-clusters into optimal reactor zones in a goal-oriented manner. Experimental evidence from a validation study using a Jet-A mechanism (119 species, 841 reactions) shows that the RL framework improves predictive fidelity over k-means clustering and captures correct LBO trends while achieving substantial speedups relative to high-fidelity computational models. This matters because the RL-driven approach provides a computationally efficient reduced-order modeling technique that can complement high-fidelity simulations for rapid design-space exploration in gas turbine combustors.

PDF