FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications

Krish Agarwal, Zhuoming Chen, Yanyuan Qin, Zhenyu Gu 2026-07-22

Inference Compiler GPU Hardware Model Parallelism Serving Agent

FlashRT addresses the problem of manually optimizing real-time multimodal application deployments across heterogeneous models and hardware. It introduces an agent harness that uses a chain-of-program paradigm to transform simple reference implementations into optimized multi-GPU deployments via iterative IR construction, validation, and measurement-gated optimization. On NVIDIA B200 GPUs, FlashRT achieves up to ~70x latency reduction and 2.8x throughput improvement, while on AMD MI355X GPUs it matches peak latency reduction and increases throughput improvement to 3.6x, outperforming expert implementations like vLLM-Omni by 65% in response latency. This matters because agent-driven optimization enables scalable, high-performance deployment across diverse hardware platforms without requiring hand-crafted expert tuning.

PDF

Hardware Mechanisms to Dynamically Throttle AI Performance

Haiyue Ma, Lauren Malek, Joseph Forzani, David Wentzlaff 2026-07-22

Microarchitecture Microarchitectural Simulation Hardware AI

The problem is the lack of fine-grained, dynamic hardware mechanisms to limit AI performance for safety, as existing software safeguards can be bypassed. The method introduces four microarchitecture knobs—L2 size, L2 latency, L2 bandwidth, and shared memory port access rate—built from established primitives like cache way masking and credit-based rate limiting. Experimental evidence shows these knobs achieve up to 80% performance reduction at 1/8 resource availability with negligible cost (<10K flip flops) and fast stabilization (5-80K cycles). This matters because it provides a hardware-level last line of defense for controlling AI intent in critical systems, with multi-knob combinations enabling a broader range of performance targets.

PDF

Formal Verification of an Out-of-Order Multiprocessor against an In-Order Weak-Memory ISA

Janggun Lee, Jeehoon Kang 2026-07-22

Microarchitecture Microarchitectural Simulation

The problem is that verifying an out-of-order multiprocessor against a weak-memory ISA is challenging due to inter-core interleaving and intra-core out-of-order execution, which produce weak outcomes and excess microarchitectural states. The method introduces a core specification capturing excess executions in a single instruction list, enabling a two-step proof: core refinement against this specification and system inclusion that serializes memory accesses into the ISA. Experimental evidence shows all proofs are mechanized in Rocq, with large language model (LLM) agents used to write proofs automatically. This matters because it is the first unbounded formal verification of an out-of-order multiprocessor against an in-order weak-memory ISA, addressing a critical gap in prior work.

PDF

Harness Engineering for LLM-Driven GPU Kernel Generation

Yue Shui, Chenyu Ma, Hangfei Xu, Shengzhao Wen 2026-07-22

Language Model LLM GPU

LLM-driven GPU kernel generation is unreliable without a system to constrain, validate, and select candidate code. This paper introduces a harness-centered system that separates an evaluation harness from a profile-backed optimization controller to enforce compilation, correctness, and timing. Across five operator definitions, the system achieved mean-latency speedups over FlashInfer baselines ranging from 1.12x to 29.68x on NVIDIA Blackwell B200 GPUs. The findings demonstrate that expert-provided optimization directions and workload context remain critical for reliable AI-driven kernel optimization.

PDF

ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernels

Bingyang Wu, Chao Jin, Zili Zhang, Xinming Wei 2026-07-22

Attention Fp8 GPU Serving LLM

ExpertPlex addresses the problem of inefficient resource utilization in serving large Mixture-of-Experts (MoE) LLMs, where existing prefill-decode disaggregation wastes resources due to coarse allocation and colocation suffers from phase interference and poor load tracking. The method shares massive MoE experts across phases while disaggregating attention modules, and introduces adaptive persistent kernels, attention-initiated MoE communication, and a tile-to-cluster model to optimize execution. Experiments serving MiniMax-M2.7 and GLM-5.1-FP8 show ExpertPlex improves goodput by up to 2.01× over instance-level disaggregation and 1.66× over colocation. This matters because it enables efficient, high-throughput serving of increasingly large MoE models without overprovisioning or phase interference.

PDF

Mapping Without Graphs: Learning Coherence Traffic for Task Placement

Guochu Xiong, Tianrui Ma, Weichen Liu 2026-07-22

Cache Coherence Microarchitecture Simulation

The problem is that existing task mapping approaches rely on predefined task graphs that fail to capture coherence-induced interactions from shared data accesses, leading to suboptimal mappings. CoTM addresses this by constructing task graphs inferred from dynamic coherence behavior and using a lightweight heuristic with a multi-start optimization strategy guided by a coherence-aware penalty function. Experimental results show CoTM reduces average link utilization by up to 47.85% and total energy consumption by up to 10.30% compared to existing approaches. This matters because it demonstrates that incorporating cache coherence into task mapping significantly improves performance and energy efficiency for future many-core NoC systems.

PDF

Coherence in Control: Bridging Many-Core Mapping and Routing through Cost Unification

Guochu Xiong, Xiangzhong Luo, Weichen Liu 2026-07-22

Cache Coherence Microarchitecture Simulation

The problem is that existing many-core mapping and routing approaches overlook cache coherence, causing a mismatch between optimization objectives and actual communication patterns. CoCo proposes a unified cost model integrating communication cost, coherence overhead, and load imbalance to jointly optimize mapping and routing. Experiments show CoCo reduces link utilization by 88.46%, packet delay by 17.40%, and execution time by 17.58% over existing methods. This matters because it demonstrates that coherence-aware co-optimization is essential for improving performance in data-intensive many-core systems.

PDF

SEAM-V: A Hybrid-Decoupled RISC-V Vector Processor with Backend-Visible EP Context for Sustained Vector Throughput

Weiying Wang, Zhiwei Zhang 2026-07-22

Prefetching Microarchitecture Simulation

SEAM-V addresses the problem of vector-instruction supply gaps and scalar-side progression delays in tightly coupled RISC-V Vector Extension implementations. The method introduces a hybrid-decoupled architecture that forms continuous execute packets via task-level decoupling, local instruction supply, and VLIW-style packing, with backend-visible EP context for hazard suppression and prefetching. Cycle-accurate RTL evaluation shows a geometric-mean speedup of 1.34x across 17 kernels compared to an Ara-based tightly coupled implementation, with up to 3x speedup for short-vector kernels at AVL=32. This matters because it demonstrates a scalable approach to sustaining vector throughput in data-parallel workloads without relying solely on scalar core progression.

PDF

ADEPT: Architecture-Driven Energy-Efficient CNN Fine-Tuning on PIM Accelerators

Pratyush Dhingra, Vibhanshu Sharma, Janardhan Rao Doppa, Partha Pratim Pande 2026-07-22

Hardware Codesign Compiler Runtime Architecture Accelerator

ADEPT addresses the problem of high computational and memory overhead in CNN fine-tuning on PIM accelerators, caused by frequent off-chip memory access for intermediate activations. The method introduces a hardware-aware framework that adaptively trains models using a novel metric balancing gradient-based sensitivity with architecture-specific Energy-Delay Product (EDP). Experimental evidence shows ADEPT reduces total trainable parameters and off-chip data access during fine-tuning with minimal loss in predictive accuracy compared to full-parameter fine-tuning. This matters because it enables energy-efficient fine-tuning on PIM accelerators, making edge deployment of CNNs more practical without sacrificing model accuracy.

PDF

Isolation Failure From Shared Storage: Characterizing and Exploiting Page-Cache SCA Leakage Across Containers and VMs

Alon Abudraham, Xingyu Chen, Itamar Levi, Ari Trachtenberg 2026-07-22

Microarchitecture Microarchitectural Simulation Cache

The problem is that modern cloud platforms share the host page cache across containers and VMs, creating an OS-mediated microarchitectural timing side channel. The method involves unprivileged timing measurements to detect page-cache residency across Docker, gVisor, Kata Containers, and QEMU/KVM with various I/O paths. Experimental evidence shows the timing signal persists with shared host-cacheable file-backed objects like OverlayFS and virtio-fs, but direct I/O and dedicated block devices attenuate it; a case study recovers coarse-grained activity from a WordPress/MySQL deployment. This matters because it characterizes page-cache attacks as OS-mediated channels, motivating coordinated hardware, virtualization, and OS support for timing isolation.

PDF

HyMCache: A KV Cache Framework for Multi-Turn LLM Serving with CXL-Hybrid Memory

Hakbeom Jang, Inho Song, Sam H. Noh, Jongryool Kim 2026-07-22

Prefetching Microarchitecture Simulation Cache LLM Serving

HyMCache addresses the problem of high memory costs in multi-turn LLM serving by proposing a KV-cache framework that uses CXL-hybrid memory (CXL-HM), combining a small in-device DRAM with large SSD-backed capacity. The method exploits the read-dominant, predictable, and append-only nature of multi-turn KV-cache access, using request-level prefix prefetching and opportunistic write buffering to stage latency-critical reads in device DRAM. Experimental evidence on a real CXL-HM prototype shows that under the same DRAM budget, HyMCache outperforms local LMCache by 3.0x in single-node serving and 1.45x in PD-disaggregated serving, and compared to 1 TB distributed-DRAM Mooncake, it incurs about 30% lower performance but uses 16x less DRAM. This matters because it enables TB-scale SSD-backed KV reuse at DRAM-scale efficiency and SSD-level cost, significantly reducing memory expenses for long-context and agentic LLM workloads.

PDF

A Flexible Sparsity-Aware FPGA Accelerator with Column-Wise Compression for Efficient CNN Inference

Amirhossein Zarei, Shervin Vakili 2026-07-22

Microarchitecture Simulation Sparsity Accelerator Inference

Problem: Efficient CNN inference on resource-constrained FPGAs is challenged by the irregularity of sparsity patterns, where unstructured sparsity causes hardware inefficiencies and structured sparsity sacrifices flexibility. Method: SparHiXcel-v2 introduces a scalable two-dimensional MAC array with column-wise kernel compression for irregular sparsity, plus a hardware-algorithm co-design framework with ordering optimization and multi-phase structured pruning and revival. Finding: On a cost-effective AMD Kintex UltraScale+ FPGA, SparHiXcel-v2 achieves over 2.5 TOPS and 210 GOP/s/W for VGG16, and over 1.1 TOPS and 72 GOP/s/W for ResNet18 in structured sparsity mode with modest accuracy loss. Why it matters: This work provides a flexible, energy-efficient FPGA accelerator that balances sparsity flexibility and hardware efficiency, enabling practical CNN deployment on resource-constrained platforms.

PDF

A Reinforcement-Learning-Augmented Liquid-Fueled Reactor Network Model for Predicting Lean Blowout in Gas Turbine Combustors

Philip John, Eloghosa Ikponmwoba, Pinaki Pal, Opeoluwa Owoyele 2026-07-22

Design Space Exploration Agentic Architecture Network

The problem is that existing methods for determining cluster boundaries in reactor network models rely on manual heuristics or distance-based metrics, which are not optimized for target metrics like lean blowout (LBO) prediction accuracy. The method introduces a reinforcement learning (RL) framework that uses a multi-stage clustering-classification strategy, where an actor-critic RL agent merges micro-clusters into optimal reactor zones to explicitly improve LBO prediction. Experimental evidence using a Jet-A mechanism (119 species, 841 reactions) shows the RL framework improves predictive fidelity over k-means clustering and captures correct LBO trends while achieving substantial speedups relative to high-fidelity simulations. This matters because the RL-driven approach provides a computationally efficient reduced-order modeling technique that can complement high-fidelity simulations for rapid design-space exploration in gas turbine combustors.

PDF

High-Level Synthesis of Efficient Pipelines with Visibility Control

Jungin Rhee, Minseong Jang, Jaewoo Kim, Jeehoon Kang 2026-07-22

Design Space Exploration Agentic Architecture

The problem is that existing high-level synthesis (HLS) tools for pipelining either lack fine-grained control over pipeline structure and hazard resolution or sacrifice sequential semantics to provide it. The method introduces an HLS tool based on visibility control, a programming abstraction that unifies hazard resolution strategies such as stalling, bypassing, speculation, deferred commit, and register renaming within a sequential programming model. Experimental evidence on in-order RISC-V cores, histograms, and an AES accelerator shows that compiled pipelines outperform HLS tools with sequential semantics and achieve power, performance, and area (PPA) comparable to hand-written RTL. This matters because it enables rapid design-space exploration for efficient pipelines without requiring RTL or concurrent programming models.

PDF

Cross-Domain Acceleration of Open Modification Search: From Commodity Platforms to Emerging Memory and Storage Devices

Sumukh Pinge, Chang Eun Song, Po-Kai Hsu, Zheyu Li 2026-07-22

Search Data Workload Accelerator Architecture GPU

The problem is that open modification search (OMS) in mass spectrometry is data-intensive and limited by reference data movement, not computation. The method introduces a binary hyperdimensional computing (HDC)-based OMS formulation that reduces similarity evaluation to lightweight bitwise primitives and tolerates device-level non-idealities. Experimental evidence shows that memory- and storage-centric architectures deliver up to >100x speedup and >40,000x improvement in energy efficiency over commodity platforms. This matters because it identifies memory- and storage-centric architectures as a key breakthrough for large-scale, high-speed search acceleration.

PDF