Filtered by: GPU × Clear all

FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications

Krish Agarwal, Zhuoming Chen, Yanyuan Qin, Zhenyu Gu 2026-07-22
Inference Compiler GPU × Hardware Model Parallelism Serving Agent

FlashRT addresses the problem of manually optimizing real-time multimodal application deployments across heterogeneous models and hardware. It introduces an agent harness that uses a chain-of-program paradigm to transform simple reference implementations into optimized multi-GPU deployments via iterative IR construction, validation, and measurement-gated optimization. On NVIDIA B200 GPUs, FlashRT achieves up to ~70x latency reduction and 2.8x throughput improvement, while on AMD MI355X GPUs it matches peak latency reduction and increases throughput improvement to 3.6x, outperforming expert implementations like vLLM-Omni by 65% in response latency. This matters because agent-driven optimization enables scalable, high-performance deployment across diverse hardware platforms without requiring hand-crafted expert tuning.

PDF

Harness Engineering for LLM-Driven GPU Kernel Generation

Yue Shui, Chenyu Ma, Hangfei Xu, Shengzhao Wen 2026-07-22
Language Model LLM GPU ×

LLM-driven GPU kernel generation is unreliable without a system to constrain, validate, and select candidate code. This paper introduces a harness-centered system that separates an evaluation harness from a profile-backed optimization controller to enforce compilation, correctness, and timing. Across five operator definitions, the system achieved mean-latency speedups over FlashInfer baselines ranging from 1.12x to 29.68x on NVIDIA Blackwell B200 GPUs. The findings demonstrate that expert-provided optimization directions and workload context remain critical for reliable AI-driven kernel optimization.

PDF

ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernels

Bingyang Wu, Chao Jin, Zili Zhang, Xinming Wei 2026-07-22
Attention Fp8 GPU × Serving LLM

ExpertPlex addresses the problem of inefficient resource utilization in serving large Mixture-of-Experts (MoE) LLMs, where existing prefill-decode disaggregation wastes resources due to coarse allocation and colocation suffers from phase interference and poor load tracking. The method shares massive MoE experts across phases while disaggregating attention modules, and introduces adaptive persistent kernels, attention-initiated MoE communication, and a tile-to-cluster model to optimize execution. Experiments serving MiniMax-M2.7 and GLM-5.1-FP8 show ExpertPlex improves goodput by up to 2.01× over instance-level disaggregation and 1.66× over colocation. This matters because it enables efficient, high-throughput serving of increasingly large MoE models without overprovisioning or phase interference.

PDF

Cross-Domain Acceleration of Open Modification Search: From Commodity Platforms to Emerging Memory and Storage Devices

Sumukh Pinge, Chang Eun Song, Po-Kai Hsu, Zheyu Li 2026-07-22
Search Data Workload Accelerator Architecture GPU ×

The problem is that open modification search (OMS) in mass spectrometry is data-intensive and limited by reference data movement, not computation. The method introduces a binary hyperdimensional computing (HDC)-based OMS formulation that reduces similarity evaluation to lightweight bitwise primitives and tolerates device-level non-idealities. Experimental evidence shows that memory- and storage-centric architectures deliver up to >100x speedup and >40,000x improvement in energy efficiency over commodity platforms. This matters because it identifies memory- and storage-centric architectures as a key breakthrough for large-scale, high-speed search acceleration.

PDF