Filtered by: Compiler × Clear all

FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications

Krish Agarwal, Zhuoming Chen, Yanyuan Qin, Zhenyu Gu 2026-07-22
Inference Compiler × GPU Hardware Model Parallelism Serving Agent

FlashRT addresses the problem of manually optimizing real-time multimodal application deployments across heterogeneous models and hardware. It introduces an agent harness that uses a chain-of-program paradigm to transform simple reference implementations into optimized multi-GPU deployments via iterative IR construction, validation, and measurement-gated optimization. On NVIDIA B200 GPUs, FlashRT achieves up to ~70x latency reduction and 2.8x throughput improvement, while on AMD MI355X GPUs it matches peak latency reduction and increases throughput improvement to 3.6x, outperforming expert implementations like vLLM-Omni by 65% in response latency. This matters because agent-driven optimization enables scalable, high-performance deployment across diverse hardware platforms without requiring hand-crafted expert tuning.

PDF

ADEPT: Architecture-Driven Energy-Efficient CNN Fine-Tuning on PIM Accelerators

Pratyush Dhingra, Vibhanshu Sharma, Janardhan Rao Doppa, Partha Pratim Pande 2026-07-22
Hardware Codesign Compiler × Runtime Architecture Accelerator

ADEPT addresses the problem of high computational and memory overhead in CNN fine-tuning on PIM accelerators, caused by frequent off-chip memory access for intermediate activations. The method introduces a hardware-aware framework that adaptively trains models using a novel metric balancing gradient-based sensitivity with architecture-specific Energy-Delay Product (EDP). Experimental evidence shows ADEPT reduces total trainable parameters and off-chip data access during fine-tuning with minimal loss in predictive accuracy compared to full-parameter fine-tuning. This matters because it enables energy-efficient fine-tuning on PIM accelerators, making edge deployment of CNNs more practical without sacrificing model accuracy.

PDF