Filtered by: Hardware × Clear all

FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications

Krish Agarwal, Zhuoming Chen, Yanyuan Qin, Zhenyu Gu 2026-07-22
Inference Compiler GPU Hardware × Model Parallelism Serving Agent

FlashRT addresses the problem of manually optimizing real-time multimodal application deployments across heterogeneous models and hardware. It introduces an agent harness that uses a chain-of-program paradigm to transform simple reference implementations into optimized multi-GPU deployments via iterative IR construction, validation, and measurement-gated optimization. On NVIDIA B200 GPUs, FlashRT achieves up to ~70x latency reduction and 2.8x throughput improvement, while on AMD MI355X GPUs it matches peak latency reduction and increases throughput improvement to 3.6x, outperforming expert implementations like vLLM-Omni by 65% in response latency. This matters because agent-driven optimization enables scalable, high-performance deployment across diverse hardware platforms without requiring hand-crafted expert tuning.

PDF

Hardware Mechanisms to Dynamically Throttle AI Performance

Haiyue Ma, Lauren Malek, Joseph Forzani, David Wentzlaff 2026-07-22
Microarchitecture Microarchitectural Simulation Hardware × AI

The problem is the lack of fine-grained, dynamic hardware mechanisms to limit AI performance for safety, as existing software safeguards can be bypassed. The method introduces four microarchitecture knobs—L2 size, L2 latency, L2 bandwidth, and shared memory port access rate—built from established primitives like cache way masking and credit-based rate limiting. Experimental evidence shows these knobs achieve up to 80% performance reduction at 1/8 resource availability with negligible cost (<10K flip flops) and fast stabilization (5-80K cycles). This matters because it provides a hardware-level last line of defense for controlling AI intent in critical systems, with multi-knob combinations enabling a broader range of performance targets.

PDF

ADEPT: Architecture-Driven Energy-Efficient CNN Fine-Tuning on PIM Accelerators

Pratyush Dhingra, Vibhanshu Sharma, Janardhan Rao Doppa, Partha Pratim Pande 2026-07-22
Hardware × Codesign Compiler Runtime Architecture Accelerator

ADEPT addresses the problem of high computational and memory overhead in CNN fine-tuning on PIM accelerators, caused by frequent off-chip memory access for intermediate activations. The method introduces a hardware-aware framework that adaptively trains models using a novel metric balancing gradient-based sensitivity with architecture-specific Energy-Delay Product (EDP). Experimental evidence shows ADEPT reduces total trainable parameters and off-chip data access during fine-tuning with minimal loss in predictive accuracy compared to full-parameter fine-tuning. This matters because it enables energy-efficient fine-tuning on PIM accelerators, making edge deployment of CNNs more practical without sacrificing model accuracy.

PDF