Filtered by: Scheduling × Clear all

Agentic CPU-GPU Scheduling for Heterogeneous AI Workloads

Tianxi Lu, Sherief Reda 2026-07-27
LLM Training GPU Runtime Scheduling × Agentic AI Workload

The problem is that existing frameworks assign all GPU-capable AI tools to the GPU by default, which is suboptimal for heterogeneous workloads. The method profiles 19 AI tools across GPU and CPU, formulates device scheduling under a VRAM budget, and presents an agentic scheduler pairing an LLM agent with an algorithmic runtime monitor. Experimental evidence shows the agentic scheduler reaches brute-force optimal mapping in all 13 scenarios, matching the best classical baseline on accuracy while outperforming HEFT, StarPU, and the all-GPU policy. This matters because it enables efficient, zero-training scheduling for agentic AI systems on shared GPU/CPU infrastructure.

PDF

Sparse by Command: Task-Conditional Compute Skipping for Multi-Task Inference Accelerators

Afzal Ahmad, Gaoyu Mao, Shoubo Hu, Hui-Ling Zhen 2026-07-27
Inference Accelerator Hardware Network Scheduling × Architecture Design Sparsity

Multi-task inference models waste computation by executing identical operations regardless of the active task. We propose a HW/SW co-designed approach where a lightweight gating network predicts per-tile binary execution masks conditioned on the task command, enabling zero-overhead skipping of masked tiles. On a closed-loop visuomotor driving task in CARLA, task-conditional sparsity reduces FLOPs by 66-76%, on-device latency by 51-59% (2.1-2.4x speedup), and energy per inference from 263 to 108-128mJ. This work demonstrates that leveraging the task command as a free signal can significantly improve efficiency of multi-task inference accelerators without altering model architecture or inference pipeline.

PDF