TileSight: A First-Principles Tile-Centric Analytical GPU Performance Model from Cores to Clusters

Zhiwen Mo, Yu Cheng, Lei Wang, Zhengju Tang 2026-07-27
Triton Cache Hierarchy CUDA Roofline Microarchitecture Simulation Codesign

TileSight addresses the problem that existing GPU performance tooling relies on coarse roofline bounds, opaque ML predictors, or post-hoc profilers, failing to support tile-centric programming now dominant in AI workloads. The method introduces a first-principles analytical model that uses the tile as a unified abstraction across intra-core compute-memory overlap, inter-core cache hierarchy, and cross-device inter-node communication layers. On A100, H200, B200, and B6000, TileSight predicts single-GPU kernel latency with 12.35% MAPE, L2 cache hit rates within one percentage point, and achieves 16.18% wMAPE on fused distributed kernels up to 32 GPUs. This matters because it provides an interpretable, transferable, and accurate alternative to opaque predictors and post-hoc profilers, enabling tile configuration optimization competitive with vendor and expert baselines.

PDF

Agentic CPU-GPU Scheduling for Heterogeneous AI Workloads

Tianxi Lu, Sherief Reda 2026-07-27
LLM Training GPU Runtime Scheduling Agentic AI Workload

The problem is that existing frameworks assign all GPU-capable AI tools to the GPU by default, which is suboptimal for heterogeneous workloads. The method profiles 19 AI tools across GPU and CPU, formulates device scheduling under a VRAM budget, and presents an agentic scheduler pairing an LLM agent with an algorithmic runtime monitor. Experimental evidence shows the agentic scheduler reaches brute-force optimal mapping in all 13 scenarios, matching the best classical baseline on accuracy while outperforming HEFT, StarPU, and the all-GPU policy. This matters because it enables efficient, zero-training scheduling for agentic AI systems on shared GPU/CPU infrastructure.

PDF

Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing

Sadegh Majidi, Niloofar Mireshghallah, Kazem Taram 2026-07-27
Attention Inference Language Model Transformer Hardware Architecture

LeakyLMs introduces a set of attacks that leak proprietary model architecture and inference optimization details from production language models using only per-token generation timing. The method builds a detailed timing model for NVIDIA GPUs and performs a search over the architecture space to recover properties like transformer layers, hidden dimension size, and attention heads. Experimental evidence shows that for Llama models, the near-correct architectural configuration appears in the top-10 guesses over 90% of the time, and the attack successfully detects speculative decoding in Google Gemini Flash 2.5. This matters because it demonstrates that sensitive model and deployment information can be inferred remotely via timing side channels, posing a significant security risk to proprietary language model services.

PDF

Sparse by Command: Task-Conditional Compute Skipping for Multi-Task Inference Accelerators

Afzal Ahmad, Gaoyu Mao, Shoubo Hu, Hui-Ling Zhen 2026-07-27
Inference Accelerator Hardware Network Scheduling Architecture Design Sparsity

Multi-task inference models waste computation by executing identical operations regardless of the active task. We propose a HW/SW co-designed approach where a lightweight gating network predicts per-tile binary execution masks conditioned on the task command, enabling zero-overhead skipping of masked tiles. On a closed-loop visuomotor driving task in CARLA, task-conditional sparsity reduces FLOPs by 66-76%, on-device latency by 51-59% (2.1-2.4x speedup), and energy per inference from 263 to 108-128mJ. This work demonstrates that leveraging the task command as a free signal can significantly improve efficiency of multi-task inference accelerators without altering model architecture or inference pipeline.

PDF