Filtered by: Hardware × Clear all

The Fabric Is the Cluster Driver: Cross-Layer eBPF Policies for GPU-CXL Fabrics

Yiwei Yang, Andi Quinn 2026-07-30
Attention LLM Compiler GPU Hardware × Cxl Runtime Tensor

fabric_ext introduces a cross-layer eBPF middleware compiler and runtime for enforcing extensible OS policies across GPU-CXL fabrics. The method uses a semantic movement graph abstraction that describes data movement characteristics and is compiled into per-device eBPF programs, BPF maps, and runtime artifacts for bpftime and dputime. Experimental evidence from the LLM prefill stress case shows that fabric_ext can manage attention and FFN dataflows crossing GPU tensor execution, DPU/NIC event execution, and CXL switch-local islands. This matters because it enables unified, data-driven policy enforcement across heterogeneous fabric components, improving observability and control in disaggregated memory systems.

PDF

AgenticCANN: Automated Ascend C Operator Generation via Knowledge-Augmented Agentic Evolution

Junhao Qiu, Zidong Wang, Yansong Sun, Zhitong Ma 2026-07-30
Inference Language Model LLM Hardware × Agentic

AgenticCANN addresses the problem of automated Ascend C operator generation for NPUs, which requires deep hardware expertise and faces a severe platform knowledge deficit. The method introduces a knowledge-augmented agentic evolution framework with a knowledge-orchestrated generation system and a stage-adaptive agentic evolution strategy to dynamically align LLM interaction modes. Experiments on Huawei Ascend 910B across six operators show 90-100% feasibility on elementwise and normalization operators, 56% on fusion operators, and up to 6.65× speedup on 1B Pangu model inference kernels, with knowledge injection monotonically improving feasibility from 57% to 86%. This matters because it enables automated, high-performance operator synthesis in low-corpus NPU environments, overcoming the unique challenges of the Ascend C programming model.

PDF

Beyond Prefill-Decode Disaggregation: Dissecting LLM Inference for Heterogeneous Platforms via Dynamic Operator Scheduling

Jiaqi Yang, Jiayi Li, Yihan Fu, Hongxiao Zhao 2026-07-30
Hardware × Roofline Codesign Compiler Runtime HPC LLM Inference

The problem is that prefill-decode disaggregation and roofline-based operator placement are insufficient for partitioning LLM inference across heterogeneous systems due to workload shape, device contention, and weight layout constraints. DOPS is a hardware-aware, closed-loop framework that uses a stage-aware DAG, a Bifocal scheduler for dynamic operator placement, and a Weight Layout Arbiter (WLA) for selecting efficient weight layouts. On heterogeneous platforms combining NPUs and PIM devices, Bifocal achieves geometric-mean speedups of 1.20× to 2.23× over the PD baseline, with WLA adding 1.28× to 1.33× further speedup. This matters because DOPS enables systematic analysis of workload sensitivity and hardware scalability, improving LLM serving efficiency on diverse heterogeneous hardware.

PDF