Filtered by: Cache × Clear all

DGNA: Dissecting GPU NUMA Architecture through Microbenchmarking and Data Analysis

Changxi Liu, Yun Chen, Trevor E. Carlson 2026-07-23
Cache × Coherence Memory Hierarchy Numa Microarchitecture Simulation HPC

The problem is that modern GPU memory architectures, particularly NUMA mechanisms within L2 and DRAM, remain a black-box, hindering optimization and simulation. DGNA introduces a methodology using microbenchmarking and Gaussian mixture models to measure L2 and DRAM latency without relying on intrinsic instructions. Applied to NVIDIA's A100 and H100 GPUs, it reveals NUMA node architecture, SM-NUMA relationships, and NUMA-aware memory allocation strategies for cache coherence. This matters because it is the first work to detail GPU memory subsystem NUMA architecture, enabling better application optimization and architectural design.

PDF

Mapping Without Graphs: Learning Coherence Traffic for Task Placement

Guochu Xiong, Tianrui Ma, Weichen Liu 2026-07-23
Cache × Coherence Microarchitecture Simulation

Existing task mapping approaches rely on predefined task graphs that fail to capture coherence-induced interactions from shared data accesses, leading to suboptimal mappings. CoTM addresses this by constructing task graphs from dynamic coherence behavior and using a lightweight heuristic with a multi-start optimization strategy guided by a coherence-aware penalty function. Experimental results show CoTM reduces average link utilization by up to 47.85% and total energy consumption by up to 10.30% compared to existing approaches. This matters because it demonstrates that incorporating cache coherence into task mapping significantly improves performance and energy efficiency for future many-core NoC systems.

PDF

Coherence in Control: Bridging Many-Core Mapping and Routing through Cost Unification

Guochu Xiong, Xiangzhong Luo, Weichen Liu 2026-07-23
Cache × Coherence Microarchitecture Simulation

The problem is that existing many-core mapping and routing optimizations overlook cache coherence, causing a mismatch between optimization objectives and actual communication patterns. CoCo proposes a unified cost model integrating communication cost, coherence overhead, and load imbalance to jointly optimize task mapping and routing. Experiments show CoCo reduces link utilization by 88.46%, packet delay by 17.40%, and execution time by 17.58% compared to prior approaches. This matters because it demonstrates that coherence-aware co-optimization is critical for improving performance in data-intensive many-core systems.

PDF

HyMCache: A KV Cache Framework for Multi-Turn LLM Serving with CXL-Hybrid Memory

Hakbeom Jang, Inho Song, Sam H. Noh, Jongryool Kim 2026-07-23
Prefetching Microarchitecture Simulation Cache × LLM Serving

HyMCache addresses the problem of high memory costs in multi-turn LLM serving by proposing a KV-cache framework that uses CXL-hybrid memory (CXL-HM), combining a small in-device DRAM with large SSD-backed capacity. The method employs request-level prefix prefetching and opportunistic write buffering to stage latency-critical reads in device DRAM, exploiting the read-dominant and predictable access patterns of multi-turn KV-cache. Experimental evidence on a real CXL-HM prototype shows HyMCache outperforms local LMCache by 3.0x in single-node and 1.45x in PD-disaggregated serving under the same DRAM budget, and incurs about 30% lower performance than 1 TB distributed-DRAM Mooncake while using 16x less DRAM. This matters because it enables TB-scale shared KV-cache reuse at SSD-level cost, significantly reducing DRAM requirements for scalable multi-turn LLM serving.

PDF