Filtered by: Simulation × Clear all

CW-Ghost: Search-Free Granularity Selection for Helper-Thread Prefetching via Capacity Windows

Ya Zhang, Tong Lei, Yao Chen, Yonggang Che 2026-07-30
Prefetching Microarchitecture Simulation × Search

CW-Ghost addresses the problem of costly granularity selection in helper-thread prefetching by proposing a search-free method. The method uses a single offline profiling run to estimate demand cache line fill volume and combines it with a cache capacity budget to derive a Capacity Window for iteration granularity. Experimental evidence on Intel and AMD platforms shows geometric mean speedups of 1.54x and 1.33x over original programs, and 15.8% and 10.8% improvements over Ghost Threading, achieving over 99% of optimal performance. This matters because it demonstrates that cache capacity constraints can effectively and efficiently guide helper-thread prefetching granularity without exhaustive configuration search.

PDF

Investigating reservoir computing for branch predictionin pipelined processors using emerging CMOS memristor devices

Harvey Samuel George Johnson, Sendy Phang 2026-07-30
Branch Prediction Physics App Ph Microarchitecture Simulation ×

The problem is that conventional branch predictors in pipelined processors struggle with adaptability and speed, motivating exploration of reservoir computing (RC) as an alternative. The method involves developing a novel memristor-based RC design framework, implemented in System Verilog and Verilog-AMS, and tested on the Dhrystone benchmark for the RISC-V RV64GC ISA. Experimental evidence shows that while the RC framework achieves impressive overall prediction accuracy, it is 15x slower to adapt to changes in branching behavior compared to the state-of-the-art TAGE predictor. This matters because RC shows promise for branch prediction but requires further refinement to overcome adaptability shortfalls before it can compete with existing predictors.

PDF

DualDecoder: Accelerate Long Context LLM Inference by Predictive Prefetch

Zuning Liang, Zhiyi Yao, Qi Chen, Yuedong Xu 2026-07-30
Prefetching Microarchitecture Simulation × LLM Inference

DualDecoder addresses the memory bottleneck in long-context LLM inference caused by the growing KV cache. It predicts critical KV entries for the next token from the preceding speculated token, enabling proactive prefetching from host memory. Experiments show up to 2.62× throughput improvement over state-of-the-art systems without degrading latency or model quality. This matters because it eliminates GPU memory overhead from auxiliary states, making high-concurrency long-context inference more efficient for agentic applications.

PDF

InferScale: GPU-Native KV Injection for Personalized LLM Serving

Peter Li, Prashant Pandey 2026-07-30
Memory Microarchitecture Simulation × GPU Cache LLM Serving

InferScale addresses the problem of increasing time-to-first-token (TTFT) in personalized LLM serving due to repeated prefilling of persistent user memory. The method precomputes KV representations for each memory fact, stores them on the GPU with semantic embeddings, and injects them directly into vLLM's paged cache using Chunked RoPE and Context-Window Encoding. On LoCoMo with three open-weight models, InferScale reduces TTFT by 72-79% (3.6-4.8x) at k=50, achieves 60.3% accuracy versus 63.3% for Mem0, and delivers 3.7-4.5x throughput under concurrent load. This matters because reusable KV state decouples memory-conditioned serving latency from retrieved-context size, enabling scalable personalized LLM serving without engine modifications or fine-tuning.

PDF