Filtered by: Hardware × Clear all

kvcache-ai/ktransformers

kvcache-ai 2026-07-20
ROCm Numa Cache GPU Hardware × Inference Npu Quantization
kvcache-ai/ktransformers Python
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
18,506 1,457

KTransformers implements a flexible framework for CPU-GPU heterogeneous LLM inference and fine-tuning, with a focus on MoE models and quantized kernels. It relates to agentic architecture and hardware/software co-design by enabling efficient deployment of large models on consumer hardware through heterogeneous expert placement and CPU-optimized AMX/AVX operations. The repository has a strong upward star trend, currently at 18,506 total stars with 360 stars today. The original paper is published in the ACM Digital Library under DOI 10.1145/3731569.3764843.

Code

CoG-Guided Weight Correction for Fault-Tolerant Deep Neural Networks

Bahram Parchekani, Samira Nazari, Ali Azarpeyvand, Mohammad Hasan Ahmadilivani 2026-07-20
Neural Network Hardware ×

The paper addresses the vulnerability of DNN weights to hardware faults in safety-critical systems. It introduces a Center of Gravity (CoG)-guided method that corrects faulty weights using spatial, distance-aware rules without retraining or architecture changes. Fault injection experiments on LSTM networks (StageNet, MTFNet) show up to 230x and 6.41x fault tolerance improvements at a BER of 10^{-3}, and on CNNs (ResNet-18, VGG-16) up to 49.55x and 20.79x improvements, with negligible accuracy loss. This matters because it is the first application of CoG to weight tensors, offering a practical, retraining-free approach to enhance DNN reliability in safety-critical applications.

PDF