AI hardware and kernels
8 stories. An explainer is being written.
AI hardware and kernels in the news
llama.cpp patches Hexagon NPU attention bug in b11696
A single fix for Qualcomm's HMX flash-attention sinks ships in the latest build
Meta details how it made AMD GPUs production-ready for training
Porting recommendation models from Nvidia to AMD took more than a driver swap
AWS details MoE kernel fixes for SageMaker, upstreams them to vLLM
Three upstreamed vLLM fixes squeeze more throughput from Hopper GPUs
llama.cpp tunes CUDA kernel for Gated DeltaNet state updates
Release b11494 defaults GDN state updates to four columns per warp, a CUDA scheduling change in PR #30087.
llama.cpp adds BF16 support for XIELU on CUDA
Release b11457 closes a gap in the CUDA kernel, letting XIELU activations run in bfloat16 on Nvidia GPUs.
Benchmark finds LLM-generated GPU kernels fail in production
FastKernels shows kernel-level speedups mostly vanish once agents' code runs inside real models.
Study: TPU speed gains for AlphaFold2 vanish without proper batching
JAX's default path leaves seven of eight TPU chips idle. That erases the hardware's cost edge over a GPU.
Study maps when KV cache, not model weights, bottlenecks LLM inference
A paper derives exact context lengths where cache traffic overtakes weight traffic on H100, B200 and MI300X.