Article URL: https://www.asayers.com/jetrelay/
Comments URL: https://news.ycombinator.com/item?id=44003708
Points: 4
# Comments: 1
Creată
11h
|
16 mai 2025, 12:40:13
Autentifică-te pentru a adăuga comentarii
Alte posturi din acest grup

I discovered that in LLM inference, keys and values in the KV cache have very different quantization sensitivities. Keys need higher precision than values to maintain quality.
I patched llama.cp
Article URL: https://clojurescript.org/news/2025-05-16-release




Article URL: https://bobacollection.staxmuseum.org/
Comments URL: https://news.y