Hacker news

Article URL: https://cacm.acm.org/news/the-collapse-of-gpt/

Comments URL: https://news.ycombinator.com/item?id=44010705

Points: 18

# Comments: 7

https://cacm.acm.org/news/the-collapse-of-gpt/

I discovered that in LLM inference, keys and values in the KV cache have very different quantization sensitivities. Keys need higher precision than values to maintain quality.

I patched llama.cpp to enable different bit-widths for keys vs. values on Apple Silicon. The results are surprising:

- K8V4 (8-bit keys, 4-bit values): 59% memory reduction with only 0.86% perplexity loss - K4V8 (4-bit keys, 8-bit values): 59% memory reduction but 6.06% perplexity loss - The configurations use the

Article URL: https://clojurescript.org/news/2025-05-16-release

Comments URL: https://news.ycombinator.com/item?id=44009464

Points: 32

# Comments: 2

https://clojurescript.org/news/2025-05-16-release

Article URL: https://aeon.co/essays/who-needs-ai-text-generation-when-theres-erasmus-of-rotterdam

Comments URL: https://news.ycombinator.com/item?id=44009621

Points: 14

# Comments: 1

https://aeon.co/essays/who-needs-ai-text-generation-when-theres-erasmus-of-rotterdam

Article URL: https://cloud.google.com/blog/products/databases/techniques-for-improving-text-to-sql

Comments URL: https://news.ycombinator.com/item?id=44009848

Points: 10

# Comments: 1

https://cloud.google.com/blog/products/databases/techniques-for-improving-text-to-sql

Article URL: https://www.quantamagazine.org/new-superdiffusion-proof-probes-the-mysterious-math-of-turbulence-20250516/

Comments URL: https://news.ycombinator.com/item?id=44007714

Points: 17

# Comments: 2

https://www.quantamagazine.org/new-superdiffusion-proof-probes-the-mysterious-math-o

Article URL: https://bobacollection.staxmuseum.org/

Comments URL: https://news.ycombinator.com/item?id=44007854

Points: 15

# Comments: 4

https://bobacollection.staxmuseum.org/

Article URL: https://www.lrb.co.uk/the-paper/v47/n09/david-thomson/cool-tricking

Comments URL: https://news.ycombinator.com/item?id=44008412

Points: 16

# Comments: 8

https://www.lrb.co.uk/the-paper/v47/n09/david-thomson/cool-tricking

Article URL: https://www.nytimes.com/2025/05/15/business/trump-debanking-crypto.html

Comments URL: https://news.ycombinator.com/item?id=44008525

Points: 16

# Comments: 4

https://www.nytimes.com/2025/05/15/business/trump-debanking-crypto.html

Article URL: https://twitter.com/jonathan_blow/status/1923414922484232404

Comments URL: https://news.ycombinator.com/item?id=44008538

Points: 4

# Comments: 1

https://twitter.com/jonathan_blow/status/1923414922484232404