Sam said yesterday that chatgpt handles ~700M weekly users. Meanwhile, I can't even run a single GPT-4-class model locally without insane VRAM or painfully slow speeds.
Sure, they have huge GPU clusters, but there must be more going on - model optimizations, sharding, custom hardware, clever load balancing, etc.
What engineering tricks make this possible at such massive scale while keeping latency low?
Curious to hear insights from people who've built large-scale ML systems.
Comments URL: https://news.ycombinator.com/item?id=44840728
Points: 51
# Comments: 36
Inicia sesión para agregar comentarios
Otros mensajes en este grupo.
![How to rig elections [video]](https://www.cdn5.niftycent.com/a/D/Z/w/p/G/6/how-to-rig-elections-video.webp)
Hello everyone. This is Yujong from the Hyprnote team (https://github.com/fastrepl/hyprnote).
We built OWhisper for 2 reasons: (
Hi all, Aram and Eduard here - authors of Modelence (https://github.com/modelence/modelence), an all-in-one backend platform for
Article URL: https://pluralistic.net/2025/08/14/bellovin/
Comments URL: ht
