TokenDagger is a drop-in replacement for OpenAI’s Tiktoken (the tokenizer behind Llama 3, Mistral, GPT-3.*, etc.). It’s written in C++ 17 with thin Python bindings, keeps the exact same BPE vocab/special-token rules, and focuses on raw speed.
I’m teaching myself LLM internals by re-implementing the stack from first principles. Profiling TikToken’s Python/Rust implementation showed a lot of time was spent doing regex matching. Most of my perf gains come from a) using a faster jit-compiled r
Article URL: https://www.fosskers.ca/en/blog/cl-windows
Comments URL: https://news.ycombinator.com/item?id=44401358
Points: 27
# Comments: 1
Article URL: https://www.systemicmisalignment.com/
Comments URL: https://news.ycombinator.com/item?id=44418692
Points: 28
# Comments: 14
Article URL: https://letsencrypt.org/2025/06/26/expiration-notification-service-has-ended/
Comments URL: https://news.ycombinator.com/item?id=44419496
Points: 38
# Comments: 12
https://letsencrypt.org/2025/06/26/expiration-notification-service-has-ended/
Article URL: https://notes.eatonphil.com/2025-06-28-want-to-meet-people-charge-them.html
Comments URL: https://news.ycombinator.com/item?id=44419986
Points: 50
# Comments: 6
https://notes.eatonphil.com/2025-06-28-want-to-meet-people-charge-them.html
Article URL: https://www.imperialviolet.org/2025/06/29/cgm.html
Comments URL: https://news.ycombinator.com/item?id=44418520
Points: 46
# Comments: 14
Article URL: https://github.com/BitFunnel/NativeJIT
Comments URL: https://news.ycombinator.com/item?id=44418840
Points: 18
# Comments: 6
Article URL: https://gridfinity.xyz/
Comments URL: https://news.ycombinator.com/item?id=44419091
Points: 25
# Comments: 3