Show HN: Blast – Fast, multi-threaded serving engine for web browsing AI agents

Hi HN!

BLAST is a high-performance serving engine for browser-augmented LLMs, designed to make deploying web-browsing AI easy, fast, and cost-manageable.

The goal with BLAST is to ultimately achieve google search level latencies for tasks that currently require a lot of typing and clicking around inside a browser. We're starting off with automatic parallelism, prefix caching, budgeting (memory and LLM cost), and an OpenAI-Compatible API but have a ton of ideas in the pipe!

Website & Docs: https://blastproject.org/ https://docs.blastproject.org/

MIT-Licensed Open-Source: https://github.com/stanford-mast/blast

Hope some folks here find this useful! Please let me know what you think in the comments or ping me on Discord.

— Caleb (PhD student @ Stanford CS)

Comments URL: https://news.ycombinator.com/item?id=43872761

Points: 43

# Comments: 16

https://github.com/stanford-mast/blast