Show HN: Zeekstd – Rust Implementation of the ZSTD Seekable Format

Hello,

I would like to share a Rust implementation of the Zstandard seekable format I've been working on.

Regular zstd compressed files consist of a single frame, meaning you have to start decompression at the beginning. The seekable format splits compressed data into a series of independent frames, each compressed individually, so that decompression of a section in the middle of an archive only requires zstd to decompress at most a frame's worth of extra data, instead of the entire archive.

I started working with the seekable format because I wanted to resume downloads of big zstd compressed files that are decompressed and written to disk on the fly. At first I created and used bindings to the C functions that are available upstream[1], however, I stumbled over the first segfault rather quickly (it's now fixed) and found out that the functions only allow basic things. After looking closer at the upstream implementation, I noticed that is uses functions of the core API that are now deprecated and it doesn't allow access to low-level (de)compression contexts. To me it looks like a PoC/demo implementation that isn't maintained the same way as the zstd core API, probably that's also the reason it's in the contrib directory.

My use-case seemed to require a complete rewrite of the seekable format, so I decided to implement it from scratch in Rust using bindings to the advanced zstd compression API, available from zstd 1.4.0.

The result is a single dependency library crate[2], and a CLI crate[3] for the seekable format that feels similar to the regular zstd tool.

Any feedback is highly appreciated!

[1]: https://github.com/facebook/zstd/tree/dev/contrib/seekable_f... [2]: https://crates.io/crates/zeekstd [3]: https://github.com/rorosen/zeekstd/tree/main/cli


Comments URL: https://news.ycombinator.com/item?id=44284871

Points: 56

# Comments: 3

https://github.com/rorosen/zeekstd

Utworzony 23d | 16 cze 2025, 12:50:02


Zaloguj się, aby dodać komentarz

Inne posty w tej grupie

Google fails to dismiss wiretapping claims on SJ, settles with app users

https://www.bloomberglaw.com/api/v1/public/download/X7QEMPLP9PM8MTA9A06HJHGH0V8/11?public_link=FrascovFloHealthIncDocketNo321cv00757NDCalJan292021CourtDocket


Comments URL:

9 lip 2025, 21:50:07 | Hacker news
Let Kids Be Loud
9 lip 2025, 21:50:06 | Hacker news
Show HN: MCP server for searching and downloading documents from Anna's Archive

I was looking around for an MCP server that could connect Anna's Archive to Claude Desktop, as I wanted to be able to search and download books directly through the interface.

I couldn't find an

9 lip 2025, 21:50:02 | Hacker news