Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

compress - byte-stream compression

Enable with use compress;. gzip, zlib, and raw DEFLATE - bytes in, bytes out - plus a streaming compressor for large data. The algorithm is a string argument, the same shape as hash.compute(b, "sha-256") and crc.compute(b, "crc32"); the pack / unpack verbs pair with archive’s (byte streams here, file bundles there). Distinct from encoding, which is reversible representation (hex / base64, which don’t reduce information); this is entropy-based size reduction. Backed by Go’s compress/*; works on both binaries.

Surface

CallReturnsNotes
compress.pack(b, algo [, level])bytesCompress; algo is "gzip" / "zlib" / "deflate".
compress.unpack(b, algo)bytesDecompress; same algo.
compress.stream(algo [, level])compress.StreamStart a streaming compressor.
compress.update(stream, b)nullFeed one chunk of input.
compress.finalize(stream)bytesClose and return the full compressed output.

level is optional: "fast", "default" (when omitted), or "best". Decompressing malformed input, or an unknown algo, is a positioned runtime error (catchable with try / catch).

The three algorithms:

  • "gzip" (RFC 1952) - magic + CRC-32 + size. gzip(1)-compatible; the reliable HTTP Content-Encoding: gzip. Use for standalone files.
  • "zlib" (RFC 1950) - a compact DEFLATE + Adler-32 wrapper. This is what HTTP Content-Encoding: deflate officially means (and it’s the wrapper inside PNG, zip entries, etc.) - but that coding is inconsistently implemented in the wild, so prefer "gzip" for HTTP.
  • "deflate" (RFC 1951) - raw DEFLATE, no framing. Smallest, when you supply your own framing. Not the HTTP deflate content-coding despite the name - that one is "zlib".

One-shot

use io;
use compress;
use convert;

def raw as bytes init convert.bytesFromString("hello hello hello world", "utf-8");
def packed as bytes init compress.pack($raw, "gzip", "best");
def back as bytes init compress.unpack($packed, "gzip");
io.printf("%t\n", convert.stringFromBytes($back, "utf-8") == "hello hello hello world");

Streaming

Feed input in chunks (so you never hold it all at once); the full compressed result comes back at finalize:

def s as compress.Stream init compress.stream("gzip");
compress.update($s, chunkOne);
compress.update($s, chunkTwo);
def packed as bytes init compress.finalize($s);   # equals compress.pack of the concatenation

compress.Stream is a handle (like hash.Stream): copies share the underlying state, and finalize consumes it - a second finalize or update on the same handle errors.

See also

  • encoding.md - hex / base64 (representation, not compression).
  • hash.md - the same algo-string + streaming-handle shape.
  • archive.md - pack / unpack for file bundles ("tar" / "zip").