wire / wire-format / snappy-format

Snappy compressed data format

Snappy is Google's fast compression format. The frame format: stream identifier (chunk type 0xFF, 'snappy' magic), then chunks. Each chunk: 1-byte type (0x00 compressed, 0x01 uncompressed, 0x02 padding, 0x80-0xFD reserved, 0xFE CRC32C, 0xFF stream identifier), 3-byte length, then data. Compressed chunks use a simple LZ77-like scheme with varint lengths.

wire-format kind wire-format status standard verification verified tier B wire-format@1

aka: Snappy compressed data format

wire format

spec: Snappy (github.com/google/snappy)

Snappy is Google's fast compression format. The frame format: stream identifier (chunk type 0xFF, 'snappy' magic), then chunks. Each chunk: 1-byte type (0x00 compressed, 0x01 uncompressed, 0x02 padding, 0x80-0xFD reserved, 0xFE CRC32C, 0xFF stream identifier), 3-byte length, then data. Compressed chunks use a simple LZ77-like scheme with varint lengths.

fieldsizemeaning
Stream identifier chunk10 bytesType 0xFF, length 6, data 'sNaPpY' (0x73 0x4E 0x61 0x50 0x70 0x59).
Compressed chunkvariableType 0x00, 3-byte length, then: varint uncompressed length, then tagged literals/matches.
Uncompressed chunkvariableType 0x01, 3-byte length, then raw data.
CRC32C chunk4 bytesType 0xFE, length 4, CRC32C of the uncompressed data (optional).
Tagged literalvariableTag 00 (2 bits): literal; length in 6 bits or varint; then literal bytes.
Tagged copyvariableTag 01 (offset < 2048, len 4-11), 10 (offset < 65536, len 1-64), 11 (offset any, len 1-64): back-reference.

example:

ff 06 00 00 73 4e 61 50 70 59

Snappy stream identifier chunk


provenance

agent: curl -H 'accept: application/json' wire.phall.io/wire-format/snappy-format or /wire-format/snappy-format.json