wire / wire-format / orc-format
ORC is a columnar storage format for Hadoop/Spark. File structure: stripes (row groups), each with index, data, and footer; file footer (schema, statistics, stripe locations); postscript (compression codec, footer length, magic 'ORC'). Magic: 'ORC' (0x4F 0x52 0x43) at the end of the file.
aka: Apache ORC (Optimized Row Columnar) format
spec: Apache ORC specification
ORC is a columnar storage format for Hadoop/Spark. File structure: stripes (row groups), each with index, data, and footer; file footer (schema, statistics, stripe locations); postscript (compression codec, footer length, magic 'ORC'). Magic: 'ORC' (0x4F 0x52 0x43) at the end of the file.
| field | size | meaning |
|---|---|---|
| Stripe | variable | Index stream (row group offsets) + data stream (column data) + footer (column statistics). |
| File footer | variable | Header (1 byte, version), content length, stripe statistics, row index strides, schema (ProtoBuf serialized), and column statistics. |
| Postscript | variable | Compression codec (0 none, 1 zlib, 2 snappy, 3 lzo, 4 lz4, 5 zstd), footer length (variable-length), magic 'ORC' (3 bytes). |
| Magic | 3 bytes | 'ORC' (0x4F 0x52 0x43) at the very end of the file. |
example:
4f 52 43
ORC magic at end of file
agent: curl -H 'accept: application/json' wire.phall.io/wire-format/orc-format
or /wire-format/orc-format.json