Author here. qdf is a schemaless binary serializer for Go — encoding/json-style
API, no schema/IDL/codegen. The idea is to compress across records (inline
string/key interning + columnar per-column codecs), so on batch telemetry it's
-24..-77% vs protobuf and both smaller and faster to DECODE than json/msgpack on
the default tier.
The part I had most fun with: a columnar mode you can run WHERE-predicates over
without fully decoding, and a lossy vector codec for AI embeddings that beats
Google's TurboQuant at equal recall (-22% bytes).
Honest about tradeoffs — protobuf/flatbuffers win raw single-message decode, and
msgpack can edge encode on high-cardinality data; it's alpha. Pure Go, zero deps.
Built solo; happy to answer anything.
Author here. qdf is a schemaless binary serializer for Go — encoding/json-style API, no schema/IDL/codegen. The idea is to compress across records (inline string/key interning + columnar per-column codecs), so on batch telemetry it's -24..-77% vs protobuf and both smaller and faster to DECODE than json/msgpack on the default tier.
The part I had most fun with: a columnar mode you can run WHERE-predicates over without fully decoding, and a lossy vector codec for AI embeddings that beats Google's TurboQuant at equal recall (-22% bytes).
Honest about tradeoffs — protobuf/flatbuffers win raw single-message decode, and msgpack can edge encode on high-cardinality data; it's alpha. Pure Go, zero deps. Built solo; happy to answer anything.