As a longtime Raft user (via hashicorp/raft), I'm curious about your Raft implementation! You mention etcd's Raft library, but it isn't natively Multi-Raft is it? Is your implementation similar to https://tikv.org/deep-dive/scalability/multi-raft/ ? I'd love to hear about your experience implementing and testing it!
Awesome question! We'd experimented with https://github.com/lni/dragonboat and the hashicorp/raft in the early implementation, the etcd/raft library had been ported to a multi-raft style implementation by CockroachDB way back when, but they went the way of TigerBeetle and coupled their consensus deeply with the kv storage. Etcd has recently in v3.6 abstracted out their raft implementation and gave a pluggable interface into the transport layer, which meant that we could implement our own multi-raft style transport layer with heartbeat and multi-node message buffering on top of HTTP/Quic.
We implemented chaos testing suites akin to Jepsen to cover as many scenarios as possible and are currently implementing TigerBeetle style simulation tests on top of that for harder to reproduce scenarios!
Fascinating! We settled on Quic with Protobuf because it was more performant in our testing than the gRPC when coupled with the backoff, failure cases (node startup ordering server/client connections), and to not be coupled with the gRPC library versions in Go, which has bitten us a number of times when dealing with dependency management when you're trying to juggle k8s, etcd, and google dependencies in the same Go project. Plus the performance bottleneck in most of the use cases we're specializing in are on the embedding/ml side of things.
Of course the two most visionary people I worked with at Lytics went and built this. Just in time... this is the vector database I actually need. Termite is the killer feature for me, native ML inference in a single binary means I can stop duct-taping together embedding APIs for my projects. Excited to spend the upcoming weekends hacking on the Antfly ecosystem.
I’ve got a project right now, separate vector DB, Elasticsearch, graph store, all for an agent system.
When you say Antfly combines all three, what does that actually look like at query time? Can I write one query that does semantic similarity + full-text + graph traversal together, or is it more like three separate indexes that happen to live in the same binary?
Does it ship with a CLI that's actually good? I’m pivoting away from MCP. Like can I pipe stuff in, run queries, manage indexes from the terminal without needing to write a client? That matters more to me than the MCP server honestly.
And re: Termite + single binary, is the idea that I can just run `antfly swarm`, throw docs and images at it, and have a working local RAG setup with no API keys? If so, that might save me a lot of docker-compose work.
Who's actually running this distributed vs. single-node? Curious what the typical user experience looks like.
Exactly the use case I built it for!
I wanted a world where you could build your indexes and the query planner could just be smart enough to use them in a single query. I've not quite nailed down the agentic query planner side 100% (it's getting there), but the JSON query DSL allows you to pipeline, join, fuse all the full-text, semantic, graph, reranking, pruning (score/token pruning) all in one query.
The CLI is my primary development tool with antfly, I am definitely looking for feedback on what people would like to see there, it's a little chonky with the flags --pruner e.g. requires writing the JSON for the config because I didn't want users to have to memorize 1000 subflags. It's definitely a first class citizen.
With respect to "Termite + single binary" that's exactly right, Termite handles chunking, multimodal chunking, embeddings (sparse + dense), reranking, fused chunking/embedding models, and we're excitedly getting more support for a variety of onnx based llms/ner models to help with data extraction use cases (functiongemma/gliner2/etc) so you don't have to setup 10 different services for testing vs deployment.
We run Antfly ourselves for our https://platform.searchaf.com (cheeky search AntFly) Algolia style search product in a distributed setup, and some users run Antfly in single node with large instances (more at the Postgres size datasets with millions of documents vs. large multitenant depoys). But we really wanted to build something with a more seamless experience of going back and forth between a distributed vs single node instance than elasticsearch or postgres can offer.
Hope that helps! Let me know if I can help you with anything!
A quick note, on platform.searchaf.com
The account creation process hits a snag with verify-email links received on email giving a 404. hope it helps.
On a parallel note, It would be nice to put an architecture diagram in the github repo.
Are there particular aspects of the current implementation which you want to actively improve/rearchitect/change?
I agree with the goals set out for the project and can testify that elasticsearch's DX is pretty annoying.
Having said that, distributed indexing with pluggable ingestion/query custom indexes may be a good goal to aim for.
- Finite State Transducers (FST) or Finite state automata based memory efficient indexes for specific data mimetypes
- adding hashing based search semantic search indexes.
And even changing the indexer/reranker implementation would help make things super hackable.
Oh thanks for the 404 on the verify link (I abstracted out the auth OIDC for cross domain login and must have missed a path).
Yes good call, I tried to start that on the website with a react-flows based architectural flow chart a little bit but it's a bit high level, and not consumable directly in github markdown files but I'll work on that!
That's exactly the direction I've been working on, the reranking, embedders and chunkers are all plugable and the schema design (using jsonschema for our "schema-ish" approach allows for fine-grained index backend hints for individual data types etc.) I'll work on getting a good architecture doc up today and tomorrow!
The one area I've seen knowledge graphs come up are: Product Knowledge Graphs (PKGs), which are a centralized, semantic, and highly interconnected data structure that brings together information about products, customers, and their interactions into a single, comprehensive "360-degree" view. Basically, it's the idea of combing through all the data (CRMs, codebases, Ticketing System, Churn Management System, sales calls, ...) that the company has digitally about their customers, and building one giant knowledge graph that they can use to determine a bunch of business intelligence use cases, or using it to power how to create new features. Then you slap an answer bar or semantic search on top of it, and you have a powerful way of getting insights or doing gap analysis on your product versus your customer needs.
Anyway, that's just one example of why you might want to use a knowledge graph. I'm sure there are literally hundreds, of more examples.
I can't speak for everyone, knowledge graphs are the "new hotness" of the ai space (RAG and MCP are seeing a lull in their hype cycles I guess). But I've used graphs professionally for a long time to connect relationships that SQL normal forms have trouble expressing non-recursively. E.g. I used graphs to define identity relationships between data sources hierarchically, and then had a another graph relationship on top of that to define connections between those identities, user at one level and organizations at the next. Graphs as indexes allow you to express arbitrary relationships between data to allow for more efficient lookups by a database. Some folks use it to express conceptual relationship between data for AI now, so if I have a bunch of images stored in google drive, I might want to abstract the concept of pets and pets have relationship with a human etc. then my database queries for looking up all pictures related to the dog-pets owned by some human becomes a tractable search instead of a scan of the corpus!
There's some examples in the quickstart on the website but I'll add an explicit e2e example case for that too. Otherwise the tests for that are a little lower level in the code! I'll add the RSF (merging of the two lists) example for that too!! Thanks for the feedback.
I built this for myself because I hated running a large ElasticSearch instance at work and wanted something that would autoscale and something that allowed for reindexing data. I also had a lot of experience running a large BigTable/Elasticsearch custom graph database I thought could be unified into a single database to cut costs. Started adding an embedding index for fun based on some Google papers and now here we are!
I have a variety of blogs that I used too and reference implementations!
It's a Rabit[Q]uantized Hierchical Balanced Clustering algorithm we use for the vector index and we use a chunked segment index for the sparse index if you're curious! Happy to discuss more!
Yes we do use SIMD heavily! https://github.com/ajroetker/go-highway I also added SME support for Darwin for most algorithms. We use it in the full-text index, all over the vector indexes and heavily for the ml inference we do in go especially.
As a longtime Raft user (via hashicorp/raft), I'm curious about your Raft implementation! You mention etcd's Raft library, but it isn't natively Multi-Raft is it? Is your implementation similar to https://tikv.org/deep-dive/scalability/multi-raft/ ? I'd love to hear about your experience implementing and testing it!
Awesome question! We'd experimented with https://github.com/lni/dragonboat and the hashicorp/raft in the early implementation, the etcd/raft library had been ported to a multi-raft style implementation by CockroachDB way back when, but they went the way of TigerBeetle and coupled their consensus deeply with the kv storage. Etcd has recently in v3.6 abstracted out their raft implementation and gave a pluggable interface into the transport layer, which meant that we could implement our own multi-raft style transport layer with heartbeat and multi-node message buffering on top of HTTP/Quic.
We implemented chaos testing suites akin to Jepsen to cover as many scenarios as possible and are currently implementing TigerBeetle style simulation tests on top of that for harder to reproduce scenarios!
HTTP/QUIC, so no gRPC then? Or is https://github.com/grpc/grpc/issues/19126 not the blocker to gRPC over QUIC I thought it was.
I've long wished for QUIC with Nomad! [1] We've always used a weird QUIC-over-TCP multiplexer called yamux. [2]
[1] https://github.com/hashicorp/nomad/issues/23848
[2] https://github.com/hashicorp/yamux (I'm fairly certain libp2p's fork is actually better)
Fascinating! We settled on Quic with Protobuf because it was more performant in our testing than the gRPC when coupled with the backoff, failure cases (node startup ordering server/client connections), and to not be coupled with the gRPC library versions in Go, which has bitten us a number of times when dealing with dependency management when you're trying to juggle k8s, etcd, and google dependencies in the same Go project. Plus the performance bottleneck in most of the use cases we're specializing in are on the embedding/ml side of things.
Thanks for the links! I hadn't seem yamux before!
Of course the two most visionary people I worked with at Lytics went and built this. Just in time... this is the vector database I actually need. Termite is the killer feature for me, native ML inference in a single binary means I can stop duct-taping together embedding APIs for my projects. Excited to spend the upcoming weekends hacking on the Antfly ecosystem.
I totally agree. I'm looking forward to what AJ and James build here. And I'm also planning on using it at my current company.
Interesting project.
I’ve got a project right now, separate vector DB, Elasticsearch, graph store, all for an agent system.
When you say Antfly combines all three, what does that actually look like at query time? Can I write one query that does semantic similarity + full-text + graph traversal together, or is it more like three separate indexes that happen to live in the same binary?
Does it ship with a CLI that's actually good? I’m pivoting away from MCP. Like can I pipe stuff in, run queries, manage indexes from the terminal without needing to write a client? That matters more to me than the MCP server honestly.
And re: Termite + single binary, is the idea that I can just run `antfly swarm`, throw docs and images at it, and have a working local RAG setup with no API keys? If so, that might save me a lot of docker-compose work.
Who's actually running this distributed vs. single-node? Curious what the typical user experience looks like.
Thanks for the awesome questions!!
Exactly the use case I built it for! I wanted a world where you could build your indexes and the query planner could just be smart enough to use them in a single query. I've not quite nailed down the agentic query planner side 100% (it's getting there), but the JSON query DSL allows you to pipeline, join, fuse all the full-text, semantic, graph, reranking, pruning (score/token pruning) all in one query.
The CLI is my primary development tool with antfly, I am definitely looking for feedback on what people would like to see there, it's a little chonky with the flags --pruner e.g. requires writing the JSON for the config because I didn't want users to have to memorize 1000 subflags. It's definitely a first class citizen.
With respect to "Termite + single binary" that's exactly right, Termite handles chunking, multimodal chunking, embeddings (sparse + dense), reranking, fused chunking/embedding models, and we're excitedly getting more support for a variety of onnx based llms/ner models to help with data extraction use cases (functiongemma/gliner2/etc) so you don't have to setup 10 different services for testing vs deployment.
We run Antfly ourselves for our https://platform.searchaf.com (cheeky search AntFly) Algolia style search product in a distributed setup, and some users run Antfly in single node with large instances (more at the Postgres size datasets with millions of documents vs. large multitenant depoys). But we really wanted to build something with a more seamless experience of going back and forth between a distributed vs single node instance than elasticsearch or postgres can offer.
Hope that helps! Let me know if I can help you with anything!
A quick note, on platform.searchaf.com The account creation process hits a snag with verify-email links received on email giving a 404. hope it helps.
On a parallel note, It would be nice to put an architecture diagram in the github repo. Are there particular aspects of the current implementation which you want to actively improve/rearchitect/change?
I agree with the goals set out for the project and can testify that elasticsearch's DX is pretty annoying. Having said that, distributed indexing with pluggable ingestion/query custom indexes may be a good goal to aim for. - Finite State Transducers (FST) or Finite state automata based memory efficient indexes for specific data mimetypes - adding hashing based search semantic search indexes.
And even changing the indexer/reranker implementation would help make things super hackable.
Oh thanks for the 404 on the verify link (I abstracted out the auth OIDC for cross domain login and must have missed a path).
Yes good call, I tried to start that on the website with a react-flows based architectural flow chart a little bit but it's a bit high level, and not consumable directly in github markdown files but I'll work on that!
That's exactly the direction I've been working on, the reranking, embedders and chunkers are all plugable and the schema design (using jsonschema for our "schema-ish" approach allows for fine-grained index backend hints for individual data types etc.) I'll work on getting a good architecture doc up today and tomorrow!
This is very interesting! I noticed that your TypeScript SDK link results in a 404: https://antfly.io/docs/sdks -> https://github.com/antflydb/antfly-ts
Thanks! Fixed that up!
Can you help me understand what type of practical features Graph Traversal unlocks?
I've seen it on a few products and it doesn't click with me how people are using it.
The one area I've seen knowledge graphs come up are: Product Knowledge Graphs (PKGs), which are a centralized, semantic, and highly interconnected data structure that brings together information about products, customers, and their interactions into a single, comprehensive "360-degree" view. Basically, it's the idea of combing through all the data (CRMs, codebases, Ticketing System, Churn Management System, sales calls, ...) that the company has digitally about their customers, and building one giant knowledge graph that they can use to determine a bunch of business intelligence use cases, or using it to power how to create new features. Then you slap an answer bar or semantic search on top of it, and you have a powerful way of getting insights or doing gap analysis on your product versus your customer needs.
Anyway, that's just one example of why you might want to use a knowledge graph. I'm sure there are literally hundreds, of more examples.
I can't speak for everyone, knowledge graphs are the "new hotness" of the ai space (RAG and MCP are seeing a lull in their hype cycles I guess). But I've used graphs professionally for a long time to connect relationships that SQL normal forms have trouble expressing non-recursively. E.g. I used graphs to define identity relationships between data sources hierarchically, and then had a another graph relationship on top of that to define connections between those identities, user at one level and organizations at the next. Graphs as indexes allow you to express arbitrary relationships between data to allow for more efficient lookups by a database. Some folks use it to express conceptual relationship between data for AI now, so if I have a bunch of images stored in google drive, I might want to abstract the concept of pets and pets have relationship with a human etc. then my database queries for looking up all pictures related to the dog-pets owned by some human becomes a tractable search instead of a scan of the corpus!
in the query_test.go, I don’t see how the hybrid search is being exercised.
For fun I am making hybrid search too and would love to see how you merge the two list (semantic and keyword) and rerank the importance score.
There's some examples in the quickstart on the website but I'll add an explicit e2e example case for that too. Otherwise the tests for that are a little lower level in the code! I'll add the RSF (merging of the two lists) example for that too!! Thanks for the feedback.
I've added a specific example for that using the go-sdk https://github.com/antflydb/antfly/pull/5 here!
Was thinking to create something similar, well done!
This looks sick!
Did you build this for yourself?
I built this for myself because I hated running a large ElasticSearch instance at work and wanted something that would autoscale and something that allowed for reindexing data. I also had a lot of experience running a large BigTable/Elasticsearch custom graph database I thought could be unified into a single database to cut costs. Started adding an embedding index for fun based on some Google papers and now here we are!
what google papers?
Not strictly google but microsoft/bing too, here's the top ones from my notes:
https://arxiv.org/abs/2410.14452 spfresh, https://arxiv.org/abs/2111.08566 spann, https://arxiv.org/abs/2405.12497 rabitq, https://arxiv.org/abs/2509.06046 diskann,
I have a variety of blogs that I used too and reference implementations!
It's a Rabit[Q]uantized Hierchical Balanced Clustering algorithm we use for the vector index and we use a chunked segment index for the sparse index if you're curious! Happy to discuss more!
Curious if you’re using any SIMD optimizations for numerical calculations.
Yes we do use SIMD heavily! https://github.com/ajroetker/go-highway I also added SME support for Darwin for most algorithms. We use it in the full-text index, all over the vector indexes and heavily for the ml inference we do in go especially.