A follow up to my previous post - icestream is an asynchronous compaction service for iceberg tables with many equality deletes (a symptom of frequent streaming writes on tables with "primary keys"). Now, instead of relying on Cassandra + Spark to index Apache Iceberg table data, Icestream uses Flink and Apache Paimon - enabling a separation between compute and storage and keeping an LSM tree style index on disk.
A follow up to my previous post - icestream is an asynchronous compaction service for iceberg tables with many equality deletes (a symptom of frequent streaming writes on tables with "primary keys"). Now, instead of relying on Cassandra + Spark to index Apache Iceberg table data, Icestream uses Flink and Apache Paimon - enabling a separation between compute and storage and keeping an LSM tree style index on disk.