DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability to do out of core bigger than memory data processing on lower end consumer grade hardware is remarkable.
Curious to learn more about how people are using it?
Are they downloading parquet files and running analyses locally, or are they connecting to Iceberg-like data lake and leveraging DuckDBs query engine capabilities or have you exposed an interface (REST, UI) to query your data?
We use WASM DuckDB as the target for an in-browser agentic feature. Generated SQL runs against the user's individual tables that then feed in-browser dashboards.
We use DuckDB WASM with parquet to build dashboards in-browser. It's cool to be able to write SQL directly in a browser and not have to rely on REST/Graphql/etc to access the data layer.
I <3 DuckDB. It has become one of my go to tools for storing, data processing , integrations and now even graph. More importantly it's fun to use because it is so portable. Looking forward to v2.
The last year of DuckDB enhancements feel like the shift from in-process execution engine (which it is phenomenal at) to an engine that can serve as the foundation of a cloud data warehouse. I know the founders were reticent about not wanting to build that, but I have a feeling it is in the works.
I think you're right and there's no doubt they've now got the infrastructure to support it. IIRC, Hannes sort of hinted at it during his talk at this year's AI Council and they've made vague allusions to _something big_ being announced this fall and, as exciting as this announcement is, I don't think this release announcement is it. I wonder if it could involve bringing MotherDuck into the fold? I know it's OSS and DuckDB doesn't owe MD anything but (IMO) they've been a big part of DuckDB's success and undercutting them would probably result in a lot of ill will.
What do you plan on stripping and what's your target? The Emscripten based build is ~10Mb. I have a component build so I'd be interesting on how you'd like to break it up.
DuckDB is so cool, game changer when it comes to local data processing.
DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability to do out of core bigger than memory data processing on lower end consumer grade hardware is remarkable.
Thanks to the team for everything!
Curious to learn more about how people are using it?
Are they downloading parquet files and running analyses locally, or are they connecting to Iceberg-like data lake and leveraging DuckDBs query engine capabilities or have you exposed an interface (REST, UI) to query your data?
We use WASM DuckDB as the target for an in-browser agentic feature. Generated SQL runs against the user's individual tables that then feed in-browser dashboards.
Excellent performance.
We use DuckDB WASM with parquet to build dashboards in-browser. It's cool to be able to write SQL directly in a browser and not have to rely on REST/Graphql/etc to access the data layer.
I <3 DuckDB. It has become one of my go to tools for storing, data processing , integrations and now even graph. More importantly it's fun to use because it is so portable. Looking forward to v2.
Ditto! Very happy with the upcoming async support! Now it'll be a nice little db for serving http traffic as well!
The last year of DuckDB enhancements feel like the shift from in-process execution engine (which it is phenomenal at) to an engine that can serve as the foundation of a cloud data warehouse. I know the founders were reticent about not wanting to build that, but I have a feeling it is in the works.
> I have a feeling it is in the works
I think you're right and there's no doubt they've now got the infrastructure to support it. IIRC, Hannes sort of hinted at it during his talk at this year's AI Council and they've made vague allusions to _something big_ being announced this fall and, as exciting as this announcement is, I don't think this release announcement is it. I wonder if it could involve bringing MotherDuck into the fold? I know it's OSS and DuckDB doesn't owe MD anything but (IMO) they've been a big part of DuckDB's success and undercutting them would probably result in a lot of ill will.
Wait, what? I‘ve always assumed MotherDuck is the company by the main developers of DuckDB? They are unrelated?
I've been using MotherDuck as a data warehouse for some time now. No regrets.
This is cool
What about the runtime size? I care this because I intend to run a stripped WASM version of DuckDB in browser.
What do you plan on stripping and what's your target? The Emscripten based build is ~10Mb. I have a component build so I'd be interesting on how you'd like to break it up.
Was hoping to see procedural functionality like PL/pgSQL... regardless, an astonishing project overall.