reminder that you can just use write() and read() from files instead of tables and everything will be fine, you don’t need to pull up a 100Mloc dependency to your project just to save data.
not saying don’t use it, just saying if you have no users or less than a million, you probably don’t need it.
if some analyst requests a database to analyze, you can easily run a batch importer into a read only sql db thing of your analyst’s or orgs preference.
There arw usecases that are fine with using JSON or something similar as a store rather than a database, but databases absolutely make sense below 1mio users. You could argue that a lot of those cases could do with SQLite, but no database is just not a serious option for a lot of things. You will end up reinventing a database but in shitty. Consistency, atomic transaction, indexes etc. are not niche use cases.
A database is not trivial software. If you need database features without your performance being complete shit for non-trivial datasets, you will need to effectively write your own db engine, which will most likely be much worse than postgress or sqlite unless you know a lot about writing databases and invest a lot of time into it.
Using a database is not the same as using the isEven API.
It varies a lot depending on the usecase, but in the end, answering those kinds of questions is what we get paid for.
For example if it's a write only metric that you'll only ever analyze in a future stage of your company, you just write it and store it in the node that handles it.
If you need to read the data for applciation logic, you can build an index, replicate it along all of your nodes, partition based on the index.
And btw that last thing, (which isn't particularly complex), is something that you'll need at 100k+ users. Below that you can live just fine with just 1 server, maybe 2 redundant symmetric servers tops.
reminder that you can just use write() and read() from files instead of tables and everything will be fine, you don’t need to pull up a 100Mloc dependency to your project just to save data.
not saying don’t use it, just saying if you have no users or less than a million, you probably don’t need it.
if some analyst requests a database to analyze, you can easily run a batch importer into a read only sql db thing of your analyst’s or orgs preference.
even postgres can be dropped, it’s fine
Databases do a lot more than a regular file.
There arw usecases that are fine with using JSON or something similar as a store rather than a database, but databases absolutely make sense below 1mio users. You could argue that a lot of those cases could do with SQLite, but no database is just not a serious option for a lot of things. You will end up reinventing a database but in shitty. Consistency, atomic transaction, indexes etc. are not niche use cases.
>Databases do a lot more than a regular file.
Yeah, you need to write a lot of code that does a lot of work, that's our job.
No, that's "not invented here syndrome".
Well if there exists a peyorative term that include 'syndrome' then it must be bad. You win
A database is not trivial software. If you need database features without your performance being complete shit for non-trivial datasets, you will need to effectively write your own db engine, which will most likely be much worse than postgress or sqlite unless you know a lot about writing databases and invest a lot of time into it.
Using a database is not the same as using the isEven API.
A common misconception is conflating the complexity of a dependency with the complexity of replacing that dependency.
> you will need to effectively write your own db engine,
Very wrong, you don't need a web framework to make a webpage, you don't need a game engine to write a game, you don't need a factory to make a nail
how does that work with >1 machine?
It varies a lot depending on the usecase, but in the end, answering those kinds of questions is what we get paid for.
For example if it's a write only metric that you'll only ever analyze in a future stage of your company, you just write it and store it in the node that handles it.
If you need to read the data for applciation logic, you can build an index, replicate it along all of your nodes, partition based on the index.
And btw that last thing, (which isn't particularly complex), is something that you'll need at 100k+ users. Below that you can live just fine with just 1 server, maybe 2 redundant symmetric servers tops.