One thing I marvel at is despite the apparent usefulness of graph databases no major cloud provider offers a hosted graph db solution, with the possible exception of CosmoDB and marketplace Neo4j. It makes me wonder whether there is more operational complexity than value, or if your average developer/architect isn’t finding enough use cases that can’t be done with more traditional RDBMS.
> or if your average developer/architect isn’t finding enough use cases that can’t be done with more traditional RDBMS
Graph DBs are not magic. If you drill down into their query/execution plans it becomes very obvious that there isn't really a technical distinction between a traditional RDBMs and a Graph DB.
If you need graph traversal you can just as well use a table with edge-like indexing and recursive CTEs, and in the future with implementations of SQL/PGQ you can also do that in a "nice" Cypher-like syntax.
2010s was the decade of web scale (e-commerce, social, as tech) and graph dbs struggle with low level control over storage layout, denormalization and custom indexes to optimize queries etc needed above a certain scale. In the 2020s I see a retreat back to “simple long tail tech for the 97%” (helped by exponentially more compute) - with a focus on UI, and now early exploration of AI which is driving need for fluent and fluid service connectivity and distributed data models across denormalized silos created in the 2010s. Which is setting up the 2030s to be about linked data and graphs. Maybe?
A lot of the Sr. system architects I’ve talked to complained how time consuming/complex (lack of a better word) it is while managing both relational and graph databases. Every CRUD operation leads to more dusting/cleaning your latent factors/connections in your graph databases.
There is ongoing research that the complexity arises due to the independent nature of your data across multiple databases, but the latent factors by definition are data-about-your-data. So the task is to make your latent factors dependent to their pertaining relational data, which enables easier house cleaning.
If you think about other data structures, we don't feel the need for a specialised database for them either. You don't need a set, map, heap, stack, array, hash etc database. You can do all of those just using a relational database. It's really not obvious to me why people think graphs need their own database, and I speak as someone who has stored certainly hundreds of thousands and possibly millions of graphs in databases over the years and almost exclusively used either an RDBMS, something like parquet or a key/value store to do it (to great success). I absolutely love graphs in programming and use them all the time but I've never once thought to myself that I would like to have graph semantics in a database over and above the links between relations you get in an rdbms or the links between whatever you like that you get in a key-value store.
More concretely I'm pretty sure you could demonstrate using relational algebra that any graph can be formed pretty trivially using relations. So I think there is probably provably no use case that can't be done with a more traditional RDBMS. Likewise it seems obvious that any graph can be stored in a key/value store (bunch of nodes, bunch of edges, badda bing badda boom) and therefore there is similarly no graph db use case that can't be done using a k/v store if that's your thing.
> You don't need a set, map, heap, stack, array, hash etc database.
On the contrary, redis is super popular, and that is basically what it is.
> More concretely I'm pretty sure you could demonstrate using relational algebra that any graph can be formed pretty trivially using relations. So I think there is probably provably no use case that can't be done with a more traditional RDBMS.
The question would be more about efficiency & ease of use than if you could literally just store a graph in a rdbms
One thing I marvel at is despite the apparent usefulness of graph databases no major cloud provider offers a hosted graph db solution, with the possible exception of CosmoDB and marketplace Neo4j. It makes me wonder whether there is more operational complexity than value, or if your average developer/architect isn’t finding enough use cases that can’t be done with more traditional RDBMS.
> or if your average developer/architect isn’t finding enough use cases that can’t be done with more traditional RDBMS
Graph DBs are not magic. If you drill down into their query/execution plans it becomes very obvious that there isn't really a technical distinction between a traditional RDBMs and a Graph DB.
If you need graph traversal you can just as well use a table with edge-like indexing and recursive CTEs, and in the future with implementations of SQL/PGQ you can also do that in a "nice" Cypher-like syntax.
AWS Neptune is really two different graph DBs, one a property graph model with Cypher queries and one a triple store with SPARQL queries.
2010s was the decade of web scale (e-commerce, social, as tech) and graph dbs struggle with low level control over storage layout, denormalization and custom indexes to optimize queries etc needed above a certain scale. In the 2020s I see a retreat back to “simple long tail tech for the 97%” (helped by exponentially more compute) - with a focus on UI, and now early exploration of AI which is driving need for fluent and fluid service connectivity and distributed data models across denormalized silos created in the 2010s. Which is setting up the 2030s to be about linked data and graphs. Maybe?
A lot of the Sr. system architects I’ve talked to complained how time consuming/complex (lack of a better word) it is while managing both relational and graph databases. Every CRUD operation leads to more dusting/cleaning your latent factors/connections in your graph databases. There is ongoing research that the complexity arises due to the independent nature of your data across multiple databases, but the latent factors by definition are data-about-your-data. So the task is to make your latent factors dependent to their pertaining relational data, which enables easier house cleaning.
If you think about other data structures, we don't feel the need for a specialised database for them either. You don't need a set, map, heap, stack, array, hash etc database. You can do all of those just using a relational database. It's really not obvious to me why people think graphs need their own database, and I speak as someone who has stored certainly hundreds of thousands and possibly millions of graphs in databases over the years and almost exclusively used either an RDBMS, something like parquet or a key/value store to do it (to great success). I absolutely love graphs in programming and use them all the time but I've never once thought to myself that I would like to have graph semantics in a database over and above the links between relations you get in an rdbms or the links between whatever you like that you get in a key-value store.
More concretely I'm pretty sure you could demonstrate using relational algebra that any graph can be formed pretty trivially using relations. So I think there is probably provably no use case that can't be done with a more traditional RDBMS. Likewise it seems obvious that any graph can be stored in a key/value store (bunch of nodes, bunch of edges, badda bing badda boom) and therefore there is similarly no graph db use case that can't be done using a k/v store if that's your thing.
> You don't need a set, map, heap, stack, array, hash etc database.
On the contrary, redis is super popular, and that is basically what it is.
> More concretely I'm pretty sure you could demonstrate using relational algebra that any graph can be formed pretty trivially using relations. So I think there is probably provably no use case that can't be done with a more traditional RDBMS.
The question would be more about efficiency & ease of use than if you could literally just store a graph in a rdbms
Transitive closure is something that historically was not supported in SQL and the relational model.
However it was added to the 1999 revision of the SQL standard, so its been there for 25 years.
One of the major uses of graph databases is to sell companies on the need to use graph databases.
> more operational complexity than value
Graph databases are somewhat notoriously hard to optimize at scale.
Does Amazon Neptune not count?
Neptune, from AWS is one, right?
it seems like if you do incremental query evaluation, then this problem, including the locking, just kinda vanishes (?)
[dead]