Toasty, an async ORM for Rust

(tokio.rs)

140 points | by steveklabnik 2 days ago ago

58 comments

  • fulafel 27 minutes ago

    Why is asynchronity (sp?) a concern of the ORM in this case?

    • BiteCode_dev 11 minutes ago

      Because ORM attribute access usually trigger requests, and you must design the API so that those requests, which trigger called to the network, don't block.

  • alilleybrinker 2 days ago

    Very interested in exploring how this will compare to Diesel [1] and SeaORM [2], the other two options in this space today. Joshua Mo at Shuttle did a comparison between Diesel and SeaORM in January of this year that was really interesting [3].

    [1]: https://diesel.rs/

    [2]: https://www.sea-ql.org/SeaORM/

    [3]: https://www.shuttle.dev/blog/2024/01/16/best-orm-rust

    • tuetuopay 2 days ago

      My first reaction is this feels like a nice middleground between Diesel and SeaORM.

      The codegen part makes all columns and tables and stuff checked at compile-time (name and type) like Diesel, with a query builder that's more natural like SeaORM. I hope the query builder does not end up too magical like SQLAlchemy with its load of footguns, and stay close in spirit to Diesel that's "write sql in rust syntax".

      I think time will tell, and for now I'm keeping my Diesel in production :D

    • karunamurti 2 days ago

      Sea ORM is too opinionated in my experience. Even making migration is not trivial with their own DSL. Diesel was ok, but I never use it anymore since rocket moved to async.

      I'm mainly use sqlx, it's simple to use, there's query! and query_as! macro which is good enough for most of the case.

      • sverro2 a day ago

        I like sqlx, but have been eyeing diesel for some time. Any reasons you don't use diesel_async?

        • malodyets 7 hours ago

          With Diesel async integrating everything with the pooling is a bit hairy. With sqlx everything just works.

    • Onavo 2 days ago

      It's nice seeing more Django/Prisma style ORMs where the non-SQL source code is the source of truth for the schema and migrations are automatically generated.

  • aabhay 3 hours ago

    Interesting take!

    In my experience, Dynamo and other NoSQL systems are really expressive and powerful when you take the plunge and make your own ORM. That’s because the model of nosql can often play much nicer with somewhat unique structures like

    - single table patterns - fully denormalized or graph style structures - compound sort keys (e.g. category prefixed)

    Because of that, I would personally recommend developing your own ORM layer, despite the initial cost

    • smt88 3 hours ago

      Why does a NoSQL or denormalized database need an ORM?

      Developing your own ORM is almost always a waste of time and a bad idea.

  • tricked 2 days ago

    Looks well thought out i like that for the most part this seems faster/easier than rolling your own sql query mapping etc compared to the other solutions I've come across in rust

  • Sytten 2 days ago

    For me diesel hits right balance since it is more a query builder and it is close to the SQL syntax. But sometimes it doesn't work because it is very strongly typed, right now I use sea-query for those scenarios and I built the bridge between the two.

    Ideally I would use something akin to Go Jet.

  • colesantiago 2 days ago

    I don't get the pent up anger with ORMs, I used it for my SaaS on Flask that I run and own for 4 years bringing in over $2M+ ARR with no issues.

    Great to see some development in this for Rust, perhaps after it becomes stable I may even switch my SaaS to it.

    • jeremyloy_wt 5 hours ago

      The second that you would benefit from using a DBMS specific feature, the ORM begins getting in the way. It is highly unlikely that an ORM provides support, much less a good abstraction, over features that only 1/N supported DBMS have.

      Your code ends up using the driver raw in these cases, so why not just use the driver for everything? Your codebase would be consistent at that point

      • rtpg an hour ago

        I have found that ORM arguments in context don’t stick very well to Django’s ORM, but see the argument applying well to most all the others.

        Case in point Django is really good about DB-specific functionality and letting you easily add in extension-specific stuff. They treat “you can only do this with raw” more or less as an ORM design API issue.

        My biggest critique of Django’s ORM is its grouping and select clause behavior can be pretty magical, but I’ve never been able to find a good API improvement to tackle that.

        • globular-toast 43 minutes ago

          Django's ORM is the worst for object-relational impedance mismatch, though. Django is great if you're happy with thinly-veiled database tables. But it absolutely sucks if what you want is real objects representing business entities.

          The simplest example is you can't build a Django object with a collection on it. Take the simplest toy example: a todo list. The natural model is simple: a todo list has a name and a list of items. You can't do that in Django. Instead you have to do exactly what you would do in SQL: two tables with item having a foreign key. There's no way to just construct a list with items in it. You can't test any business rules on the list without creating persistent objects in a db. It's crazy.

          So yeah, Django lets you do loads with the relational side, but that's because it's doing a half-arsed job of mapping these to objects.

      • fiedzia 3 hours ago

        >The second that you would benefit from using a DBMS specific feature, the ORM begins getting in the way.

        You can extend diesel (and probably many other orms, Diesel is just particularly easy here) to support any db feature you want.

        > It is highly unlikely that an ORM provides support, much less a good abstraction, over features that only 1/N supported DBMS have.

        That depends on orm flexibility and popularity. It may not provide support OOTB, but can make it easy to add it.

        > Your code ends up using the driver raw in these cases, so why not just use the driver for everything? Your codebase would be consistent at that point

        Main point of using orm for me is that I have type verification, raw (as in text) breaks too easily.

      • viraptor 3 hours ago

        Because you only need the specific features in a tiny amount of cases, while 99% is some flavour of SELECT * ... LEFT JOIN ... (If it's not, then sure, ORM would be annoying)

        Making that 99% smaller, simpler and automatically mapping to common types makes development a lot easier/faster. This applies to pretty much any higher level language. It's why you can write in C, but embed an ASM fragment for that one very specific thing instead of going 100% with either one.

    • kyleee 2 days ago

      Has it benefited you? Have you moved to a different underlying SQL software without having to make any changes to your codebase? Or some other benefit?

      • carlgreene 5 hours ago

        For me it’s speed of development. I’m frankly not very good at SQL, but an ORM in a familiar syntax to the language I use most (Typescript) increases my dev speed tremendously.

        I also have a relatively successful saas that uses Prisma and it’s been phenomenal. Queries are more than fast enough for my use case and it allows me to just focus on writing more difficult business logic than dealing with complex joins

  • cyndunlop 2 days ago

    Toasty was the focus of Carl Lerche's P99 CONF keynote on Wednesday. It provoked some interesting discussion in the chat.

  • arandomusername 2 days ago

    Looks awesome. Would love to see the table definitions that are generated from the schema as well.

  • revskill 2 days ago

    O in orm is misleading term. To me orm is about table.

  • andrewstuart 6 hours ago

    The days of the ORM have passed.

    AI writes amazing SQL, modern SQL databases are incredible and the best way to get the most out of your DB is write SQL.

    Invest your learning budget in SQL, not in some random developers abstraction.

    • t-writescode 5 hours ago

      A good ORM is just a pre-built and externally tested validator of common DB work.

      There's plenty of value in knowing both.

      "AI writes amazing SQL" and "AI writes amazing DB to Application Translation Layer Code" just means "AI can write your company's bespoke ORM".

    • benatkin 5 hours ago

      AI can give better responses to a lot of requests when it has a well designed high level API available. And many ORMs don’t produce well designed APIs but it seems this one will.

      • andrewstuart 5 hours ago

        Ai is as good as its training data and there’s vast sql documentation and source code that has been ingested by the AI engines.

        • benatkin 4 hours ago

          I think that for some things user intent would be better expressed as customized ORM than SQL queries. If the ORM isn’t customized and is just generated from the tables then, yeah, not much of a help.

  • mbrumlow 2 days ago

    [flagged]

    • alain_gilbert 2 days ago

      I never understood why people are so stubborn about hating on orm.

      For example I'm familiar with https://gorm.io and it does save me a lot of time and useless boilerplate.

      And guess what, if I ever need to make a complex query, I also happen to know SQL, and I'm just going to make a "raw" query https://gorm.io/docs/sql_builder.html#Raw-SQL and be done with it.

      It's not all that hard.

      edit:

      the other common complaint usually is: "but I don't know what query the orm is going to make..."

      Use "Debug" https://gorm.io/docs/session.html#Debug and it will print out exactly what query it's making. Not happy with it? Make a raw query.

      • throwaway19972 2 days ago

        > I never understood why people are so stubborn about hating on orm.

        Objects are just bad abstractions for representing database interactions. They never map cleanly and any attempt to cover up the incoherence introduces further problems.

        Avoiding boilerplate is understandable, but typed schemas and queries exist without hauling in a full ORM.

        Of course you can pump out a lot of SQL very quickly with ORMs. There's a lot positive to say about this approach! But you don't tend to end up with code where you can easily tell what's going on.

        • Daishiman 2 days ago

          > Objects are just bad abstractions for representing database interactions.

          But they're excellent abstractions for business entities.

          > Of course you can pump out a lot of SQL very quickly with ORMs. There's a lot positive to say about this approach! But you don't tend to end up with code where you can easily tell what's going on.

          15 years building up massive apps with Django and SQLAlchemy and this has never been a problem.

          • throwaway19972 2 days ago

            > 15 years building up massive apps with Django and SQLAlchemy and this has never been a problem.

            I guarantee you it is if the reader isn't already intimately familiar with SQLAlchemy.

            • Daishiman 2 days ago

              If your job literally entails using a database abstraction and you can't arse yourself into learning said abstraction then it's a skills issue, just as much as people who want to use ORMs without learning SQL.

              • throwaway19972 2 days ago

                I can't speak to SQLAlchemy itself, but ActiveRecord (which I'm sure has both shared and unshared issues) still imposes a high maintenance burden with a small team of highly-skilled developers, both on the performance front and the validating-changes front. I, personally, find it very annoying to read, having to hop around an object hierarchy just to figure out what kind of query is generated and what kind of logic is "silently" added to the query somewhere in said object hierarchy (or, god forbid, mixins/monkey-patching).

                You definitely make good points, and all my issues are over-comable with time and effort and knowing which tricks to use, so this truly is a matter of opinion and taste. I'm just pointing out that the idea that this produces more readable code seems far from obvious.

                • Daishiman a day ago

                  IMHO your critiques have more to do with Ruby's promotion of class hierarchies and the fact that ActiveRecord requires inspecting a DB to infer what's going on and Rail's abuse of monkeypatching.

    • thimabi 2 days ago

      I concur. I’ve used ORMs in certain projects, but iterating and optimizing them later on became untenable. I’ve learned my lesson — all my new projects forgo ORMs by default.

      • Daishiman 2 days ago

        How? ORM logic is as simple or as complicated as the underlying queries and results. In 15 years of building web apps with Python ORMs this has never been a problem across many, many different teams.

    • whazor 2 days ago

      A lot of people think SQL is antiquated, and therefore not worth learning. However, SQL is based upon Relation Algebra, which is the foundation of database optimalisation. Any data querying language/library that is not based on relational algebra will lack the performance. In particular ORMs fall under this category. Having said that, there are other languages/libraries out there that do use the relational algebra constructs.

    • carllerche 2 days ago

      I respect that some prefer just to use SQL, but that isn't where most stand.

      Also, instead of a reactionary "all ORMs are trash," where ORM probably means different things to different people, maybe you could provide some value to the conversation by providing specific points and/or arguments supporting your feelings about ORMs. At the very least, you could provide some citation to an article that does the summarization.

    • jasfi 2 days ago

      Moving fast is often crucial, but you typically also get more readable code, which is also worth it.

      A query/DML that is badly optimized is usually very complex, which means that the ORM's syntax isn't a good fit. No problem, since they typically support raw SQL calls, so nothing is really lost.

      You just have to know when to use the right tool for the job, as always.

      • throwaway19972 2 days ago

        > Moving fast is often crucial, but you typically also get more readable code, which is also worth it.

        I think the exact opposite is true, actually—because of the introduction of stuff like lifecycle hooks it becomes very difficult to figure out how the domain semantics translate to query/execution semantics. Of course some are lighter than that and just e.g. map a table to a domain type (which is much more readable), but that's not typically what catches criticism.

    • stevenally 2 days ago

      Yep, plus learn SQL and you can use it everywhere.

      Every language has it's own (multiple) language specific ORMs.

    • jiripospisil 2 days ago

      Nonsense. People use ORMs because the vast majority of queries are trivial. If you need something the ORM doesn't provide sufficient control over, only then you move to raw queries for these cases.

      • mbrumlow 2 days ago

        It’s has nothing to do with being simple and everything to do with wha the database looks like at the end of the day.

        Some ORMs are better than others but if if you have ever looked at a database created by a ORM it always has weird lookup tables, funny names and even with simple objects completely unusable without the ORM.

        We live in a multi language world. There is a high chance you are going to want to access this data with a different language. This langue is not going to have the same ORM as such you will have a horrid time access the data.

        ORMs often lay their data out in a way that is highly language dependent.

        The fact of the matter is SQL is not hard and bypassing the main interface to the database and hand wave it away will always bring you regret when writing any sooty of software other than a toy project.

        Your best case is you become an expert in the ORM. Resulting in a skill set that does not transfer easy, language locked and the worst of all bottle necks any changes to the data layer to your ORM expert who at first will be proud and happy and end smug and bitchy as all the data layers request changes will simply be redirected to them.

        When people like me say ORMs are trash it’s much more than any of the surface level rebuttals listed here. It’s about the whole life cycle is your project. Adding up all the places ORMs can fuck you just makes it a bad proposition.

        God forbid you need to upgrade your database or the ORM version.

        • Nuzzerino a day ago

          > ORMs often lay their data out in a way that is highly language dependent.

          Which ORMs did you use? This doesn't sound normal at all. Never saw this with Rails, Ecto, EF, Sybase, or even the legacy project I once worked on that had 4 different ORMs for different parts of the same backend process, using the same database (some were very old and behind slowly phased out over time). Maybe you have ORM confused with CMS (content management system). A CMS can do those things, but that is not an ORM.

          > There is a high chance you are going to want to access this data with a different language.

          There are tools for that, such as ETL, read replicas, data warehouses, code generators, raw sql, stored procedures, views, just off the top of my head.

        • alain_gilbert 2 days ago

          `if you have ever looked at a database created by a ORM`

          You do realize that you can make your own migrations using raw SQL, and still use the ORM with your tables?

        • imtringued 2 days ago

          Well, different people, different experiences.

          • mbrumlow 2 days ago

            Same experience just different time lines.

    • rwaksmunski 2 days ago

      Yeah, let's trash the 50 year old industry standard and let's obfuscate the interface to one of the most performance sensitive part of an application. Hell, lets build multiple obfuscators for each language, each with it's own astonishing behavior that turns pathological once you actually get usage on the app.

      • 7bit 2 days ago

        You know whats also a 50 year old industry Standard? Assembler. Yet, Nobody writes it any more.

    • bb88 2 days ago

      And yet, the Vietnam of Computer Science paper didn't stop the creation of ORM's.

      There's something fundamentally broken with SQL syntax, and yelling at people to "Just Use SQL" doesn't really help.

      • mbrumlow 2 days ago

        Its better than sitting silent and doing nothing. What it does is validate other peoples feelings who feel the same way. And just maybe, that validation will lead to them saying "fuck ORMs" when some junior dev comes in and tries to use one.

        • bb88 a day ago

          Fair, but it also reinforces the stereotype that the pro-SQL types are obstinate relics yelling at the youngsters to "Get off my lawn!"

          There's a pattern of "We can't change X, so we'll write Y that transpiles down to X". It happens often with closed source tools and others that can't or won't implement new languages. Verilog, SQL, Javascript, all fit that bill.

          E.g. Why is Javascript the only first class language for the browser? For the longest time JS was the only game in town.

      • throwaway19972 2 days ago

        > There's something fundamentally broken with SQL syntax

        ? Are you referring to something specific?

      • sivakon 2 days ago

        excuse me, what is Vietnam of Computer Science paper? what’s vietnam has to do with it?

    • imtringued 2 days ago

      Sorry but anyone who needs to build queries dynamically e.g. for extended/advanced search or ACL is going to significantly benefit from being able to seamlessly mix regular code and query logic. Compare that to the old way of concatenating SQL fragments. Shudder.