Making Atuin sync 32x faster with packfiles

(blog.atuin.sh)

30 points | by ossusermivami 5 days ago ago

12 comments

  • ashkankiani 4 hours ago

    I don't want to be mean, but in my personal experience and projects, I have a kind of rule. If I get a 5% to 50% speedup, then I probably did a good job optimizing. If I get a 2x to 5x speedup then I probably was fixing up a prototype. If I get >10x speedup, I fucked up the first time.

    • reacharavindh 3 hours ago

      By that logic there is a lot of fucked up software systems in the world we live in..

      See it as it is. Atuin is a useful tool that stashes shell history onto SQLite locally. Then it expands to syncing them across machines per identity. They implemented it as simple and useful as it could be for what was needed at the time(shell history is not the scale of some commerce orders). Once they actually realized where the opportunity is to improve it(upon a functional sync already in place), they do and post about it at HN. I don’t even use their sync. I’m happy to learn about packfiles and the idea of optimizing sync like this. Not doing this at first is not a fuck up.. it is the evolution of good practical software.

      • roroh4310 2 hours ago

        > By that logic there is a lot of fucked up software systems in the world we live in..

        Yes? Yes.

        (This is not a jab at Atuin per se. Just a comment on the state of software we are in.)

      • gjvc 2 hours ago

        > By that logic there is a lot of fucked up software systems in the world we live in..

        Correct.

    • s08148692 2 hours ago

      Hard disagree

      Sometimes the naive implementation is fine. It gets you a MVP, and it can take a product far without ever needing an improvement. It lets you build a suite of tests to verify behaviour, and informs you that it's a useful feature in the first place. There's no point spending time on a perfect implementation of something when the slow build might be good enough and it could be something nobody wanted in the first place.

      Once it's proved its worth you can then go back and spend some extra effort making it fast, with the safety net of tests to make sure the new (likely more complex and hard to reason about) implementation meets the spec

    • shawabawa3 4 hours ago

      Surely it's more like, at the start of the project X wasn't even close to being an issue. As the project grows and other things get improved, X suddenly appears as being way too slow

      I've seen this dozens of times. If you've never replaced something with something 10x faster you're massively over engineering at the start of a project

    • KetoManx64 3 minutes ago

      So what if you did? Have you never learned from the mistakes you made in V0.1 of your software and then proceeded to make it 10x better?

    • rq1 3 hours ago

      Every improvement is welcome. Keep up the good work.

      A happy atuin user.

    • asgeirn 3 hours ago

      Not really. Premature optimizations have their own set of problems. It's generally better to keep the design simple and tackle optimization if performance becomes an issue.

      I've been using atuin for years without noticing this, so it has definitely not been a significant problem.

  • bita_nidir 3 hours ago

    OK, I had a laugh at the TP-esque quote

      This is not just a small performance improvement. This is a paradigm shift in how Atuin does sync.
      — lots of numbers on a GPU, August 2026
  • cowlevel 5 hours ago

    > We sync many thousands of records in single-digit seconds.

    Is that meant to be impressive performance? I'd never heard of Atuin but apparent it syncs your shell history between machines using a Blockchain?