It's amusing that nixpkgs contributors have spent thousands of human-hours to craft a module system suitable for patching and wrapping any piece of software to accept static configuration, but NixOS, home-manager, and now json2dir end up producing an activation script which litters the filesystem with said configuration.
Everything runs just so much better if the binaries in your profile are wrapper scripts that essentially run "program --config /nix/store/<hash>-program.config". Each file that needs to be copied or symlinked to a "blessed" location in the global mount namespace via an activation script is a failure opportunity, which breaks the atomicity of profile activation and leaves you (or some complicated logic in NixOS/home-manager) to clean up the mess.
Even in the case that a program cannot be patched to run this way, it is easy these days to bind-mount into a clean namespace via bwrap or similar. Alas, shared libraries are kind of the Achilles' heel of this approach.
This is a legitimately hard problem (as an `emacs` user on NixOS I see both sides of it and their merits).
NixOS is directionally the future but the implementation is self-crippled by ideology in a few important places. There is absolutely no reason why `buildFHSEnv` couldn't come by default rather than `/sw/` or `/run`: links into the store are links into the store, putting them in a place that breaks everything? That's incompatible by design and you know it's intentional because symlinks are cheap you could just do both!
Ditto `nix-ld` being necessary, it's a great piece of work but the dynamic linker should be in the normal place and know about all the libraries on the system by default. It's possible to do this in my NixOS modules? `uv add flash-attention-blah`? Works without any trouble on my machine. But it was a super pain to set up that most people won't put up with.
`home-manager` is awesome, it pioneered a bunch of great stuff, but it's not maintained with the vigor it once was, and some dated ideas got wired in really deep. I still run it, and I probably will forever because it slays at some stuff, but that's the nice thing about symlinking into a a store! I can use it where it works well, and use other stuff where it's trouble. This is the magic of NixOS. The next thing I'm trying is https://github.com/outfoxxed/impurity.nix, which comes highly recommended by heavy Nix people I know.
I think it's time to just update NixOS to run things properly by default. It can be done with zero sacrifice on real pure builds and caching/substitors working properly and all of that. I sometimes call Nix "advanced alien technology that was badly damaged on crash re-entry". @jade is a boss and says kind of the same thing a different way.
But again, the beauty of NixOS is that you can do this yourself, an overlay is a pure function from the world as it is to the world as it ought to be.
> Ditto `nix-ld` being necessary, it's a great piece of work but the dynamic linker should be in the normal place and know about all the libraries on the system by default.
Isn't that part of the point of having NixOS? Dynamic linking into whatever seems best from the nix-store sounds handy until you realise it's just going back to a regular mutable distro where the state is whatever and build instructions start looking like "install all these libraries and cross your fingers just in case" and "works on my machine"™ reigns.
There's a range of options on this and numerous good mental models for it: flakes are "dirty", Haskell does things in a very granular regime of mutability bounded context combinators, NixOS has to cope with changing hardware (NixOS modules at the absolute apex of "official"-ness escape hatch it, you have to be pretty explicit to get a reliable pin of the kernel, I'm running `linuxPackages_testing` on this machine I'm typing from because I'm setting up a 5090 and need "the newest one" on everything: 2 months ago? 6.16-rc3, today: I think it's 6.17-rc1 or something, haven't looked).
So, much like a `nix flake check` might fail and you want your CI to flag that, a `nh home switch .` might be necessary to get the editor you need to fix the fail. Functional programming has decades of consensus on every possible variation of this.
There are any number of things you could do here (and the message linking you to the `nix-ld` website is an improvement over `libstdc++.so.6 not found blah`). But breaking a library that statically links everything NVIDIA ever wrote precisely so that it will run anywhere because you have a canonical `CC` in your own `NIX_` environment variable prefix set and you just refuse to let grubby ubuntu software see it?
That's incompatible by design. // hypermodern // nixos has a principle that I realize isn't in the `README.md`: it's never incompatible by design for no upside.
I'm kinda of the opinion that the real option to handle dotfiles is to override/overlay the package itself with the dotfiles, patching if necessary in order to make it look to the dotfile inside the store, so you can copy the closure of *your* whole app to any machine even if they don't use/can't use nix tho.
I don't think json is the right language for this. You can't really have comments, and your scripts have to be in a single line (because json doesn't support multi-line strings).
Yes, but you can generate JSON with other tools easily. home.md describes how you can do it with Nix, but just as easily you can use Cue or something else.
Edit: mentioned this in the README explicitly. Thanks!
You could at least use Hujson[0], which adds comments and trailing commas; makes for a more convenient editing experience, at the price of an additional parsing step.
I actually do that a lot, I often keep my data in dirs and files as a flexible universal format. And I have a few generic scripts to transform it to any format needed.
I have a js proxy project that auto-persists changes out live, as you change the object, rathre like this. I really need to get back to it.
The advantage of being able to see state easily is incredible. It's so scriptable. I only demo'ed it for myself, but I've also run a git-auto-commit program on the data as it changes over time, which is much more useful commits to look at over time than seeing data in a huge JSON file change.
I really really hope we can start using the hierarchical file system to hold data. For transport, its convenient to have data glommed together, but I think we're really missing out on end user programming and malleable systems by having these rich data formats everywhere and keeping the filesystem dumb.
- OP project manages contents of multiple files as a single JSON with the intention of tracking that one single file in git, and splits it into the original files when you apply it
- Your tool sounds like it can do the same thing, split one JSON file into multiple files, but it’s geared for use the other way around, to track in git as separate files the pieces that make up the total JSON as a.
Both tools can probably be used for the same, it’s up to the user to decide if the combined file is the result and the split files are for git or the other way around.
And fwiw, I agree with you that keeping the split up thing in git is more helpful for reading diffs than a single massive JSON file. I have some scripts in one of my projects too, that takes fragments split across multiple files which are separately tracked, and combine those into single JSON files when I use them.
Well, if you're generating JSON with Nix, you don't have to put everything inside of one file. It would be a better idea to split it up into multiple. You can also use builtins.readFile for reading config files which don't have to be generated in a complex manner. It's up to you to choose, I just kept everything inside of one file since it makes for a simpler example.
Edit: I have updated the documentation to mention this explicitly, thanks!
My bespoke clipboard manager also uses the filesystem as the primary data structure.
h/$serial_number/$mime_type/{data, index}
H for history. data has the actual paste data. index has metadata useful for search - window name, day name, I also include wifi network so I can find clipboard history in terms of place, if I remember it that way. It also includes a copy of the data file if it's a text paste. You can include anything really it's fairly flexible. You can write whatever executable you want to the *-posthook file and they are all executed with argv having the path to the history entry directory. You can then modify the index as you please.
I have a few frontends to actually use this clipboard history as well. One is a gtk3 frontend searchbar + list below. Another is a cli fzf based thing.
Since the data structure is just the filesystem it's really composable and amazing.
Various things like blacklisting windows, "pausing" clipboard history, etc are all just files as well.
If you create a pause file it will pause (there's an if test -f pause check). You can add a grep -E pattern to the blacklist file and it won't paste from those window names.
Unlimited history since I don't care about space. But it does support wrapping around after N items.
Sync with phones is one thing i have to figure out...syncing across my different computers is dead easy of course.
Yeah, in general (a bit of a tangent), ideas from Plan 9 are really powerful. For example, the Acme text editor exposes it's API as a file system (it's represented via Unix sockets in plan9port, but FUSE is available as well there). It's easy to write scripts to manipulate the editor, and quite fun.
I 100% have been inspired plan9, not actually a tangent at all!
I never spent much time in Acme. I did use wmii as a daily driver window manager for a number of years, which was also 9p based! I wasn't great at scripting it, and the docs have always been basically non-existent for it's 9p interfaces... but I did poke around and look at things, which was super easy to do because it was just a filesystem! I did find this short example of some wmii examples, inside a Chicken Scheme 9p write-up, which is lovely to see online: https://wiki.call-cc.org/eggref/5/9p#utility-procedures
It's a fine distinction, but I do like json2dir and my own work a bit more than 9p because with apps that expose 9p, if the app goes away, so does the filesystem! 9p is an interface to talk to some process state. But if you just keep state on the regular filesystem, it will stick around & persist beyond the life of the process. The downside is your app needs to watch for that state changing, needs to be able to reconcile changes as they happen. On the upside though, now you can use a lot of great filesystem utilities: for example btrfs or zfs snapshots or incremental-backups are going to work as usual!
Being able to easily script your world is-I think- a missing key to unlocking personal computing. Different technologies discussed, but another serendipitous thread on personal computing from today: https://news.ycombinator.com/item?id=44837783
It's amusing that nixpkgs contributors have spent thousands of human-hours to craft a module system suitable for patching and wrapping any piece of software to accept static configuration, but NixOS, home-manager, and now json2dir end up producing an activation script which litters the filesystem with said configuration.
Everything runs just so much better if the binaries in your profile are wrapper scripts that essentially run "program --config /nix/store/<hash>-program.config". Each file that needs to be copied or symlinked to a "blessed" location in the global mount namespace via an activation script is a failure opportunity, which breaks the atomicity of profile activation and leaves you (or some complicated logic in NixOS/home-manager) to clean up the mess.
Even in the case that a program cannot be patched to run this way, it is easy these days to bind-mount into a clean namespace via bwrap or similar. Alas, shared libraries are kind of the Achilles' heel of this approach.
This is a legitimately hard problem (as an `emacs` user on NixOS I see both sides of it and their merits).
NixOS is directionally the future but the implementation is self-crippled by ideology in a few important places. There is absolutely no reason why `buildFHSEnv` couldn't come by default rather than `/sw/` or `/run`: links into the store are links into the store, putting them in a place that breaks everything? That's incompatible by design and you know it's intentional because symlinks are cheap you could just do both!
Ditto `nix-ld` being necessary, it's a great piece of work but the dynamic linker should be in the normal place and know about all the libraries on the system by default. It's possible to do this in my NixOS modules? `uv add flash-attention-blah`? Works without any trouble on my machine. But it was a super pain to set up that most people won't put up with.
`home-manager` is awesome, it pioneered a bunch of great stuff, but it's not maintained with the vigor it once was, and some dated ideas got wired in really deep. I still run it, and I probably will forever because it slays at some stuff, but that's the nice thing about symlinking into a a store! I can use it where it works well, and use other stuff where it's trouble. This is the magic of NixOS. The next thing I'm trying is https://github.com/outfoxxed/impurity.nix, which comes highly recommended by heavy Nix people I know.
I think it's time to just update NixOS to run things properly by default. It can be done with zero sacrifice on real pure builds and caching/substitors working properly and all of that. I sometimes call Nix "advanced alien technology that was badly damaged on crash re-entry". @jade is a boss and says kind of the same thing a different way.
But again, the beauty of NixOS is that you can do this yourself, an overlay is a pure function from the world as it is to the world as it ought to be.
EDIT: I know talk is cheap and code wins arguments, and I know this is about a year overdue and not released yet, but it's got beta testers now, it's coming: https://gist.github.com/b7r6/721f62d6431c77b64592a55706d87fd...
> Ditto `nix-ld` being necessary, it's a great piece of work but the dynamic linker should be in the normal place and know about all the libraries on the system by default.
Isn't that part of the point of having NixOS? Dynamic linking into whatever seems best from the nix-store sounds handy until you realise it's just going back to a regular mutable distro where the state is whatever and build instructions start looking like "install all these libraries and cross your fingers just in case" and "works on my machine"™ reigns.
There's a range of options on this and numerous good mental models for it: flakes are "dirty", Haskell does things in a very granular regime of mutability bounded context combinators, NixOS has to cope with changing hardware (NixOS modules at the absolute apex of "official"-ness escape hatch it, you have to be pretty explicit to get a reliable pin of the kernel, I'm running `linuxPackages_testing` on this machine I'm typing from because I'm setting up a 5090 and need "the newest one" on everything: 2 months ago? 6.16-rc3, today: I think it's 6.17-rc1 or something, haven't looked).
So, much like a `nix flake check` might fail and you want your CI to flag that, a `nh home switch .` might be necessary to get the editor you need to fix the fail. Functional programming has decades of consensus on every possible variation of this.
There are any number of things you could do here (and the message linking you to the `nix-ld` website is an improvement over `libstdc++.so.6 not found blah`). But breaking a library that statically links everything NVIDIA ever wrote precisely so that it will run anywhere because you have a canonical `CC` in your own `NIX_` environment variable prefix set and you just refuse to let grubby ubuntu software see it?
That's incompatible by design. // hypermodern // nixos has a principle that I realize isn't in the `README.md`: it's never incompatible by design for no upside.
I agree and worry about that idea https://fzakaria.com/2025/07/07/home-manager-is-a-false-enli...
Meanwhile, more and more software are turds that insist on overwriting their own config files :-(
Cool project!
I'm kinda of the opinion that the real option to handle dotfiles is to override/overlay the package itself with the dotfiles, patching if necessary in order to make it look to the dotfile inside the store, so you can copy the closure of *your* whole app to any machine even if they don't use/can't use nix tho.
I don't think json is the right language for this. You can't really have comments, and your scripts have to be in a single line (because json doesn't support multi-line strings).
Yes, but you can generate JSON with other tools easily. home.md describes how you can do it with Nix, but just as easily you can use Cue or something else.
Edit: mentioned this in the README explicitly. Thanks!
You could at least use Hujson[0], which adds comments and trailing commas; makes for a more convenient editing experience, at the price of an additional parsing step.
[0]: https://github.com/tailscale/hujson
I guess this could be a cross-platform format but you could also just store it as a sequence of commands
Would be the same as what's in the readme:Man, I feel like I'd rather reengineer this as sh2dir, using shell commands like echo, cat, ln, mkdir, touch, and so on...
And dir2json?
I actually do that a lot, I often keep my data in dirs and files as a flexible universal format. And I have a few generic scripts to transform it to any format needed.
I have a js proxy project that auto-persists changes out live, as you change the object, rathre like this. I really need to get back to it.
The advantage of being able to see state easily is incredible. It's so scriptable. I only demo'ed it for myself, but I've also run a git-auto-commit program on the data as it changes over time, which is much more useful commits to look at over time than seeing data in a huge JSON file change.
I really really hope we can start using the hierarchical file system to hold data. For transport, its convenient to have data glommed together, but I think we're really missing out on end user programming and malleable systems by having these rich data formats everywhere and keeping the filesystem dumb.
If I understand correctly:
- OP project manages contents of multiple files as a single JSON with the intention of tracking that one single file in git, and splits it into the original files when you apply it
- Your tool sounds like it can do the same thing, split one JSON file into multiple files, but it’s geared for use the other way around, to track in git as separate files the pieces that make up the total JSON as a.
Both tools can probably be used for the same, it’s up to the user to decide if the combined file is the result and the split files are for git or the other way around.
And fwiw, I agree with you that keeping the split up thing in git is more helpful for reading diffs than a single massive JSON file. I have some scripts in one of my projects too, that takes fragments split across multiple files which are separately tracked, and combine those into single JSON files when I use them.
Well, if you're generating JSON with Nix, you don't have to put everything inside of one file. It would be a better idea to split it up into multiple. You can also use builtins.readFile for reading config files which don't have to be generated in a complex manner. It's up to you to choose, I just kept everything inside of one file since it makes for a simpler example.
Edit: I have updated the documentation to mention this explicitly, thanks!
> file system to hold data
My bespoke clipboard manager also uses the filesystem as the primary data structure.
h/$serial_number/$mime_type/{data, index}
H for history. data has the actual paste data. index has metadata useful for search - window name, day name, I also include wifi network so I can find clipboard history in terms of place, if I remember it that way. It also includes a copy of the data file if it's a text paste. You can include anything really it's fairly flexible. You can write whatever executable you want to the *-posthook file and they are all executed with argv having the path to the history entry directory. You can then modify the index as you please.
I have a few frontends to actually use this clipboard history as well. One is a gtk3 frontend searchbar + list below. Another is a cli fzf based thing.
Since the data structure is just the filesystem it's really composable and amazing.
Various things like blacklisting windows, "pausing" clipboard history, etc are all just files as well.
If you create a pause file it will pause (there's an if test -f pause check). You can add a grep -E pattern to the blacklist file and it won't paste from those window names.
Unlimited history since I don't care about space. But it does support wrapping around after N items.
Sync with phones is one thing i have to figure out...syncing across my different computers is dead easy of course.
Yeah, in general (a bit of a tangent), ideas from Plan 9 are really powerful. For example, the Acme text editor exposes it's API as a file system (it's represented via Unix sockets in plan9port, but FUSE is available as well there). It's easy to write scripts to manipulate the editor, and quite fun.
I 100% have been inspired plan9, not actually a tangent at all!
I never spent much time in Acme. I did use wmii as a daily driver window manager for a number of years, which was also 9p based! I wasn't great at scripting it, and the docs have always been basically non-existent for it's 9p interfaces... but I did poke around and look at things, which was super easy to do because it was just a filesystem! I did find this short example of some wmii examples, inside a Chicken Scheme 9p write-up, which is lovely to see online: https://wiki.call-cc.org/eggref/5/9p#utility-procedures
It's a fine distinction, but I do like json2dir and my own work a bit more than 9p because with apps that expose 9p, if the app goes away, so does the filesystem! 9p is an interface to talk to some process state. But if you just keep state on the regular filesystem, it will stick around & persist beyond the life of the process. The downside is your app needs to watch for that state changing, needs to be able to reconcile changes as they happen. On the upside though, now you can use a lot of great filesystem utilities: for example btrfs or zfs snapshots or incremental-backups are going to work as usual!
Being able to easily script your world is-I think- a missing key to unlocking personal computing. Different technologies discussed, but another serendipitous thread on personal computing from today: https://news.ycombinator.com/item?id=44837783