I wonder to what degree it’s actually necessary to explicitly manage memory spilling to disk like this. Want unified interface over non durable memory + disk? There already is one: memory with swap.
To get good performance from this strategy your memory layout already needs to be optimized to pages boundaries etc to have good sympathy for underlying swap system, and you can explicitly force pages to stay in memory with a few syscalls.
I think the article could use more on the cache invalidation and write-through (?) behavior. Are updates to the same file batched or written back to S3 immediately? Do you do anything with write conflicts, which one wins?
Foyer is a great open source contribution from RisingWave
We built a S3 read-through cache service for s2.dev so that multiple clients could share a Foyer hybrid cache with key affinity, https://github.com/s2-streamstore/cachey
No idea, if I see a medium link I just ignore it. Substack is heading the same way for me too, it seems to be self-promotion, shallow-takes, and spam more than anything real.
The page loads a "subscribe to author" modal pretty quickly after the page loads. You may have partially blocked it, so you won't see the modal but it still prevents scroll.
Firefox has a lot of weird little pop up ads these days. It seems like this is a very recent phenominon. Is this actually Firefox doing this or some kind of plug-in accidentally installed?
Yes, definitely. S3 has a time to first byte of 50-150ms (depending on how lucky you are). If you're serving from memory that goes to ~0, and if you're serving from disk, that goes to 0.2-1ms.
It will depend on your needs though, since some use cases won't want to trade off the scalability of S3's ability to serve arbitrary amounts of throughput.
In that case you run the proxy service load balanced to get desired throughput or run a sidecar/process in each compute instance where data is needed .
You are limited anyway by the network capacity of the instance you are fetching the data from .
These are, effectively, different use cases. You want to use (and pay for) Express One Zone in situations in which you need the same object reused from multiple instances repeatedly, while it looks like this on-disk or in-memory cache is for when you may want the same file repeatedly used from the same instance.
Is it the same instance ? Rising wave (and similar tools )are designed to run in production on a lot of distributed compute nodes for processing data , serving/streaming queries and running control panes .
Even for any single query it will likely run on multiple nodes with distributed workers gathering and processing data from storage layer, that is whole idea behind MapReduce after all.
I use Foyer in-memory (not hybrid) in ZeroFS [0] and had a great experience with it.
The only quirk I’ve experienced is that in-memory and hybrid modes don’t share the same invalidation behavior. In hybrid mode, there’s no way to await a value being actually discarded after deletion, while in-memory mode shows immediate deletion.
This is interesting. I would be curious to try a setup where I keep a local hybrid cache and transition blocks to deep storage for long-term archival via S3 rules.
Some napkin math suggests this could be a few dollars a month to keep a few TB of precious data nearline.
Restore costs are pricy but hopefully this is something that's only hit in case of true disaster. Are there any techniques for reducing egress on restore?
Storage Gateway is an appliance that you connect multiple instances to, this appears to be a library that you use in your program to coordinate caching for that process.
> foyer draws inspiration from Facebook/CacheLib, a highly-regarded hybrid cache library written in C++, and ben-manes/caffeine, a popular Java caching library, among other projects.
S3 Mountpoint is exposing a POSIX-like file system abstraction for you to use with your file-based applications. Foyer appears to be a library that helps your application coordinate access to S3 (with a cache), for applications that don't need files and you can change the code for.
> Cost is reduced because far fewer requests hit S3
I wonder. Given how cheap are S3 GET requests you need a massive number of requests to make provisioning and maintaining the cache server cheaper than the alternative.
I wonder to what degree it’s actually necessary to explicitly manage memory spilling to disk like this. Want unified interface over non durable memory + disk? There already is one: memory with swap.
Materialize.com switched from explicit disk cache management to “just use swap” and saw substantial performance improvement. https://materialize.com/blog/scaling-beyond-memory/
To get good performance from this strategy your memory layout already needs to be optimized to pages boundaries etc to have good sympathy for underlying swap system, and you can explicitly force pages to stay in memory with a few syscalls.
> There already is one: memory with swap.
Actually, there's already two. The other one: just read from disk (and let OS manage caches).
TIL https://kubernetes.io/blog/2025/03/25/swap-linux-improvement...
Here is a non-medium article with the same content: https://risingwave.com/blog/the-case-for-hybrid-cache-for-ob...
I think the article could use more on the cache invalidation and write-through (?) behavior. Are updates to the same file batched or written back to S3 immediately? Do you do anything with write conflicts, which one wins?
Foyer is a great open source contribution from RisingWave
We built a S3 read-through cache service for s2.dev so that multiple clients could share a Foyer hybrid cache with key affinity, https://github.com/s2-streamstore/cachey
Has Medium stopped working on Firefox for anybody else? Once the page is finished loading, it stops responding to scroll events.
No idea, if I see a medium link I just ignore it. Substack is heading the same way for me too, it seems to be self-promotion, shallow-takes, and spam more than anything real.
The page loads a "subscribe to author" modal pretty quickly after the page loads. You may have partially blocked it, so you won't see the modal but it still prevents scroll.
Same here. Meanwhile I close a link/page as soon as I realize it's on medium.
Maybe you have an ad-blocker that just hides the popup but does not restore scrolling (scrolling is usually prevented when popups are visible)
Firefox has a lot of weird little pop up ads these days. It seems like this is a very recent phenominon. Is this actually Firefox doing this or some kind of plug-in accidentally installed?
Hm, I haven't seen that. Perhaps it's worth reviewing your plugins
I avoid medium where possible.
If I could pipe text content to my terminal with confidence, I would.
Seems ok for me on Firefox 143.0.1
Have you tried using reader mode?
Distributed Chroma, the open-source project backing Chroma Cloud, uses Foyer extensively:
https://github.com/chroma-core/chroma/blob/2cb5c00d2e97ef449...
Does S3 really have that high of a latency? So high that —— if you run a static file several in an EC2, would that be faster than S3?
Yes, definitely. S3 has a time to first byte of 50-150ms (depending on how lucky you are). If you're serving from memory that goes to ~0, and if you're serving from disk, that goes to 0.2-1ms.
It will depend on your needs though, since some use cases won't want to trade off the scalability of S3's ability to serve arbitrary amounts of throughput.
In that case you run the proxy service load balanced to get desired throughput or run a sidecar/process in each compute instance where data is needed .
You are limited anyway by the network capacity of the instance you are fetching the data from .
S3 has a low-latency offering[0] which promises single digit millisecond latency, I’m surprised not to see it mentioned.
[0]: https://aws.amazon.com/s3/storage-classes/express-one-zone/
These are, effectively, different use cases. You want to use (and pay for) Express One Zone in situations in which you need the same object reused from multiple instances repeatedly, while it looks like this on-disk or in-memory cache is for when you may want the same file repeatedly used from the same instance.
Is it the same instance ? Rising wave (and similar tools )are designed to run in production on a lot of distributed compute nodes for processing data , serving/streaming queries and running control panes .
Even for any single query it will likely run on multiple nodes with distributed workers gathering and processing data from storage layer, that is whole idea behind MapReduce after all.
Also, aren't most people putting Cloudfront in front of S3 anyway?
For CDN use-cases yes, but not for DB storage-compute separation use-cases as described here.
I use Foyer in-memory (not hybrid) in ZeroFS [0] and had a great experience with it.
The only quirk I’ve experienced is that in-memory and hybrid modes don’t share the same invalidation behavior. In hybrid mode, there’s no way to await a value being actually discarded after deletion, while in-memory mode shows immediate deletion.
[0] https://github.com/Barre/ZeroFS
This is interesting. I would be curious to try a setup where I keep a local hybrid cache and transition blocks to deep storage for long-term archival via S3 rules.
Some napkin math suggests this could be a few dollars a month to keep a few TB of precious data nearline.
Restore costs are pricy but hopefully this is something that's only hit in case of true disaster. Are there any techniques for reducing egress on restore?
The easy way to achieve that, is using ZeroFS NBD server with ZFS L2ARC (L2ARC with local storage and “main” pool on ZeroFS).
Interesting! What would be a typical use-case of ZeroFS? could I use this to store my Immich and Jellyfin data on S3 so I don't need disk?
If you don't mind paying about a dollar to stream one of your own movies as well as a couple of bucks per year to store it.
You don’t have to use AWS S3, any compatible implementation will work.
That should work!
The Jellyfin metadata would certainly be a fit but what about streaming video content i.e. sequential reads of large files with random access?
If you have the network that matches, it should be perfectly fine.
Sounds exactly like AWS Storage Gateway, how does it compare?
Storage Gateway is an appliance that you connect multiple instances to, this appears to be a library that you use in your program to coordinate caching for that process.
How does it compare to cachelib
Essentially CacheLib in Rust
> foyer draws inspiration from Facebook/CacheLib, a highly-regarded hybrid cache library written in C++, and ben-manes/caffeine, a popular Java caching library, among other projects.
https://github.com/foyer-rs/foyer
How does this compare to S3 Mountpoint with caching?
S3 Mountpoint is exposing a POSIX-like file system abstraction for you to use with your file-based applications. Foyer appears to be a library that helps your application coordinate access to S3 (with a cache), for applications that don't need files and you can change the code for.
Interesting to compare against ZeroFs
I haven’t used it yet but I have been looking for something like this for a long time. Kudos!
Very curious about comparison between the rclone etc.. s3 caching vs this one.
Foyer is great!
> Cost is reduced because far fewer requests hit S3
I wonder. Given how cheap are S3 GET requests you need a massive number of requests to make provisioning and maintaining the cache server cheaper than the alternative.