> No CFO approves a $1.4 million annual commitment on the strength of a chart showing that the commitment scales.
Except the company probably approved a budget for AWS or another cloud provider, and basically gave a blank check to developers to deploy whatever is needed. So developers are going to just deploy MSK or whatever is trendy, instead of trying to get the most throughput from the servers they got from IT.
“My formative memory of Python was when the Quake Live team used it for the back end work, and we wound up having serious performance problems with a few million users. My bias is that a lot (not all!) of complex “scalable” systems can be done with a simple, single C++ server.”
Other than batch jobs, I can't think of a problem that can be solved these days that doesn't also require high availability - at the very least they require a warm standby.
So take 16 independent computers and have them each solve the problem separately. You'd still be saving 80% compared to the paper's benchmarks. It wasn't close.
McSherry does a lot of interesting work on making monotonic/incremental distributed systems efficient (e.g. Differential and Timely Dataflow). Those kinds of systems scale much more linearly.
I dunno, it often matters a lot less than you think when something goes down. And distributed systems have a knack for going down in a much less predictable way, it's not going to automatically make your system more reliable.
(modern server hardware and operating systems are also surprisingly reliable nowadays, which makes it harder to reach breakeven with a distributed design)
> No CFO approves a $1.4 million annual commitment on the strength of a chart showing that the commitment scales.
Except the company probably approved a budget for AWS or another cloud provider, and basically gave a blank check to developers to deploy whatever is needed. So developers are going to just deploy MSK or whatever is trendy, instead of trying to get the most throughput from the servers they got from IT.
“My formative memory of Python was when the Quake Live team used it for the back end work, and we wound up having serious performance problems with a few million users. My bias is that a lot (not all!) of complex “scalable” systems can be done with a simple, single C++ server.”
https://x.com/ID_AA_Carmack/status/1210997702152069120
Other than batch jobs, I can't think of a problem that can be solved these days that doesn't also require high availability - at the very least they require a warm standby.
So take 16 independent computers and have them each solve the problem separately. You'd still be saving 80% compared to the paper's benchmarks. It wasn't close.
McSherry does a lot of interesting work on making monotonic/incremental distributed systems efficient (e.g. Differential and Timely Dataflow). Those kinds of systems scale much more linearly.
I dunno, it often matters a lot less than you think when something goes down. And distributed systems have a knack for going down in a much less predictable way, it's not going to automatically make your system more reliable.
(modern server hardware and operating systems are also surprisingly reliable nowadays, which makes it harder to reach breakeven with a distributed design)
Sometimes your entire business is just a laptop with a python dictionary and a backup power supply
the point the article makes is good (albeit not new). the style sounds very LLM to me.