> For the price of a Mac Studio, I could have bought 1 billion uncached Opus tokens... or 4 years of the $200/month Max plan. And that’s not counting all the hours I spent chasing the whale through git diffs, network logs, and benchmark results.
perhaps private inference cloud APIs would be a good solution.
Doesn't that presume, inter alia, that token subsidization will continue for the next four years? Perhaps dramatically increasing token rates, forthcoming efficiencies, and the privacy dividend of local-only inference would make this a really great option.
> For the price of a Mac Studio, I could have bought 1 billion uncached Opus tokens... or 4 years of the $200/month Max plan. And that’s not counting all the hours I spent chasing the whale through git diffs, network logs, and benchmark results.
perhaps private inference cloud APIs would be a good solution.
Doesn't that presume, inter alia, that token subsidization will continue for the next four years? Perhaps dramatically increasing token rates, forthcoming efficiencies, and the privacy dividend of local-only inference would make this a really great option.