Auth to OpenAI's developer platform isn't working as well, so something is seriously wrong. Initially, I thought this was related to Astra, but seeing as how xai, anthropic, and openai are all having issues at the same time, it has to be a common provider, right? Maybe cloudflare?
As an aside, reminder to consider keeping your status page on a separate domain lest the problem be e.g. with DNS. You can see cloudflare does this (and we all know github does too as they give us frequent reasons to check it)
We've all noticed that when one company announces a new model, the other one announces a new model usually within hours. I guess now we're syncing downtime too.
A nice chance to try sonnet again, must say, I'm not missing the load bearing assumptions, honest read of the permission matrix, what's genuinely a Django convention and what's a design choice, what's worth my thinking, and what's worth being precise about.
Long story short, it seems to be faster and less vocal but not much dumber (it's just my thinking partner, so I read a lot of the output as I create a large data model).
I've almost entirely stopped using Opus 5, at least with Sonnet I know what i'm getting -- Fable for the complex stuff and Sonnet for the precision changes.
The verbosity of Opus 5 isn't even my issue, it's consistency. For every 10 tasks Opus 5 accomplishes, there's at least one task that Opus 5 does just an atrocious job of, or a debugging investigation that it just completely goes off the rails on.
My engineering has changed so much that I'm now using this time to catch up on emails and think about design and architecture for the next leg of work. (And post on HN, of course.)
I would never have predicted this a year ago.
I always scoffed about engineers not doing work during Github outages - I'd always find some kind of other engineering to do if PRs or builds were piling up.
But during model outages, and since I'm not actively set up to use OpenAI/Codex at the moment, I'll just find something else productive to do. I don't think I'm ever going to write code by hand again unless I'm fixing something the AI can't manage or the change is small enough. Why even try when the machine is 100x faster?
Should I give Codex another try? It's been a month or so since I last used it, and it always felt inferior at Rust and TypeScript.
I think if you've got the money, you should set up a coding harness that you can plug to openrouter and then be able to switch quickly, as these new models seem to be coming out pretty regularly now
switching subs is kind of a PITA since you need to commit to a month at least
Isn't this the classic blackout scenario? Claude goes down, then people move over to Codex, which is overwhelmed, and crashes, so people move to Grok...
Bit of a misleading status page, if you click in you can see that grok 4.5 and 4.6 etc are totally down, with the rest of the models showing as "up". I_strongly_ suspect they are not weighting it to actual number of requests!
The interesting thing is they changed default mode for Claude code to auto mode and auto mode uses sonnet to decide whether the command is safe or not. With their Sonnet model outage, the entire thing stopped working.
here is the error it was throwing:
> Error: claude-sonnet-5[1m] is temporarily unavailable (overloaded), so auto mode cannot determine the safety of Edit right now. Wait a moment and then try this action again. If it keeps failing, continue with other tasks that don't require this action and come back to it later. Note: reading files, searching code, and other read-only operations do not require the classifier and can still be used.
Demand > supply. It's impressive that customers have not migrated en masse to other providers, given the frequency of these outages. Perhaps switching costs are greater than some would believe. Or, qualitative differences between models continue to exist, despite matching on public benchmarks.
I did migrate 80% of my tokens. But for some tasks claude models are still the best.
Easy workaround is to work outside US peak hours (europe morning). I love this outages, I am hardly affected, and weekly reset usually promptly follows!
That's quite a reach. More likely someone merged and deployed their vibe-coded PR and is now figuring out how to bring the service back up.
If there's a lot of demand for rollercoaster rides, the rollercoaster will not stop operating; instead the queue of people in front of it will increase.
It's not like a bridge or elevator where we have a certain number of people that can use it, and if one more person joins, the whole structure breaks apart and everybody perishes.
Those guys are running a website that provides an interface for some specific hardware. Just like file hosting providers back in the day selling their terabyte-sized hard disks in 100MB-increments.
> instead the queue of people in front of it will increase.
This can ultimately result in the system breaking. I don't think Anthropic engineers are that much worse than their peers, such that they are 10x more prone to causing outages due to bad deployments.
>> Perhaps switching costs are greater than some would believe.
I haven't switched because there's nothing to switch to that is anywhere as good. I've been making dedicated attempts at using Sol but it falls short, despite what some people claim.
Do you really think it’s a coincidence claude outages always happen when US and EU workdays overlap?
Anthropic is a trillion dollar company and employs way more skilled high-paid engineers than you are btw, you really think it’s a systems issue not capacity
It's been a while since I messed with serious hardware, what's the cooling noise level? I guess they have to be water cooled or the rack format 8u ones would be about 100db
Setting aside annoyance at the downtime, I'm really curious about the reasons for the failures, because I have to imagine there are some novel failure modes when serving these giant models that I haven't experienced with the kind of work I've done.
Anyone out there working in this space who can elucidate us on interesting failure scenarios unique to the space?
This of course could be a coincidence. But I suppose also that if one goes down the others get slammed, so there could be a cascade. If I were another provider I would possibly monitor Claude outages (or vice versa) as a load predictor.
chat.com is down for me but the models seem to still be working. But actually i noticed instability last night at 3am eastern when thinking kept repeatedly failing.
If I didn't know any better, I would have said Grok is using Claude behind the hood. But definitely curious now why it’s happening with both these LLM providers around the same time
Kind of cool how fable 5.1 just worked around these by reassigning subagents to non-failing models with more supervision and now with a hint to Sol and Flash 3.8.
Officially signing up for the $200 Codex plan now. I do like CC, but these errors are happening every week now, and if they (Anthropic) gave regular resets that's one thing, but somehow I'm already at 62% weekly usage after the random reset this Tuesday using Fable 5.1 medium. Just not worth it any longer compared to Codex IMO. I already set up my skills and everything to be generic and not CC specific, so it shouldn't be too hard to transition over. I'll run both for a month and decide.
Couldn't recommend it more. I've been on their $100/mo. plan and they keep randomly resetting my weekly usage limit. It's been wild. Don't know how long it will last, but I couldn't go through all my usage if I tried.
Their Sol model has been doing great for most of my tasks, even at its "Medium" setting.
Much different picture then back when I was using Claude. I would sneeze and half my usage would be gone and then it go down several times in between.
Haha, I was more like: I have to think alone now. But I get the feeling, as someone who has always made a point of using only foss on my own machines (own the means of production so to say), this is a weird time. I don't like it, I hope soon local models get good. But in the mean time... We have this... dependency.
I use a lot of spec driven development today. It does allow me to make workflows that were on my Life's backlog for months or years in 10~20 minutes. It's amazing, but also I lack ownership over the thing made. It's like I paid a contractor.
I can feel that my actual cognitive engineering skills are in decline. Does anyone else see this? To those of you who haven't hand written a PR in months, can you still write your own personal software as easily as you once could?
There are little tools/optimisations that I have been wanting to do for years that I can just throw at AI now. I am almost annoyed sometimes because the little easy things I could do to 'relax' are now gone: if I am working on it is something AI can't do, normally because there are moving parts outside of the repository it can't see (i.e. integration problems, and I hate those).
I just experienced this through OpenRouter and filed a refund request. I'm curious how OpenRouter handles incidents like this across such a large number of models and providers.
Cost and reliability are the two reasons why we don't use Claude in our product. Getting close to one nine, that's not something one can build a reliable product upon. We now use OpenAI with Gemini fallback (or vice versa depending on use case).
Personally I like Claude and have the 20x Max plan but even there I burned through the whole weekly quota with 3 prompts in less than a day using the new Fable 5.1 which is crazy. Now Opus 5 is down. These two issues are really testing my patience.
Every time I see Anthropic ship an issue like this, I'm reminded of Boris Cherny's glorious quote: "At this point it's safe to say that coding is largely solved."
During each forced workflow interruption I look for alternatives. No matter if I can't use the product due to a server outage or due to their weird quota limitations.
This can't be good for retention numbers. Old-school VCs would've ripped them apart in the air. Where did all the expertise go?
I suspect the only reason those alternative providers have better up-time and more generous quotas, is because they don't have nearly the same amount of demand. Notice that Deepseek recently had to increase their pricing, once it gained it popularity.
Yeah, I really like GLM 5.3 Flash with vision. Reminds me of when Claude was lots of fun to work with. It does feel slow, thinking away on the Chinese servers, but it's so incredibly cheap that for a lot of tasks I don't mind it taking a bit longer. I'm finding it mindblowing having entire software features built out for 5 cents, 20 cents.
Codex seems to also be down for some folks (me included) https://www.reddit.com/r/codex/comments/1w69ff6/whats_this_e...
Their status page is hard to believe given how many folks are chiming in to report on Reddit https://status.openai.com/
Auth to OpenAI's developer platform isn't working as well, so something is seriously wrong. Initially, I thought this was related to Astra, but seeing as how xai, anthropic, and openai are all having issues at the same time, it has to be a common provider, right? Maybe cloudflare?
https://status.x.ai
https://status.claude.com
https://status.openai.com
https://www.cloudflarestatus.com
As an aside, reminder to consider keeping your status page on a separate domain lest the problem be e.g. with DNS. You can see cloudflare does this (and we all know github does too as they give us frequent reasons to check it)
https://news.ycombinator.com/item?id=27102020
Codex is down for me too.
Status page is very funny - claiming "We’re fully operational" while https://chatgpt.com gives 404, lol.
Codex/chatgpt working just fine here. FWIW I'm on NordVPN to Buffalo with 8.8.8.8 as my dns.
Codex is also down for me
chatgpt is down as well - interesting
codex is down for me as well. getting a bunch of 404s spit back at me.
We've all noticed that when one company announces a new model, the other one announces a new model usually within hours. I guess now we're syncing downtime too.
A nice chance to try sonnet again, must say, I'm not missing the load bearing assumptions, honest read of the permission matrix, what's genuinely a Django convention and what's a design choice, what's worth my thinking, and what's worth being precise about.
Long story short, it seems to be faster and less vocal but not much dumber (it's just my thinking partner, so I read a lot of the output as I create a large data model).
Reading this made me ill. Nice job!
The other bizarre word I've now noticed and can't understand is "pathological". As in "bug has turned pathological"
The new ones for me are adding extra terms like the 3 faces, to park at the door. (It’s 3 methods that block the thread)
Way leas capable of taking high level instructions and making sensible changes across a codebase.
Good for more precise changes.
I've almost entirely stopped using Opus 5, at least with Sonnet I know what i'm getting -- Fable for the complex stuff and Sonnet for the precision changes.
The verbosity of Opus 5 isn't even my issue, it's consistency. For every 10 tasks Opus 5 accomplishes, there's at least one task that Opus 5 does just an atrocious job of, or a debugging investigation that it just completely goes off the rails on.
My engineering has changed so much that I'm now using this time to catch up on emails and think about design and architecture for the next leg of work. (And post on HN, of course.)
I would never have predicted this a year ago.
I always scoffed about engineers not doing work during Github outages - I'd always find some kind of other engineering to do if PRs or builds were piling up.
But during model outages, and since I'm not actively set up to use OpenAI/Codex at the moment, I'll just find something else productive to do. I don't think I'm ever going to write code by hand again unless I'm fixing something the AI can't manage or the change is small enough. Why even try when the machine is 100x faster?
Should I give Codex another try? It's been a month or so since I last used it, and it always felt inferior at Rust and TypeScript.
I think if you've got the money, you should set up a coding harness that you can plug to openrouter and then be able to switch quickly, as these new models seem to be coming out pretty regularly now
switching subs is kind of a PITA since you need to commit to a month at least
Grok models are struggling too: https://status.x.ai/ reply
Looks like trouble in the SpaceX datacenters.
Isn't this the classic blackout scenario? Claude goes down, then people move over to Codex, which is overwhelmed, and crashes, so people move to Grok...
This ain't just slow/at capacity, this is dead down.
exactly, I think it will be an interesting story in retrospective, what happened here
What could go wrong with building them as fast as physically possible?
What works for me to avoid these problems is not scaling.
Their figures are only getting worse with time interestingly enough with non-inference dipping under 100%. Wonder what is going on
Bit of a misleading status page, if you click in you can see that grok 4.5 and 4.6 etc are totally down, with the rest of the models showing as "up". I_strongly_ suspect they are not weighting it to actual number of requests!
I wonder where eu-west is.
French Guiana ?
Far more likely Ireland, like it is for AWS. Could also be Paris, London, Amsterdam, etc.
I would be _very_ surprised if anthropic relied heavily on spacex datacenters already.
I wouldn't. When you're operating at say 95% realtime capacity suddenly losing even lets say 10% of your compute leads to major pain.
did you forget about the insane Claude Code usage limits a few months back?
those were only relieved once they did the Colossus deal with SpaceX
are you only saying that because of Elon?
The interesting thing is they changed default mode for Claude code to auto mode and auto mode uses sonnet to decide whether the command is safe or not. With their Sonnet model outage, the entire thing stopped working.
here is the error it was throwing:
> Error: claude-sonnet-5[1m] is temporarily unavailable (overloaded), so auto mode cannot determine the safety of Edit right now. Wait a moment and then try this action again. If it keeps failing, continue with other tasks that don't require this action and come back to it later. Note: reading files, searching code, and other read-only operations do not require the classifier and can still be used.
ahhh, sonnet shitting the bed explains why I started getting prompted for basic approvals last night and early this morning
I am on YOLO (not auto), still the same problem.
Is Cloudflare the corrupt for multiple AI labs having problems?
They are experiencing increased errors themselves.
https://www.cloudflarestatus.com/?t=1
Demand > supply. It's impressive that customers have not migrated en masse to other providers, given the frequency of these outages. Perhaps switching costs are greater than some would believe. Or, qualitative differences between models continue to exist, despite matching on public benchmarks.
I think a lot of people, like myself, have. Codex paid users have grown a lot in the past few months.
Demand is actually shared between users and internal. Anthropic uses their own compute for training and building.
I did migrate 80% of my tokens. But for some tasks claude models are still the best.
Easy workaround is to work outside US peak hours (europe morning). I love this outages, I am hardly affected, and weekly reset usually promptly follows!
That's quite a reach. More likely someone merged and deployed their vibe-coded PR and is now figuring out how to bring the service back up.
If there's a lot of demand for rollercoaster rides, the rollercoaster will not stop operating; instead the queue of people in front of it will increase.
It's not like a bridge or elevator where we have a certain number of people that can use it, and if one more person joins, the whole structure breaks apart and everybody perishes.
Those guys are running a website that provides an interface for some specific hardware. Just like file hosting providers back in the day selling their terabyte-sized hard disks in 100MB-increments.
> instead the queue of people in front of it will increase.
This can ultimately result in the system breaking. I don't think Anthropic engineers are that much worse than their peers, such that they are 10x more prone to causing outages due to bad deployments.
>> Perhaps switching costs are greater than some would believe.
I haven't switched because there's nothing to switch to that is anywhere as good. I've been making dedicated attempts at using Sol but it falls short, despite what some people claim.
If only every outage was due to “demand”.
Reduced weekly limits, higher prices than competitors, paying well above spot price for space-x compute, all point to supply issues.
Do you really think it’s a coincidence claude outages always happen when US and EU workdays overlap?
Anthropic is a trillion dollar company and employs way more skilled high-paid engineers than you are btw, you really think it’s a systems issue not capacity
It would be really funny if Anthropic engs run open weight models locally when they need to fix Claude API issues :)
Or just their own models locally, since 8x GPU pods fit under a desk (makes a nice foot heater too)
It's been a while since I messed with serious hardware, what's the cooling noise level? I guess they have to be water cooled or the rack format 8u ones would be about 100db
The competition for the least reliable developer service continues between GitHub.com and Claude.com...
I was using Opus 5 and now I'm forced to switch to Sonnet 5 and bust the prompt cache which will eat a large percentage of my 5h usage...
Setting aside annoyance at the downtime, I'm really curious about the reasons for the failures, because I have to imagine there are some novel failure modes when serving these giant models that I haven't experienced with the kind of work I've done.
Anyone out there working in this space who can elucidate us on interesting failure scenarios unique to the space?
It's (mostly?) compute shortages. Right now it seems there is an issue in the SpaceX datacentres, so they will have less compute than normal.
Why would an issue in the SpaceX data centers impact Claude / Anthropic? Work shifting over from Grok->Claude?
AFAIK Anthropic uses their datacenters
Anthropic is buying compute from SpaceX
https://x.ai/news/anthropic-compute-partnership
Amusingly and frustratingly, this coincides with Codex also being down right now:
https://www.reddit.com/r/codex/comments/1w69ff6/whats_this_e...
This of course could be a coincidence. But I suppose also that if one goes down the others get slammed, so there could be a cascade. If I were another provider I would possibly monitor Claude outages (or vice versa) as a load predictor.
Quite possibly, yeah! Although in the past, whenever Anthropic has been down, I've always been able to rely on using OpenAI.
Possibly people switching work from Claude to Codex?
Open AI models just went down as well. What is going on?
chat.com is down for me but the models seem to still be working. But actually i noticed instability last night at 3am eastern when thinking kept repeatedly failing.
Zed is not able to access the API
I'm monitoring a process started in codex so maybe already in-flight things are still going / are about to hit a wall.
Are we amidst some sort of 'attack on titan'?
Both claude and codex down at the same time! Might actually get some quality code merged in today for a change!
Only Sonnet working over here, but it feels really fast and good. I could get used to this... Really wanted to take Fable 5.1 for a spin though
Not just claude, even grok seems to be down - https://status.x.ai/
If I didn't know any better, I would have said Grok is using Claude behind the hood. But definitely curious now why it’s happening with both these LLM providers around the same time
Looks like a SpaceX issue then.
Ahh, thanks for pointing it out. But it's rather interesting that Claude recognised the outage quicker on their status page than Grok.
Don't they just share the colossus 1 data center?
Potentially because they run from the same colossus DCs?
Gemini (free) throws errors for me too.
EDIT: it was just 3.6 Flash, 3.1 Pro works it seems.
What about AWS Bedrock??
looks like I am going back to the stone age now and write code by hand. Or maybe I am just going to take a walk
If y'all could just log off for the day, that would be great!
grass touching not allowed. shareholder profits must increase
Don't think of it as being unable to work, just like github outages, it's two weeks a year of mandatory time off
Grok, Codex and Claude Code all not working for me now (server issues)
Slack was/is also having issues
Kind of cool how fable 5.1 just worked around these by reassigning subagents to non-failing models with more supervision and now with a hint to Sol and Flash 3.8.
which harness?
Claude code
What's funny is I had just told Claude to kill whatever coding loop it had running. Then got a 529 immediately.
Maybe you triggered a cascade of events that took down a few datacenters.
https://xkcd.com/903/
Officially signing up for the $200 Codex plan now. I do like CC, but these errors are happening every week now, and if they (Anthropic) gave regular resets that's one thing, but somehow I'm already at 62% weekly usage after the random reset this Tuesday using Fable 5.1 medium. Just not worth it any longer compared to Codex IMO. I already set up my skills and everything to be generic and not CC specific, so it shouldn't be too hard to transition over. I'll run both for a month and decide.
Couldn't recommend it more. I've been on their $100/mo. plan and they keep randomly resetting my weekly usage limit. It's been wild. Don't know how long it will last, but I couldn't go through all my usage if I tried.
Their Sol model has been doing great for most of my tasks, even at its "Medium" setting.
Much different picture then back when I was using Claude. I would sneeze and half my usage would be gone and then it go down several times in between.
Hope it goes well for you too!
Grok is also down. Codex is also down. Is this the full meltdown?
Damn, I'll have to think for myself now...
Haha, I was more like: I have to think alone now. But I get the feeling, as someone who has always made a point of using only foss on my own machines (own the means of production so to say), this is a weird time. I don't like it, I hope soon local models get good. But in the mean time... We have this... dependency.
I use a lot of spec driven development today. It does allow me to make workflows that were on my Life's backlog for months or years in 10~20 minutes. It's amazing, but also I lack ownership over the thing made. It's like I paid a contractor.
I can feel that my actual cognitive engineering skills are in decline. Does anyone else see this? To those of you who haven't hand written a PR in months, can you still write your own personal software as easily as you once could?
There are little tools/optimisations that I have been wanting to do for years that I can just throw at AI now. I am almost annoyed sometimes because the little easy things I could do to 'relax' are now gone: if I am working on it is something AI can't do, normally because there are moving parts outside of the repository it can't see (i.e. integration problems, and I hate those).
Take a math class
Cursor Grok models are down too...maybe a datacenter outage?
It goes down so often that I've advocated my company create enterprise Codex accounts as backup for devs.
Using the backup compute capacity stored above their shoulders is somehow unthinkable?
Maybe the models have gone rogue again
ChatGPT also has started failing. Impossible to work.
This pushed me off of a cheaper model into a more expensive one :'(
I just experienced this through OpenRouter and filed a refund request. I'm curious how OpenRouter handles incidents like this across such a large number of models and providers.
Are they going to reset?
sure hope so lol
Cost and reliability are the two reasons why we don't use Claude in our product. Getting close to one nine, that's not something one can build a reliable product upon. We now use OpenAI with Gemini fallback (or vice versa depending on use case). Personally I like Claude and have the 20x Max plan but even there I burned through the whole weekly quota with 3 prompts in less than a day using the new Fable 5.1 which is crazy. Now Opus 5 is down. These two issues are really testing my patience.
We just use bedrock in prod
Every time I see Anthropic ship an issue like this, I'm reminded of Boris Cherny's glorious quote: "At this point it's safe to say that coding is largely solved."
lol
> Elevated errors
Anthropic, please hire a literate human.
During each forced workflow interruption I look for alternatives. No matter if I can't use the product due to a server outage or due to their weird quota limitations.
This can't be good for retention numbers. Old-school VCs would've ripped them apart in the air. Where did all the expertise go?
I suspect the only reason those alternative providers have better up-time and more generous quotas, is because they don't have nearly the same amount of demand. Notice that Deepseek recently had to increase their pricing, once it gained it popularity.
Good day to try omp and GLM 5.3
Yeah, I really like GLM 5.3 Flash with vision. Reminds me of when Claude was lots of fun to work with. It does feel slow, thinking away on the Chinese servers, but it's so incredibly cheap that for a lot of tasks I don't mind it taking a bit longer. I'm finding it mindblowing having entire software features built out for 5 cents, 20 cents.
Nobody uses this shit
HN is not a claude uptime tracker!!
Since an overwhelming number of people who use it are Hackers who come here to seek News it seems relevant