Unfortunately, this is the type of statements we can't verify. I'm not sure why these types of news are still coming out when we all have AI at work.
Whenever someone does such a huge drastic change like this, it's by ignoring a large chunk of code that most people were afraid to touch for good reasons. Now, that code is gone, AI is celebrated, things will break, people will work very hard in the background to fix it, with no fanfare.
> I'm not sure why these types of news are still coming out when we all have AI at work.
Because OpenAI is burning $15 billion/year, and outrageous stories like those get parroted in the media. It's free marketing for a company desperate to get middle management to believe that a $500/mo subscription is absolutely crucial for every single employee.
Exactly. You can simply close all open tickets with <WONTFIX> and claim that you've cleared 5 years of engineering work in 5 minutes. It doesn't mean anything.
It’s just sales claims.
Do you see that “contact sales” button?
You’re not the target of that.
Deranged AI-psychotic C-suites with a two-digit IQ and too much confidence are the target.
Phrased this way, my aspiring product leader hat activates: that 5 years of work cannot be of much value to a company worth billions because a 5 year estimate is the lowest possible qualification you can make. Its a “will never happen” shelving.
all work is stack ranked against other opportunities. no matter how many teams help to parallelize, leadership basically comes down to ranking the stack correctly. Valuable work will be at the top by any means necessary. “mvp”, probes, task forces, code yellows, and so on.
The five-sentence prompt beating an elaborate setup fits what I keep seeing with these tools. The trick is carving off well-scoped migrations, not handing over open-ended design work.
> For comparison: the previous plan was expected to take at least five years and estimated to cost roughly $6M.
If that's an estimate from an LLM, those have always been way off for me. I'm constantly amazed whenever something that an agent estimates would take weeks ends up being completed in an hour.
The LLM is reasoning about estimates from its training data... which is to say, from human engineering timescales.
I suspect the labs could improve the models such that they are estimating these sorts of things but they don't prioritize doing so (or perhaps RLHF selects it away) because, as you say, it feels amazing to do a week's worth of work in an hour.
Is this the react test framework enzyme? To me, $6 million and 5 years seems like a very high estimate for this even without any automation (AI or otherwise) for even relatively large applications. How could they possibly have a test suite this large?
> An engineer checked progress twice a day and reviewed every proposed change.
It doesn’t sound like that much changed. If one engineer can really review all the changes in under two weeks, the original estimate of 5 years of engineering work has to be waaaaaaay off.
People may think it’s an overstatement. But I think it’s not.
This kind of projects are where AI is most helpful. Long tedious and highly testable projects like ports or legacy system replacements where humans have to grind through millions of lines of code without really thinking are the perfect target for AI.
I once did a C to C# port of several math libraries, and while I was able to automate most of it, it still required a ton of manual work. I bet if I had Claude, I could do what I did in 3 months in a day or two.
I would love to know the responsible people at Asana who approved this joint PR. Like, do you not get the blatant second degree negative implications of what’s stated here, albeit manipulated and hyperbolic, for a project management software?
When everything is “estimates”, anything sounds amazing.
The impressive thing here is that Asana has such a poorly run engineering org that replacing their testing framework was estimated to take 5 years and $6 million.
The fun part is how the article just says they removed "Enzyme" and there is nothing else mentioned that can make you gauge the technical complexity of what was done here
Comparisons such as 5 years and $6M are meaningless. Instead, they should compare with other AI, such as how much it would have cost using self-hosted open-weight models, or using a API that offers open-weight models etc.
The latter sounds about as attractive to me as the former, actually more because my customers are mostly hoping for their bugs to be fixed , not for the latest novel feature.
Unfortunately, this is the type of statements we can't verify. I'm not sure why these types of news are still coming out when we all have AI at work.
Whenever someone does such a huge drastic change like this, it's by ignoring a large chunk of code that most people were afraid to touch for good reasons. Now, that code is gone, AI is celebrated, things will break, people will work very hard in the background to fix it, with no fanfare.
> I'm not sure why these types of news are still coming out when we all have AI at work.
Because OpenAI is burning $15 billion/year, and outrageous stories like those get parroted in the media. It's free marketing for a company desperate to get middle management to believe that a $500/mo subscription is absolutely crucial for every single employee.
> people will work very hard in the background to fix it, with no fanfare.
This. It looks like AI companies have managed to use this for their advantage. Can't really blame them.
> where afraid to touch for good reasons
This appears entirely unreasonable.
Normally the reason is not good. The reason is that unit testing is missing or that downstream effects are not entirely mapped out.
Exactly activities that traditional software developers are loathing because they are boring and mentally straining.
Exactly. You can simply close all open tickets with <WONTFIX> and claim that you've cleared 5 years of engineering work in 5 minutes. It doesn't mean anything.
It’s just sales claims. Do you see that “contact sales” button? You’re not the target of that. Deranged AI-psychotic C-suites with a two-digit IQ and too much confidence are the target.
"I took a speed-reading course and read War and Peace in twenty minutes. It involves Russia."
― Woody Allen
I think there might be a mixup here.
You see: normally one reads for recreation or enlightenment.
If that same one would approach their job as a recreational activity, or solely for enlightenment, you might not have that job for very long.
Phrased this way, my aspiring product leader hat activates: that 5 years of work cannot be of much value to a company worth billions because a 5 year estimate is the lowest possible qualification you can make. Its a “will never happen” shelving.
all work is stack ranked against other opportunities. no matter how many teams help to parallelize, leadership basically comes down to ranking the stack correctly. Valuable work will be at the top by any means necessary. “mvp”, probes, task forces, code yellows, and so on.
The five-sentence prompt beating an elaborate setup fits what I keep seeing with these tools. The trick is carving off well-scoped migrations, not handing over open-ended design work.
From the article:
> For comparison: the previous plan was expected to take at least five years and estimated to cost roughly $6M.
If that's an estimate from an LLM, those have always been way off for me. I'm constantly amazed whenever something that an agent estimates would take weeks ends up being completed in an hour.
The LLM is reasoning about estimates from its training data... which is to say, from human engineering timescales.
I suspect the labs could improve the models such that they are estimating these sorts of things but they don't prioritize doing so (or perhaps RLHF selects it away) because, as you say, it feels amazing to do a week's worth of work in an hour.
Is this the react test framework enzyme? To me, $6 million and 5 years seems like a very high estimate for this even without any automation (AI or otherwise) for even relatively large applications. How could they possibly have a test suite this large?
The more tangible example for me around LLM enabling what's previously long if not impossible tasks: pgrust. https://github.com/malisper/pgrust
They probably meant 1 employee part time for 5 years on a project that was never a priority.
So clearly it doesn't need the workforce required to delivery 5 years man work, as per project estimations, thus brace for impact.
"Their old testing tool, Enzyme, had fallen out of active maintenance and was becoming a blocker to modernizing Asana’s frontend stack"
Non prod code, good use case.
I believe this about as much as the claim that the Bun rewrite cost $165k
> An engineer checked progress twice a day and reviewed every proposed change.
It doesn’t sound like that much changed. If one engineer can really review all the changes in under two weeks, the original estimate of 5 years of engineering work has to be waaaaaaay off.
A nitpick. While this definitely does not apply to the problem described, some changes are indeed mostly alignment, not implementation.
Pretty big self-own
You see, I call bs on that. Not on the part that agents are an accelerator, they might be, but at the part that it was a 5 year project.
We engineers, had the power to dictate how our days will look, and we were able to give ballooned timelines to give us room to braeth.
This has changed. The power is no longer in our hands, for good or bad.
I suspect it as a two month project without agents. Max.
People may think it’s an overstatement. But I think it’s not.
This kind of projects are where AI is most helpful. Long tedious and highly testable projects like ports or legacy system replacements where humans have to grind through millions of lines of code without really thinking are the perfect target for AI.
I once did a C to C# port of several math libraries, and while I was able to automate most of it, it still required a ton of manual work. I bet if I had Claude, I could do what I did in 3 months in a day or two.
I would love to know the responsible people at Asana who approved this joint PR. Like, do you not get the blatant second degree negative implications of what’s stated here, albeit manipulated and hyperbolic, for a project management software?
When everything is “estimates”, anything sounds amazing.
The impressive thing here is that Asana has such a poorly run engineering org that replacing their testing framework was estimated to take 5 years and $6 million.
Also I think the marketing post compares removing a test framework (how is that not trivial?) with replacing it with another.
The fun part is how the article just says they removed "Enzyme" and there is nothing else mentioned that can make you gauge the technical complexity of what was done here
When "5 years of work" take two weeks, you know two things. The estimates were wrong, and nobody will need a project management tool soon.
Idunno, maybe the point of the project management system was to be able to make grandiose statements about your progress
Comparisons such as 5 years and $6M are meaningless. Instead, they should compare with other AI, such as how much it would have cost using self-hosted open-weight models, or using a API that offers open-weight models etc.
It is a good PR for Asana. I checked this with Gemini & here is what Gemini said - https://share.gemini.google/B8nocObwgiih
Summary: The Takeaway Did an AI generate 5 years' worth of highly creative, novel software features from scratch in two weeks? No.
Did an AI complete 5 years' worth of tedious, widespread code-migration technical debt in two weeks? Yes.
The latter sounds about as attractive to me as the former, actually more because my customers are mostly hoping for their bugs to be fixed , not for the latest novel feature.