Sorry, but the ROI on enterprise AI is abysmal

(theregister.com)

40 points | by lolinder 2 days ago ago

27 comments

  • jnwatson 2 days ago

    Most enterprises have abysmal documentation on internal processes and standards.

    There's a good reason that an early use case for LLMs is source code generation, because there's a large quantity of mostly good examples.

    If you trawl the average company's internal docs (if you could even get a hold of them), you'd see a hodge podge of uneven quality of coverage.

    It is hard to get any sort of automation to work when the input is bad and the desired output is underspecified.

    • mrjin 2 days ago

      Even code generation is barely usable if you really start to use it. Currently VisualStudio's intellisense is generating completely junk code half of the time.

  • physicsguy 2 days ago

    My large corporate has just declined continuing its Copilot 365, citing that there wasn’t much usage and people didn’t find it very useful.

    • carlmr 2 days ago

      Have you tried other AI tools? Is Copilot the only one you're discontinuing?

      • physicsguy a day ago

        We still have access to GH Copilot through an enterprise deal, it's a bit hobbled though vs the full thing since there's extra filters that MS offer.

  • egorfine 2 days ago

    I believe it's either a case of:

    1. Selling snake oil to corporate. How many days since you have seen another GPT wrapper?

    2. Unmotivated employees not motivated to learn new tools. Like, why would they care, AI or not.

    • exe34 2 days ago

      The second one is easy - if a genuine tool exists that can boost your productivity[0], you can choose to either embrace it or be replaced by somebody who did.

      [0] I'm neither claiming nor denying that current "AI" is that tool.

      • yifanl a day ago

        It's not obvious to me that employees are motivated to be more productive for the sake of productivity.

        • exe34 a day ago

          is it not obvious that there must be some who will be? my experience has been that new grads are willing to try out new things - it may be that this particular AI won't get them anywhere and maybe they'll become disillusioned, but the next crop of graduates will embrace the next big thing, and if that turns out to make a huge difference, they will reap the rewards.

          • yifanl a day ago

            It's not obvious that in the general case, employees can be motivated by getting work done if the reward is more work to do.

            • exe34 a day ago

              no, the reward is a CV that will bag them a major payrise at the next gig. not everybody wants that, so it'll never be the general case. but they might still take your job - along with several others'.

              • brailsafe a day ago

                It's not clear to me that anything novel done at one enterprisey job has a predictible impact on how much more you could ask for next time, beyond learning the skills to do the work and proving you can take on responsibility or make decisions. As an IC, there are so many other bottlenecks, that at best it'll just be another tool; in many cases, I don't think the value derived from a faster coding tool would surpass the value of reducing the number of meetings, which are tremendously burdensome and always creep into what would otherwise be the time you'd spend thinking about a sufficiently important problem.

      • austinjp 2 days ago

        Sure, but that's a race with very few winners. If there's no cap on notional performance maxima, there's always scope to lose your job.

        • exe34 a day ago

          I like to automate parts of my own job, so when it's done, I'm the guy who automates jobs, instead of the guy who isn't needed anymore. some day of course I'll be obsolete. no avoiding that.

      • carlmr 2 days ago

        I regularly use ChatGPT at work, it's helpful in refining wording for emails and documentation. It's helpful for getting a starting point for Python scripts. I haven't seen it being a game changer though.

        Copilot has been a bit mixed. It kind of gives me too many obviously bad code snippets that I find it super useful. But when I code in a more unfamiliar language it's helpful.

        • exe34 a day ago

          that's been my experience with chat gpt, it allowed me to write some kotlin and sql which I'm not familiar with.

          I think it's important to remember how far and how fast these things showed up - in just a few years they went from toys to a solid beginning - I'm not sure they can progress the same amount again in the same time, but every s curve has to slow down and every s curve is followed by another eventually - so we don't know if/when the next step change will come, but if it does, it might well put us all out of business overnight.

          • carlmr a day ago

            >it might well put us all out of business overnight.

            It might well do that. But worrying about the future like that has no purpose. If the machines become intelligent enough to replace knowledge work they'll be able to improve themselves and mechanical robots too, so work won't exist anymore.

            If that happens we'll either achieve post scarcity or some kind of post human Armageddon if the system is runs awry.

            • exe34 a day ago

              that's exactly the kind of future worth worrying about - having a backup plan is a good idea. such AI won't be able to expand into the physical world as fast as they can take over knowledge work - so there would still be things worth doing that could be worth something.

  • whiplash451 2 days ago

    Appen advertising its own business (data labeling)

    • moralestapia 2 days ago

      Thanks, this context is very relevant.

  • ipnon 2 days ago

    The bull case described by EY sounds more like AI is enhancing worker productivity rather than replacing workers.

  • bloedsinnig 2 days ago

    It doesn't has to be good yet.

    We just started adding AI to every aspect of products. Things like this takes time.

    And in my company it adds already value. In another company we can now do tasks we were unable to do before. It increases data quality by a relevant amount. Hard to determine a ROI for things like this.

    • rsynnott a day ago

      Given the amount of money being pumped into it, it has a pretty short expected lifespan if it doesn't get good fast. Promises of jam tomorrow will only go so far.

      • bloedsinnig 14 hours ago

        What we do right now is to introduce it to the whole ecosystem. This costs money.

        AI / ml will not go away anymore.

        And the money we pumpe into it? Doesn't matter. Our tech companies and VCs have enough money to pump into

      • basch a day ago

        chicken and egg.

        You need people to use it to have value. It needs to have value for people to use it.

        The use/value proposition will tilt towards it as a generation turns over from people who grew up with search to people who grew up with chat.

        Suddenly itll be a commoditized free feature, that if you dont offer it, people will go work somewhere else that embraces. Give it time.

        • rsynnott a day ago

          > The use/value proposition will tilt towards it as a generation turns over from people who grew up with search to people who grew up with chat

          There is no possible way that the current level of money-burning can be sustained for anything _like_ that long. I'd say if major players don't start showing a profit on this stuff within a year or so, the bubble will become unsustainable.