$12B of US ratepayers' money wasted on a modeling mistake in PJM

(newsletter.semianalysis.com)

35 points | by _delirium 3 hours ago ago

14 comments

  • Animats 17 minutes ago

    The article's main complaint is that generating capacity has been overbuilt by maybe 10%. That's not a bad thing. There are worse things than extra generating capacity.

  • Certhas an hour ago

    I don't doubt that ISO models are not optimal, but the annual cost of power outages to US consumers is above 100 billion dollars last time I looked already. Electric system reliability is a place where you have to be somewhat conservative as costs rise really non-linear with outage intensity. [1]

    And ISOs/RTOs/TSOs operate in complex political environments and have to ensure reliability in adverserial markets. They are often constraint in what they can/can't do.

    What I will say is that their modelling should absolutely be open and transparent, and typically it's not. That would make necessary discussions around modelling assumptions, potential improvements and political constraints much much easier and more fruitful.

    [1] That said, my intuition is that current outages are probably not due to supply insufficiency, but due to transmission system failures.

  • k310 2 hours ago

    PJM Interconnection LLC (PJM) is a regional transmission organization (RTO) in the United States. It is part of the Eastern Interconnection grid operating an electric transmission system serving all or parts of Delaware, Illinois, Indiana, Kentucky, Maryland, Michigan, New Jersey, North Carolina, Ohio, Pennsylvania, Tennessee, Virginia, West Virginia, and the District of Columbia. PJM is the largest power grid operator in the United States, serving 67 million customers from Chicago to New Jersey. (Wikipedia)

    That was not clear in the article.

  • stymaar an hour ago

    Unfortunately, the article misses the forest for the tree: it points the shortcomings of one particular electricity market model, but never acknowledge that the whole endeavor is doomed to fail: the way electricity works physically (coupled with the way societies have been built around it being readily available) makes it fundamentally impossible to manage through market forces. And as a result, every attempt results in an apparently nonsensical failure (see also how the European electricity market's "copper plate" model leads to the insanely high re-dispach costs).

    • logicchains 19 minutes ago

      >the way electricity works physically (coupled with the way societies have been built around it being readily available) makes it fundamentally impossible to manage through market force

      This is anti-scientific woo. Both theoretically and empirically market-based electricity systems are much more efficient, that's why even China is adopting one: https://www.enerdata.net/publications/daily-energy-news/chin...

  • readthenotes1 2 hours ago

    This reminds me of the 2013 finding that showed an earlier 2010 study praising austerity as a recovery mechanism was flawed because of an Excel calculation error and lack of diligence.

    https://inthesetimes.com/article/the-excel-error-heard-round...

    It also reminds me of the flaws in the London epidemiological model about how to respond to covid.

    It just may be that these things should be reviewed a little closer by people who are obsessive about correctness

    • logicchains 17 minutes ago

      >It just may be that these things should be reviewed a little closer by people who are obsessive about correctness

      No amount of review is enough. As anyone who's worked in quant science will attest, a model that hasn't repeatedly made correct out-of-sample predictions is most likely worthless, as it's far far easier to overfit a model to data than to predict the future.

    • Nursie 42 minutes ago

      > flaws in the London epidemiological model about how to respond to covid.

      That one was fun. I got into an argument with a scientist who was (IIRC) working with or for the specific professor that made that mess, and he blamed the entire software engineering/computer science industry for making C++ too hard to use, and said it should never have been released if mistakes like this could be made. And then accused me of gatekeeping when I said perhaps they should get an expert to look over this stuff before submitting results to governments to form a policy basis.

      The attitude seemed to be "I am a scientist and am clearly very smart, and therefore anything you do must be trivial in comparison. If I can't pick up in seconds what takes you years to learn and more years to perfect, you've clearly done it all wrong and it's your fault."

  • GianFabien 2 hours ago

    Every "wasted $" ended up in some parasitical grifter's pocket.

    Whilst these sorts of analyses are informative, they lack answers to the who profits? question.

    • jakewins an hour ago

      The money has gone to pay power plants that can supply winter capacity, exactly as the market is designed to do.

      The intention of the payments is to increase revenue for that kind of power generation capability to encourage more such plants be constructed.

      The argument in the article is dubious to me. Of course the higher price isn’t leading to more generation today, that’s not the point, the point is to reward developers that build and built capacity CA needs in winter. The disagreement then becomes which model is correct about how much capacity is actually needed.. but the fact that a tiny move in demand moves the price so substantially seems to me to undermine the entire premise of the blog post, clearly supply is severely constrained?

      • dgacmu an hour ago

        But that's why they're arguing that repricing the entire generation fleet is bad compared to having a different auction for new capacity. You don't need the same incentives to keep an existing, profitable plant online as you do to invest in a new plant.

        • jakewins an hour ago

          I can assure you the spreadsheets at firms building plants factor in forecasts of revenue for the life of the plant into investment decisions. If they don’t pay past the first few years, that directly translates to lower forecast lifetime plant value for new plants.

          I’m not a quant, and I’ve worked energy trading desks long enough to know there is a lot I don’t understand.. but I don’t see how separating auctions by plant age does anything other than move numbers around while keeping the total bill the same. Plants still need the same lifetime revenue to make investment decisions pencil out; whether you front-load payments or spread them evenly, the total in current value needs to be the same.

          • zmgsabst an hour ago

            Even if the total cost is the same, if our payments better align with the behavior we want to incentivize, we may gain greater utility from the spending.

            So it can matter how we distribute that revenue as to whether or not the business responds in the desired way, eg, actually investing in new capacity by linking payments directly to new capacity.

      • bjourne 29 minutes ago

        What's to say that encouragement works? Does the 12B subsidy stipulate that more plants must be built? If not, keeping supply limited and price per Watt high may be more profitable.