Pith. sign in

REVIEW 4 major objections 5 minor 33 references

Causality analysis of electricity market liberalization on electricity price using novel Machine Learning methods

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read Opening US state electricity markets to competition caused residential prices to fall by about 0.795 cents per kWh (≈7%) in the two years after individual producers entered, according to a causal machine-learning analysis.

desk verdict A useful methodological comparison of causal ML forecasting methods on short panels, but the headline 7% price drop is built on a timing assumption that likely mixes pre- and post-treatment months. read the letter →

arxiv 2507.12331 v1 pith:F6BYP5XV submitted 2025-07-16 econ.GN q-fin.EC

classification econ.GNq-fin.EC
keywords electricitymarketliberalizationcausalmachinelearningsyntheticcontrolcounterfactualforecastingDeepProbCPresidentialpricesUSenergypolicyevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether liberalizing previously monopolized US electricity markets actually lowered residential electricity prices, and it argues that the answer is yes in the short run. Using state-level data from 1990 to 1999 and a global counterfactual forecasting model (DeepProbCP) that predicts what treated states' prices would have been without deregulation, it estimates an average price reduction of 0.795 cents per kWh over the first two years after individual producers entered the market—about 7% below the average price in the pre-intervention year. The paper also compares four causal ML frameworks and contends that the global, non-parametric forecasting approach outperforms TSMixer, Augmented Synthetic Control, and Causal ARIMA, both in synthetic experiments and on the real data. The relevance: if correct, competition at the retail level delivered measurable consumer benefits even when longer-run effects remain unclear, and causal ML offers a credible alternative to difference-in-differences for evaluating energy policy interventions.

What carries the argument

The analysis is carried by a counterfactual forecasting mechanism in the synthetic-control family. A model is trained globally on the pre-intervention data of both treated and control states, with lagged electricity prices and external covariates (state-level income for demand, natural gas price for supply) as inputs, and then forecasts what each treated state's price would have been after 1998 absent liberalization. The ATT is the difference between the observed price and this predicted counterfactual, averaged over treated units and post-intervention time. The validity of this mechanism rests on two assumptions stated in the paper: the pre-intervention relationship between prices and covariates continues after the intervention (Assumption 2, Equations 3–4), and time-varying confounders do not affect treated and control units differently; the paper itself notes that the presence of such confounders invalidates the model. DeepProbCP—the paper's preferred implementation—uses an LSTM layer with a moving-window, seasonal-exogenous strategy and is trained with quantile loss and CRPS, giving it an edge on the short, many-series panels examined here.

What would settle it

Re-run the analysis with a placebo treatment assigned to non-liberalized states at the same calendar date (1998–1999), using the same model and covariates. If the placebo ATT for these untreated states is as large, in absolute value, as the −0.795 ¢/kWh estimated for the liberalized states, then the effect is not specific to liberalization and the causal claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, its central discovery is a causal claim: US electricity market liberalization—timed by the entry of individual producers rather than by the year of legislative approval—caused a short-term reduction in residential electricity prices. Across the eight states where individual producers' market share spiked in 1998–1999, the average treatment effect on the treated estimated by the DeepProbCP model is −0.795 ¢/kWh over the two post-intervention years, a 7% decrease relative to the average price in the year before the intervention. The paper further claims that its preferred model, a global non-parametric counterfactual forecaster, is the most reliable of the four examined for this kind of intervention, and that all models pass the placebo test, so the sign and ordering of the effect estimate are consistent across specifications.

Load-bearing premise

The entire estimate depends on the assumption that, had liberalization never occurred, each treated state's electricity price would have continued to follow the same relationship to its past prices, income, and gas prices observed before 1998, and that nothing else changed these states differently from the controls at the same moment.

Editorial extensions

If this is right

  • Residential prices in liberalized US states fell by about 7% within two years of individual producers' entry, implying a short-term consumer benefit from retail competition.
  • Dating liberalization by producer entry rather than by legislative approval changes the estimated effect and should become standard in evaluating such interventions.
  • The DeepProbCP global forecasting framework appears better suited to short, many-series policy panels than TSMixer, ASCM, or Causal ARIMA.
  • All four models pass the placebo test and estimate a negative treatment effect, so the direction of the price reduction is robust across specifications, though the magnitude varies from roughly −0.44 to −1.06 ¢/kWh.
  • The paper's conclusions are limited to a two-year window and say nothing about whether the price reduction persists after 1999.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If liberalization reduces prices only in the short term, as both this paper and earlier difference-in-differences work suggest, the consumer gains may be a transition phenomenon rather than a durable benefit; a longer-horizon replication with the same causal ML framework could settle this.
  • The range of ATT estimates across models (from −0.44 to −1.06 ¢/kWh) implies that policy conclusions from a single causal ML model should be treated with caution, and ensembles of estimators might give a more stable basis for policy decisions.
  • The intervention-timing correction (using the year of producer entry rather than policy approval) could improve causal ML evaluations of deregulation in other sectors, such as natural gas or telecommunications, where implementation lags legislation.
  • The probabilistic forecasting capability that the authors note in DeepProbCP could be used to construct prediction intervals around the ATT, giving policymakers a measure of uncertainty that the current analysis does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper applies four counterfactual forecasting methods (DeepProbCP, TSMixer, ASCM, and Causal ARIMA) to estimate the short-term causal effect of electricity market liberalization on residential electricity prices in US states. Using state-level price data from 1990 to 1999, with income and gas prices as covariates, the authors define treatment by the first spike in individual producers' market share and set 1998-1999 as the post-intervention window. They report that DeepProbCP performs best on synthetic ground-truth experiments and on control-unit holdout error, and they estimate an ATT of -0.795 cents/kWh, described as a 7% price reduction relative to the pre-intervention year. The paper also argues for causal machine learning as a useful approach for energy-policy intervention analysis.

Significance. If the estimate is valid, the paper provides policy-relevant evidence that retail competition produced a short-term residential price reduction in treated US states. The study has notable strengths: the counterfactual predictions are genuinely out-of-sample forecasts from pre-intervention data, the synthetic experiments have known treatment effects and support the choice of DeepProbCP, placebo tests on control units are reported, and the code is publicly available. The main limitations are the timing definition of treatment, the absence of uncertainty quantification for the headline ATT, and the reliance on control-unit fit for model selection; these affect the precision and interpretation of the central claim but do not undermine the value of the comparison exercise.

major comments (4)
  1. [§4.1, Figure 4, Table 4] The treated group is defined as states whose individual-producer share "jumped in 1998-1999," yet the post-intervention period is set uniformly to 1998-1999. For a state whose first spike occurred in 1999, the entire 1998 calendar year is a pre-treatment year, so the pooled ATT over 1998-1999 mixes post-treatment and no-treatment periods. This biases the reported -0.795 cents/kWh toward zero if the effect is a price decline, and it makes the headline claim of "2 years following liberalization" inaccurate for late-entry states. The paper never reports the entry year per state or a year-by-year ATT; please provide these, or an event-time alignment, to support the causal interpretation.
  2. [§5, Table 4] The headline ATT of -0.795 cents/kWh is a single point estimate with no confidence interval or other uncertainty measure. The Wilcoxon p-values in Table 3 are placebo-test results, not uncertainty intervals for the treated-unit counterfactual. Given that the four models' ATT estimates range from -0.441 to -1.064 cents/kWh, the model choice is consequential; please report uncertainty intervals (for example, from DeepProbCP's probabilistic forecasts or a bootstrap) and a sensitivity table showing how the 7% claim changes across models and post-window definitions.
  3. [§4.1, Figure 3] The text states "Out of the 17 states, in 8 states, the share of individual producers jumped in 1998-1999" and then lists nine states: California, Connecticut, Illinois, Maine, Maryland, New Jersey, New York, Pennsylvania, and Rhode Island. Figure 3's caption also says "Only 8 of them." This numerical inconsistency directly affects the treatment-group definition and the resulting ATT; it must be corrected and the intended treatment group stated unambiguously.
  4. [§3, Eqs. (3)-(4)] The paper explicitly acknowledges that "time-varying confounders, or a failure to closely match control units in the pre-treatment period, results in the invalidity of the model," but the only covariates included are income, gas price, and past prices. No test is provided for state-specific time-varying shocks, such as transition-cost recovery or restructuring-specific regulatory changes, beyond the control-unit forecast errors. Please address this threat more directly, for example with a pre-treatment placebo distribution or a sensitivity analysis adding additional covariates.
minor comments (5)
  1. [§4.1, Eq. (7)] The notation \(\sin(2\pi/1 t)\) is ambiguous; if a period-1 daily cycle is intended, write \(\sin(2\pi t)\) or define the period explicitly.
  2. [§4.1] The phrase "regarded as able to stimulate energy load data" appears to be a typo for "simulate energy load data."
  3. [§3] The abbreviation "SUTV A" should be "SUTVA" (Stable Unit Treatment Value Assumption).
  4. [§5, Table 3] The text describes the Wilcoxon test as testing for a "difference in means," but the Wilcoxon rank-sum test compares distributions or locations, not means.
  5. [§4, Eq. (6)] The seasonal period \(S\) used in the MASE denominator is not defined in the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: counterfactual forecasts are out-of-sample and model selection avoids treated outcomes.

full rationale

The core ATT estimate is not circular. DeepProbCP (and the other models) are trained only on pre-intervention data: the paper states that 'the pre-intervention data of all units is used as training data, and the post-intervention period is used as the test data to be forecasted by the model.' Treated post-intervention outcomes are never used in training or in model selection. Model selection is justified by synthetic experiments with known ground truth and by control-unit forecast errors under the null intervention assumption, not by fitting a parameter to the real treated outcome. The ATT is then computed as observed minus forecast, which is a genuine out-of-sample prediction error rather than a fitted quantity renamed as a prediction. There are no load-bearing self-citations: DeepProbCP is taken from Grecov et al. (a different group) and used via its published code, and no uniqueness theorem or author-overlapping citation is invoked to force the modeling choice. The paper does contain a potential timing concern: it sets 1998-1999 as the common post-period for states whose producer-share spike may have occurred in either 1998 or 1999, and the sentence 'Upon inspecting the price data, we can observe a visible change as the share of individual producers starts to go up' could suggest outcome-informed timing inspection. However, these are identification and design issues, not circular reductions: no equation, fitted parameter, or self-citation makes the estimated effect equal to an input by construction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The paper relies on standard potential-outcome assumptions plus two domain-specific choices: the intervention date is the first year of producer-share increase, and the forecasting model learned pre-1998 remains valid post-1998. The synthetic experiments add hand-chosen parameters that guide model selection but do not enter the ATT.

free parameters (2)
  • synthetic trend_rate = 1.00005
    Multiplicative exponential trend added to synthetic series; chosen by hand to test models under trend, not used on real data.
  • synthetic treatment effect quantile constants = 0.3 sigma, 0.6 sigma, 0.9 sigma, 1.2 sigma, 1.5 sigma
    Chosen to create heterogeneous treatment effects in synthetic data; these do not enter the real-data ATT but drive model selection benchmarks.
assumptions (5)
  • domain assumption Unconfoundedness: conditional on covariates (income, gas price) and pre-treatment outcomes, treatment assignment is independent of potential outcomes.
    Stated in Section 3 as required for identification; only two covariates are included, so this is a strong assumption.
  • domain assumption Assumption 2: the relationship between treated unit outcomes and predictors estimated before the intervention remains the same after the intervention (Eqs. 3 and 4).
    Needed to forecast counterfactuals; the paper acknowledges time-varying confounders would invalidate the model.
  • ad hoc to paper No anticipation: policy approval did not change prices until individual producers actually entered, based on inspection of price data.
    Section 4.1 argues this from monopoly behavior and observed prices; it is not formally tested.
  • domain assumption Consistency and SUTVA: control states are unaffected by liberalization and treated states' observed outcomes equal their treatment potential outcomes.
    Standard POF consistency in Section 3; spillovers from liberalized to neighboring states would violate it.
  • ad hoc to paper Synthetic data generation (sum of sine waves plus Gaussian process) resembles real electricity price dynamics enough for model selection.
    The authors rely on this to choose DeepProbCP for the real-data estimate; no formal similarity test is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Causality analysis of electricity market liberalization on electricity price using novel Machine Learning methods." pith.science (2026). https://pith.science/paper/F6BYP5XV

@misc{pith2026250712331,
  author       = {Pith},
  title        = {Pith review of: Causality analysis of electricity market liberalization on electricity price using novel Machine Learning methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F6BYP5XV}},
  note         = {Machine review of arXiv:2507.12331}
}
read the original abstract

Relationships between the energy and the finance markets are increasingly important. Understanding these relationships is vital for policymakers and other stakeholders as the world faces challenges such as satisfying humanity's increasing need for energy and the effects of climate change. In this paper, we investigate the causal effect of electricity market liberalization on the electricity price in the US. By performing this analysis, we aim to provide new insights into the ongoing debate about the benefits of electricity market liberalization. We introduce Causal Machine Learning as a new approach for interventions in the energy-finance field. The development of machine learning in recent years opened the door for a new branch of machine learning models for causality impact, with the ability to extract complex patterns and relationships from the data. We discuss the advantages of causal ML methods and compare the performance of ML-based models to shed light on the applicability of causal ML frameworks to energy policy intervention cases. We find that the DeepProbCP framework outperforms the other frameworks examined. In addition, we find that liberalization of, and individual players' entry to, the electricity market resulted in a 7% decrease in price in the short term.

Figures

Figures reproduced from arXiv: 2507.12331 by the authors.

Figure 1
Figure 1. Synthetic time series used in our experiments [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗
Figure 2
Figure 2. Simulated dataset with intervention. The grey lines in the background are the time series. [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. 17 states that passed deregulation policies between 1996 and 2000. Only 8 of them saw [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The average of individual producers’ share across treated states. A visible spike occurs [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Electricity price for monopolized and liberalized states. A visible drop in liberalized states [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Counterfactual predictions of all models. [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: DeepProbCP forecasts of treatment and control units. The treatment units’ forecasts [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 32 canonical work pages

  1. [1]

    Abadie, A. (2021). Using synthetic controls: Feasibility, data requirements, and methodological aspects. Journal of Economic Literature, 59(2):391–425

  2. [2]

    Administration, U. E. I. (2024). Eia-861m monthly electric power industry report

  3. [3]

    Allcott, H. (2011). Rethinking real-time electricity pricing. Resource and Energy Economics, 33(4):820–842. Special section: Sustainable Resource Use and Economic Dynamics

  4. [4]

    Amenta, C., Aronica, M., and Stagnaro, C. (2022). Is more competition better? retail electricity prices and switching rates in the european union. Utilities Policy, 78:101405

  5. [5]

    Annweiler, C., Fantino, B., and Parot-Schinkel, E. (2011). Alzheimer’s disease - input of vitamin d with memantine assay (ad-idea trial): study protocol for a randomized controlled trial. Trials 12, 230

  6. [6]

    and Imbens, G

    Athey, S. and Imbens, G. W. (2006). Identification and inference in nonlinear difference-in-differences models. Econometrica, 74(2):431–497

  7. [7]

    T., and Trujillo-Baute, E

    Batalla-Bejerano, J., Costa-Campi, M. T., and Trujillo-Baute, E. (2016). Collateral effects of liberalisation: Metering, losses, load profiles and cost settlement in spain’s electricity system. Energy Policy, 94:421–431

  8. [8]

    and Furceri, D

    Blagrave, P. and Furceri, D. (2021). The macroeconomic effects of electricity-sector privatization. Energy Economics, 100:105245

Show all 33 references
  1. [9]

    O., Yoder, N

    Chen, S.-A., Li, C.-L., Arik, S. O., Yoder, N. C., and Pfister, T. (2023). TSMixer: An all-MLP architecture for time series forecast-ing. Transactions on Machine Learning Research. Originally available as arXiv:2303.06053

  2. [10]

    Cheng, X., Liu, P., and Zhu, L. (2024). The impact of electricity market reform on renewable energy production. Energy Policy, 194:114306

  3. [11]

    Eli Ben-Michael, A. F. and Rothstein, J. (2021). The augmented synthetic control method. Journal of the American Statistical Association, 116(536):1789–1803

  4. [12]

    Erdogdu, E. (2013). Implications of liberalization policies on government support to r and d: Lessons from electricity markets. Renewable and Sustainable Energy Reviews, 17:110–118. 24

  5. [13]

    Grecov, P., Bandara, K., Bergmeir, C., Ackermann, K., Campbell, S., Scott, D., and Lubman, D. (2021). Causal inference using global forecasting models for counterfactual prediction. In Advances in Knowledge Discovery and Data Mining (PAKDD 2021), LNCS, vol.12713, pages 282–294...

  6. [14]

    Grecov, P., Prasanna, A., Ackermann, K., Campbell, S., Scott, D., Lubman, D., and Bergmeir, C. (2024). Probabilistic causal effect estimation with global neural network forecasting models. IEEE Transactions on Neural Networks and Learning Systems, 35(4):4999–5013

  7. [15]

    Glachant, P.L

    J.-M. Glachant, P.L. Joskow, M. P. (2022). The evolution of competitive retail electricity markets. Handbook on Electricity Markets, Edward Elgas Publishing pp. 111-155

  8. [16]

    Joskow, P. L. (2000). Deregulating and regulatory reform in the u.s. electric power sector. MIT Center for Energy and Environmental Policy Research

  9. [17]

    Joskow, P. L. (2008). Lessons learned from electricity market liberalization. The Energy Journal, 29(2_suppl):9–42

  10. [18]

    and Branson, Z

    Keller, B. and Branson, Z. (2024). Defining, identifying, and estimating causal effects with the potential outcomes framework: A review for education research. Asia Pacific Education Review, 25(3):575–594

  11. [19]

    J., Nikolova, S., and Sutton, M

    Kreif, N., Grieve, R., Hangartner, D., Turner, A. J., Nikolova, S., and Sutton, M. (2016). Examination of the synthetic control method for evaluating health policies with multiple treated units. Health Economics, 25(12):1514–1528

  12. [20]

    X., and Dustdar, S

    Liu, S., Yu, H., Liao, C., Li, J., Lin, W., Liu, A. X., and Dustdar, S. (2022). Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International Conference on Learning Representations

  13. [21]

    Menchetti, F., Cipollini, F., and Mealli, F. (2021). Estimating the causal effect of an intervention in a time series setting: the c-arima approach. arXiv preprint arXiv:2103.06740v3

  14. [22]

    Nagayama, H. (2009). Electric power sector reform liberalization models and electric power prices in developing countries: An empirical analysis using international panel data. Energy Economics, 31(3):463–472

  15. [23]

    D., López, A., and Salazar-Elena, J

    Necoechea-Porras, P. D., López, A., and Salazar-Elena, J. C. (2021). Deregulation in the energy sector and its economic effects on the power sector: A literature review. Sustainability, 13(6):Article 3429. 25

  16. [24]

    Upton, J

    Rose, K., Tarufelli, B., and Gregory B. Upton, J. (2024). Retail electricity market restructuring and retail rates. The Energy Journal, 45(1):1–49

  17. [25]

    Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology,66, 688–701

  18. [26]

    Rubin, D. B. (1977). Assignment to treatment group on the basis of a covariate. Journal of Educational and Behavioral Statistics, 2(1):1–26

  19. [27]

    Rubin, D. B. (1986). Comment: Which ifs have causal answers. Journal of the American Statistical Association, 81(396):961–962

  20. [28]

    Schuelke-Leech, B.-A., Barry, B., Muratori, M., and Yurkovich, B. (2015). Big data issues and opportunities for electric utilities. Renewable and Sustainable Energy Reviews, 52:937–947

  21. [29]

    Splawa-Neyman, J. (1990). On the application of probability theory to agricultural experiments. Essay on principles. Statistical Science, 5(4):465–480. Translated and edited by D. M. Dabrowska and T. P. Speed

  22. [30]

    Su, X. (2015). Have customers benefited from electricity retail competition? Journal of Regulatory Economics, 47(2):146–182

  23. [31]

    Trevino, L. (2008). Liberalization of the electricity market in europe: An overview of the electricity technology and the market place. Working paper, École Polytechnique Fédérale de Lausanne (EPFL). Accessed: 2025-07-12

  24. [32]

    Zhang, D. (2018). Energy finance: Background, concept, and recent developments. Emerging Markets Finance and Trade, 54(8):1687–1692

  25. [33]

    and Ji, Q

    Zhang, D. and Ji, Q. (2019). Energy finance: Frontiers and future development. Energy Economics, 83:290–292. 26

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.