{"id":"34b22516-f4f7-4917-913a-a1902cd9eeb8","arxiv_id":"2505.13422","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Inserting nonlinear machine learning predictions into the first stage of 2SLS can produce larger second-stage bias than endogenous OLS, while linear selection methods like post-Lasso and PCA perform safely.","lead":"This paper tests what happens when machine learning models are used to predict the treatment variable in the first stage of two-stage least squares. It finds that flexible models like random forests and neural networks can make the estimated causal effect more biased than simply ignoring the endogeneity problem.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"All simulated first stages are linear, so the headline ranking of nonlinear ML methods is untested in the one regime where such methods could actually help.","rationale":"The reader's weakest assumption is exactly the load-bearing gap: both DGPs impose a linear first stage, so flexible nonlinear methods are never given the opportunity to outperform linear 2SLS on the margin where they are supposed to help. The central claim of the paper is not merely a mathematical identity but a practical recommendation about when ML helps or hurts in the first stage of 2SLS. Because the recommendation is derived exclusively from linear first-stage simulations, it does not support the unqualified conclusion that nonlinear ML methods generate substantial bias or that they can be worse than ignoring endogeneity. The paper does have genuine strengths: the decomposition in Eq. (11) is a useful exact accounting identity, the simulation results are transparent, and the inclusion of traditional estimators like LIML and JIVE provides useful benchmarks. Nothing in the paper is internally inconsistent; the concern is overgeneralization from a design choice. A conditional verdict is appropriate: the paper should be accepted only after either adding nonlinear first-stage DGPs or substantially qualifying the abstract and Section 4.1 to state that the results are demonstrated for linear first-stage conditional expectations. Since the reader already reached CONDITIONAL for the same reason, no change to the verdict is needed. My specific recommendation is to require the nonlinear-DGP simulation as a condition of acceptance, and to temper the abstract's blanket claim if the authors prefer not to add simulations.","tokens_in":26534,"tokens_out":5045,"duration_ms":54624,"concrete_test":"Run the same simulation suite with one additional DGP in which the first stage is nonlinear while the instruments remain exogenous, for example x1 = z1 + z2^2 + z3*z4 + epsilon_c, with the structural error u correlated with epsilon_c and independent of z. Keep the sample size, number of iterations, and estimator implementations identical to the paper. If random-forest- or neural-network-based 2SLS produces second-stage bias no larger than OLS-2SLS, or smaller than the bias of naive OLS, the paper's headline ranking and practical guidance would need to be qualified to linear first stages. If the nonlinear methods remain more biased even when they can approximate the true first-stage function, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central practical claim is that nonlinear ML first stages (random forests, boosted trees, neural nets) generate substantial second-stage bias, potentially exceeding the bias of endogenous OLS. Every DGP in the paper has a linear first-stage conditional expectation: the low-complexity case sets g_x(z) = sum_i z_i (Section 3.1), and the high-complexity case sets x1 = pi z + epsilon_v (Section 3.2). Consequently, the nonlinear methods are evaluated only in a setting where the true first stage is linear, so their flexibility cannot improve the first-stage fit and can only add variance or recover endogenous variation. The paper never simulates a DGP with a genuinely nonlinear g_x(z), such as quadratic, interaction, or threshold terms. Figure 3 and Appendix A.2 vary the structural error by adding k-way interactions of the instruments to u, but the first stage remains linear in every case; these simulations test exclusion-restriction violations, not the ability of nonlinear learners to approximate a nonlinear first-stage regression. As a result, the abstract's unqualified statement that nonlinear methods generate large bias is a statement about a particular linear-world simulation design, not a robust practical regularity. The bias decomposition in Eq. (11) is algebraically correct for any fitted values, and the simulations appear internally consistent; the problem is external validity of the practical guidance. A researcher facing a nonlinear first stage could reasonably expect tree-based or neural-net first stages to capture useful exogenous variation that linear 2SLS misses, potentially reversing the ranking relative to naive OLS. The paper's own discussion acknowledges that practitioners rarely know the extent of nonlinearity, so omitting nonlinear first-stage DGPs is a load-bearing gap rather than a minor limitation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies what happens when a researcher replaces the linear first stage of 2SLS with an off-the-shelf machine learning predictor. It derives an exact finite-sample decomposition of the bias of the resulting second-stage estimator into three components: a term proportional to Cov(ẋ,e), where ẋ are first-stage fitted values and e are first-stage residuals; a term Cov(ẋ,u), where u is the structural disturbance; and a scaling factor 1/Var(ẋ). It then reports Monte Carlo results for two families of DGPs, one with seven strong instruments and a linear first stage, and one with one hundred instruments of varying strength and a linear first stage in three subcases. The headline finding is that linear machine-curated procedures (post-Lasso, PCA) perform as well as or better than OLS-based 2SLS, while nonlinear procedures (random forests, boosted trees, neural networks) can be substantially biased, in some cases more biased than an OLS regression that ignores endogeneity entirely. The paper also discusses the forbidden-regression interpretation, the need for strengthened exclusion restrictions when nonlinear functions of instruments are used, and the relationship to split-sample and synthetic-instrument estimators such as MLSS.","tokens_in":26809,"tokens_out":9143,"duration_ms":84431,"significance":"The bias decomposition in Eq. (11) is algebraically correct and useful: it isolates mechanisms that are often conflated in discussions of ML-based 2SLS, and it makes the forbidden-regression bias explicit. The simulation infrastructure is substantial: 1,000 iterations, several estimators, three high-complexity subcases, and an auxiliary set of exclusion-restriction-violation simulations in Figure 3. If the empirical ranking were robust, the practical guidance would be valuable, since applied researchers frequently consider whether to use ML first-stage predictions. The paper is also appropriately careful in several places, such as distinguishing the plug-in first-stage approach from split-sample and synthetic-instrument alternatives. However, the external validity of the central ranking is weakened by the fact that every simulated first-stage regression is linear, and there is at least one internal inconsistency between the summary statistics in Table 1 and the component decomposition in Table 2. These issues need to be addressed before the practical guidance can be taken at face value.","major_comments":[{"comment":"All simulated DGPs have a linear first-stage conditional expectation: in the low-complexity case g_x(z) = Σ z_i, and in the high-complexity case x1 = πz + ε_v. Nonlinear methods are therefore evaluated only in settings where they cannot improve the true first-stage model, so their flexibility can only add variance or recover endogenous variation. Figure 3 and Appendix A.2 vary the structural error by adding k-way interactions of the instruments to u, but the first stage remains linear in every case. Consequently, the abstract's unqualified claim that 'nonlinear methods generate substantial bias in second-stage estimates, potentially exceeding the bias of endogenous OLS' is not supported as a general practical regularity. It is supported only for linear-first-stage worlds. To make the practical guidance robust, the paper should either simulate genuinely nonlinear first-stage DGPs (e.g., quadratic, threshold, or interaction terms) where flexible methods could in principle capture useful exogenous variation, or explicitly qualify the conclusions to the linear-first-stage setting.","section":"3.1-3.2 and Abstract"},{"comment":"The component-decomposition table is internally inconsistent with the main results table for high-complexity case 1. In Table 1, column B reports means of 1.335 for naive OLS (bias 0.335), 1.058 for first-stage OLS (bias 0.058), and 1.562 for random forest with cross-validation (bias 0.562). Table 2, panel B reports corresponding bias entries of 0.22, 0.02, and 0.32. Since both tables are described as means over the same 1,000 iterations, these should agree to Monte Carlo error. The discrepancy is not a minor rounding issue; it is roughly 30-100 percent of the reported bias. This undermines the component-level claims for case 1 and needs to be reconciled, either by correcting the tables or by clarifying whether the 'Bias' column in Table 2 is a ratio of means rather than the mean of the per-iteration ratio.","section":"Table 2 vs. Table 1"},{"comment":"The high-complexity DGP is internally inconsistent in its definitions of the concentration parameter and the variance of x1. The paper states that σ²_v = π′Σ_zπ and that this 'forces Var(x1)=1'. But with Var(z_j)=1 for all j, Var(x1) = Var(πz) + σ²_v = π′Σ_zπ + π′Σ_zπ = 2π′Σ_zπ, so imposing Var(x1)=1 requires π′Σ_zπ = 0.5, not σ²_v = π′Σ_zπ. Moreover, under the stated definitions the concentration parameter is μ² = nπ′Σ_zπ/σ²_v = n, so it cannot be set to 180 independently of the sample size. Because the high-complexity simulations are designed around a particular concentration parameter and a particular sparsity pattern, this definitional inconsistency affects the interpretation of all three high-complexity subcases. The authors should correct the definitions and verify that the implemented DGP matches the reported μ².","section":"3.2"}],"minor_comments":[{"comment":"In Section 4.1, Lasso is listed among 'all four nonlinear ML-in-the-first-stage methods', but Lasso predictions are linear in the instruments; this is inconsistent with the abstract's classification of post-Lasso as a linear method and with the paper's own description of Class 3 in Section 2.2.","section":"4.1"},{"comment":"Equation (5) writes the wedge as f(β1Cov(ẋ,e), Cov(ẋ,u), 1/Var(ẋ)) with f non-decreasing in each argument, but the explicit formula in Eq. (11) shows that f is the product (a+b)c. It would be clearer to replace the informal f notation with the explicit expression.","section":"Eq. (5)"},{"comment":"The sentence 'The knife-edge case where a=0 appears unlikely...' uses the symbol a before it has been defined in that subsection; the reader has to infer that a refers to β1Cov(ẋ,e).","section":"Section 4.2"},{"comment":"There is a typo: 'Belloni et al. (2012)'s DPG' should be 'DGP'.","section":"3.2"},{"comment":"The caption says each cell is the mean across 1,000 iterations, but the discrepancy between the 'Bias' column in panel B and Table 1 column B suggests that the decomposition entries may not be computed on the same iterations or may summarize a nonlinear function of per-iteration components. The caption should state exactly how each column is aggregated.","section":"Table 2 caption"}],"recommendation":"major_revision","confidential_remarks":"The core decomposition is sound and the paper could make a useful contribution after revision. The two load-bearing issues are the complete absence of nonlinear first-stage DGPs and the Table 1/Table 2 inconsistency in high-complexity case 1; the latter looks like an error in the paper rather than a property of the estimator. I would also ask the authors to double-check the high-complexity DGP definitions before resubmission, since the stated σ²_v and μ² formulas cannot both hold as written."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Ed,\n\nQuick read on Lennon-Rubin-Waddell (arXiv:2505.13422). Worth your time if you supervise applied work touching ML and IV, but the headline claim is broader than the simulations actually support.\n\nThe useful core is a clean algebraic decomposition of the ML-in-2SLS wedge into three parts: beta1*Cov(xhat,e), Cov(xhat,u), and the 1/Var(xhat) amplifier. The math is correct, and the third component is the genuine insight: prediction-oriented ML deliberately shrinks the variance of fitted values, and that shrinkage mechanically inflates whatever bias the other two components create. That reframes the forbidden regression in a way that is actually helpful. The simulation sweep is broad and internally consistent: 1,000 iterations per design, Monte Carlo errors reported, traditional estimators, selection methods, and ML first stages all compared, plus a thoughtful robustness exercise that adds k-way instrument interactions to the structural error. The discussion of Chen et al.'s MLSS estimator and the strengthened exclusion restriction is honest. As practical guidance within the design, the conclusions hold: post-Lasso and PCA are safe; naive random forests are not.\n\nThe soft spots. The big one is exactly what the stress-test says: every simulated first stage is linear. The low-complexity DGP sets g_x(z) = sum of z_i, and the high-complexity design is Belloni et al.'s linear specification. Nonlinear learners are only evaluated in a world where flexibility cannot improve first-stage fit; it can only add variance or recover endogeneity. The abstract's unqualified claim that nonlinear methods generate substantial bias is a statement about a linear-world simulation, not a demonstrated regularity. A genuinely nonlinear first stage could plausibly reverse the ranking, since a tree or network might capture useful exogenous variation that linear 2SLS misses. The paper's own Section 3 admits practitioners rarely know the degree of nonlinearity, which makes the omission load-bearing, not cosmetic. Smaller issues: Section 4.1 calls Lasso a nonlinear method, which is wrong; Lasso is linear in z, and its bias in their own Table 2 comes from shrinkage, i.e., variance reduction. The code is promised but not posted. The decomposition is tautological, which is fine for a guidance paper, but they should say so plainly.\n\nWho it is for: applied economists thinking about ML first stages, and anyone teaching why the forbidden regression bites harder with flexible learners. Not a methodological breakthrough, but a solid, citable simulation study. It deserves a serious referee. My recommendation: send it to review, with required revisions--add at least one nonlinear first-stage DGP, fix the Lasso classification, temper the abstract to match the design, and post the code.","headline":"A solid, internally consistent simulation study warning against naive ML first stages in 2SLS, with a genuinely useful bias decomposition--but every DGP has a linear first stage, so the headline about nonlinear methods is narrower than the abstract claims.","tokens_in":27363,"tokens_out":6960,"would_cite":true,"duration_ms":61240,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that putting nonlinear machine-learning methods in the first stage of 2SLS can generate more bias in causal estimates than ignoring endogeneity altogether.","keywords":["machine learning","instrumental variables","two-stage least squares","first stage","bias decomposition","random forest","post-Lasso","forbidden regression"],"falsifier":"Re-run the high-complexity simulation with a genuinely nonlinear first stage, for example $x = z_1^2 + z_2 + z_3^3 + \\varepsilon$, and compare random-forest and boosted-tree 2SLS against naive OLS and post-Lasso; if the flexible methods show less bias than naive OLS in that design, the paper's across-the-board warning about nonlinear ML first stages would need a boundary.","tokens_in":26343,"feed_emoji":"📉","tokens_out":10378,"duration_ms":91876,"temperature":0.7,"pith_summary":"The paper asks whether using machine learning to predict the endogenous regressor in the first stage of two-stage least squares improves or degrades causal estimates, and it answers with a decomposition plus Monte Carlo evidence. It claims that linear machine-curated first stages, such as post-Lasso and principal-component analysis, perform at least as well as traditional 2SLS, while nonlinear methods such as random forests, boosted trees, and neural networks generate substantial second-stage bias. In the simulated designs, random-forest-based 2SLS produces more bias in the causal parameter than plain OLS that ignores endogeneity altogether. The practical stakes: a researcher who inserts a flexible predictor into the first stage can end up farther from the truth than one who makes no endogeneity correction at all.","feed_headline":"Flexible ML first stages can bias IV estimates more than no IV","feed_subtitle":"Three bias channels explain why flexible ML first stages can worsen IV estimates.","key_machinery":"The load-bearing object is the bias decomposition. For any first-stage prediction $\\hat{x}$, the wedge between the 2SLS estimate and the true effect is \\[\\hat\\$beta_1^{{\\mathrm{2SLS}}$} - \\beta_1 = \\frac{\\beta_1\\, \\widehat{\\mathrm{Cov}}(\\hat{x}, e) + \\widehat{\\mathrm{Cov}}(\\hat{x}, u)}{\\widehat{\\mathrm{Var}}(\\hat{x})}.\\] Here $e = x - \\hat{x}$ is the first-stage residual and $u$ is the structural error; the identity is derived by substituting $y = \\beta_0 + \\beta_1 x + u$ into the second-stage OLS formula. The three terms isolate the forbidden-regression correlation, the exclusion-restriction violation, and the variance-reduction amplifier, and the paper uses the decomposition to attribute the simulated bias of each estimator across the four designs.","core_discovery":"The paper establishes, in its simulations, a mechanical explanation of when ML-assisted 2SLS fails. Writing the second-stage estimate as the true effect plus a wedge that depends on the first-stage predictions, the paper shows three channels: nonlinear predictors can correlate with their own residuals, flexible algorithms can recover endogenous variation in the regressor even from valid instruments, and variance reduction in predictions amplifies whichever covariance bias exists. Across a low-complexity design and three high-complexity designs with 100 instruments, post-Lasso and PCA first stages match or beat OLS-based 2SLS, whereas Lasso, boosted trees, neural networks, and especially random forests shift the distribution of estimates away from the true effect. Random-forest 2SLS is worse on average than naive OLS in every design the paper simulates.","pith_inferences":["Editorial inference: all simulated first stages are linear in the instruments, so nonlinear methods are never evaluated in a regime where their flexibility could capture genuine nonlinear signal; a nonlinear first-stage DGP could reverse the ranking between flexible and linear methods.","Editorial inference: the decomposition suggests a practical diagnostic, since the first and third components, $\\widehat{\\mathrm{Cov}}(\\hat{x}, e)$ and $\\widehat{\\mathrm{Var}}(\\hat{x})$, can be computed from a researcher's own first-stage fit and used to detect the forbidden-regression and amplifier channels before trusting an ML-in-2SLS estimate.","Editorial inference: because the second bias channel depends on the algorithm recovering the endogenous part of $x$, the paper's guidance may shift with instrument strength and the degree of endogeneity; varying the concentration parameter and the endogeneity correlation would map where the bias ranking flips.","Editorial inference: the appendix's discussion of monotonicity and heterogeneous treatment effects suggests that with varying effects, nonlinear first stages may estimate a different weighted average of effects than linear 2SLS, a possibility the main text does not quantify."],"forward_implications":["Post-Lasso and PCA first stages preserve linearity and deliver estimates whose bias is at least as low as standard 2SLS in the simulated designs, making them the safe ML-assisted choices.","Random-forest and boosted-tree first stages can produce second-stage bias exceeding that of naive OLS, so using them off the shelf can make endogeneity problems worse rather than better.","Because the first bias component is multiplied by $\\beta_1$, larger true treatment effects produce larger absolute bias from the same nonlinear first-stage procedure.","Variance reduction, normally a virtue in prediction, inflates ML-in-2SLS bias whenever the covariance terms are nonzero, so prediction-optimal tuning is not causal-estimation-optimal tuning.","Remedies such as out-of-sample predictions or using ML output as a synthesized instrument can shut down some bias channels but require strengthening the exclusion restriction to all learnable functions of the instruments."],"supporting_citations":[{"why":"Defines the forbidden-regression problem that corresponds to the $\\widehat{\\mathrm{Cov}}(\\hat{x}, e)$ term in the decomposition.","marker":"Angrist and Krueger (2001)"},{"why":"Supplies the high-complexity DGP, the concentration-parameter calibration, and the post-Lasso procedure that performs well in the simulations.","marker":"Belloni et al. (2012)"},{"why":"Argues nonlinear first stages impose additional exclusion restrictions, supporting the $\\widehat{\\mathrm{Cov}}(\\hat{x}, u)$ channel.","marker":"Angrist and Frandsen (2022)"},{"why":"Offers the split-sample and synthetic-instrument estimator that the paper presents as the closest solution to the naive ML-in-2SLS problem.","marker":"Chen et al. (2020)"},{"why":"Motivates treating the first stage as a prediction problem and placing ML in the prediction-focused part of the econometric toolkit.","marker":"Mullainathan and Spiess (2017)"},{"why":"Defines random forests, the method with the largest simulated second-stage bias.","marker":"Breiman (2001)"},{"why":"Defines Lasso, used both directly and as the selection step in post-Lasso first stages.","marker":"Tibshirani (1996)"},{"why":"Provides the LIML estimator used as a low-bias baseline in the many-instrument designs.","marker":"Fuller (1977)"}],"fun_headline_variants":["Random forest first stages can bias IV worse than OLS","Nonlinear ML first stages can bias 2SLS past OLS","When ML first stages backfire: three bias channels","Flexible ML first stages can worsen IV bias"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The premise the results depend on is that the first-stage relationship between the instruments and the endogenous regressor is linear in every simulated design, so the nonlinear methods are never given a chance to improve on the true model.","fun_headline_variants_meta":{"raw":{"variants":["Random forest first stages can bias IV worse than OLS","Nonlinear ML first stages can bias 2SLS past OLS","When ML first stages backfire: three bias channels","Flexible ML first stages can worsen IV bias"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000868,"raw_usage":{"total_tokens":3712,"prompt_tokens":849,"completion_tokens":2863,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":2796}},"tokens_in":465,"tokens_out":2863,"duration_ms":21327,"temperature":1.0,"reasoning_tokens":2796,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:13:42.244761+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the high-complexity simulation with a genuinely nonlinear first stage, for example $x = z_1^2 + z_2 + z_3^3 + \\varepsilon$, and compare random-forest and boosted-tree 2SLS against naive OLS and post-Lasso; if the flexible methods show less bias than naive OLS in that design, the paper's across-the-board warning about nonlinear ML first stages would need a boundary.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the forbidden-regression problem that corresponds to the $\\widehat{\\mathrm{Cov}}(\\hat{x}, e)$ term in the decomposition."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the high-complexity DGP, the concentration-parameter calibration, and the post-Lasso procedure that performs well in the simulations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Argues nonlinear first stages impose additional exclusion restrictions, supporting the $\\widehat{\\mathrm{Cov}}(\\hat{x}, u)$ channel."},{"cited_title":"and Spiess, J","cited_arxiv_id":null,"evidence_quote":"Motivates treating the first stage as a prediction problem and placing ML in the prediction-focused part of the econometric toolkit."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Lasso, used both directly and as the selection step in post-Lasso first stages."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the LIML estimator used as a low-bias baseline in the many-instrument designs."}],"review_version":1}