{"id":"8a8155a5-3d30-4e62-9278-5d2356454df6","arxiv_id":"2411.11559","paper_version":2,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A general framework derives exact outcome weights for double machine learning and generalized random forest estimators, showing that standard implementations are only scale-normalized rather than fully-normalized.","lead":"Modern machine learning based treatment effect estimators, including causal forests and double machine learning, can be written exactly as weighted averages of observed outcomes, and this paper gives the exact weight formulas. It then shows that standard implementations of these methods produce weights that do not sum to one within treatment groups, a property with practical consequences for diagnostics and finite sample behavior.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Condition 1's uniqueness requirement is ill-posed: infinitely many S satisfy SY=Ŷ for observed Y, so Proposition 1's 'unique T' cannot hold as stated; weights are implementation-specific.","rationale":"The reader correctly identifies Condition 1 as the key assumption. I sharpen this into a formal problem: the stated uniqueness condition is not satisfiable because SY=hat Y is an underdetermined linear system; infinitely many matrices satisfy it for any observed Y. The paper actually relies on the algorithmically produced smoother matrix, which is unique only given the implementation and random seed. The closed-form weights are correct relative to that S, and the empirical applications (EMCS, covariate balancing) are credible and reproducible via the R package. The contradiction between Section 4.4 ('extraction ... currently not possible') and Section 5.1's use of DoubleML weights should be clarified but is secondary. The load-bearing concern is formal: Proposition 1 overstates uniqueness and Condition 1 cannot hold as written. This is fixable by rewording, so the appropriate verdict is CONDITIONAL rather than outright rejection.","tokens_in":24545,"tokens_out":12233,"duration_ms":127311,"concrete_test":"Use the OutcomeWeights R package to fit the same causal forest on a fixed dataset with two different seeds for the nuisance outcome forest, selecting seeds that yield identical fitted values hat Y (or identical hat tau) but different leaf assignments. Extract the outcome weights via Equation 13 and compare. If the weights differ while the point estimates coincide, the 'unique transformation matrix' claim fails and the paper must be revised to define weights relative to the fitted smoother. A second, purely analytical check: for any vector v with v'Y=0, S + u v' also satisfies Condition 1's equation, so uniqueness cannot be inferred from the observed data.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Proposition 1 requires a unique transformation matrix T with TY=tilde Y, and Condition 1 (Section 3.1.2) requires a unique smoother matrix S with SY=hat Y. As stated, this condition can never be satisfied: for any observed vector Y, there are infinitely many N x N matrices S satisfying SY=hat Y (add any matrix whose rows are orthogonal to Y). The paper's applications use the particular S produced by the estimation algorithm, which is unique only relative to the random seed and implementation. For adaptive/random smoothers, different fits can yield identical fitted values on the observed sample but different leaf structures, hence different outcome weights for the same estimator and same data. This does not invalidate the closed-form Corollaries 1-3, which hold for the extracted S, but it means the outcome weights are not a feature of the estimator alone; they are a feature of the estimator plus implementation and seed. The paper should replace 'unique' with 'the algorithm's' and acknowledge explicitly that weights are implementation-specific. A separate inconsistency worth flagging: Section 4.4 says extraction of outcome weights for DoubleML is 'currently not possible', yet Section 5.1 presents covariate-balancing plots using DML outcome weights; this needs clarification or the earlier sentence is misleading.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a general framework for expressing pseudo-IV estimators as linear combinations of observed outcomes. Given a pseudo-outcome vector Ỹ = T Y, any estimator that solves the moment condition E_N[(Ỹ_i − τ D̃_i) Z̃_i] = 0 has outcome weights ω' = (Z̃'D̃)^{-1} Z̃'T. The framework is applied to derive closed-form outcome weights for instrumental forests, causal forests, partially linear regression (with and without IV), AIPW, and Wald-AIPW, and it recovers known weights for OLS, TSLS, Wald, IPW, and related estimators. The paper also characterizes when the weights are scale-normalized or fully normalized, concluding that standard implementations of causal/instrumental forests and PLR-type estimators are scale-normalized but not fully normalized because the outcome smoother is not applied to the treatment. The theoretical predictions are illustrated with an empirical Monte Carlo study and a 401(k) covariate-balancing application.","tokens_in":24742,"tokens_out":10328,"duration_ms":105570,"significance":"If the results hold, the paper provides a practically valuable and broadly applicable toolbox: it gives the first closed-form outcome weights for six prominent DML/GRF estimators, includes an accompanying R package, and verifies the algebra with reproducible notebooks. The finding that standard DML/GRF implementations are not fully normalized is actionable for practice and complements the existing literature on implied weights. The framework is simple, the derivations are verifiable, and the empirical illustrations connect the weights to established diagnostics. The central derivation is sound conditional on a chosen smoother matrix; the main caveats concern the wording of the uniqueness condition and an internal inconsistency about whether DoubleML outcome weights can be extracted.","major_comments":[{"comment":"Condition 1 requires 'a unique smoother matrix' satisfying SY = Ŷ, and Proposition 1 requires a unique transformation matrix T with TY = Ỹ. As written this condition is never satisfied: for any observed Y and any fitted values Ŷ, adding to S a matrix whose rows are orthogonal to Y yields another matrix with the same action on Y. For outcome-adaptive or random smoothers, the estimation algorithm selects one such S, but different algorithms, tuning choices, or random seeds can produce identical fitted values with different smoothers and hence different outcome weights. The paper should replace 'unique' with 'the smoother matrix produced by the estimation algorithm' and state explicitly that the derived weights are implementation-specific rather than unique properties of the estimator alone. The algebraic formulas in Corollaries 1-3 remain correct for the extracted S, but Proposition 1 and the abstract's 'unique weighted representation' need revision.","section":"Section 3.1.2, Condition 1; Section 2.3, Proposition 1"},{"comment":"Section 4.4 states that for DoubleML implementations 'the extraction of the outcome weights is currently not possible because the required smoother matrices are not accessible,' yet Section 5.1 presents covariate-balancing plots for DML PLR, AIPW, PLR-IV, and Wald-AIPW that use the outcome weights derived in Section 3. Please clarify how the weights used in Figure 2 were obtained, which implementation they correspond to, and reconcile the statement in Section 4.4; otherwise the advertised application appears to rely on an extraction that the paper says is impossible.","section":"Section 4.4 vs Section 5.1"}],"minor_comments":[{"comment":"In the definition of the Wald-AIPW pseudo-treatment D̃_i^{iv-aipw}, the second treatment residual term uses λ_i^{ipw,z,1} twice; the second occurrence should presumably be λ_i^{ipw,z,0} for symmetry with the outcome part of the expression.","section":"Equation (18)"},{"comment":"The sentence reporting 'the sum of weights ranges from 0.98 to 1.02' is ambiguous: for scale-normalized causal forest weights the overall sum is zero, so the reported quantity must be a group-specific sum after the 2D_i − 1 sign adjustment. Please state explicitly which sum is being reported.","section":"Section 5.2"},{"comment":"The last bullet in the closing remarks contains a duplicated phrase: 'double robustness robustness properties' should read 'double robustness properties.'","section":"Section 6"},{"comment":"The note 'The shadowed boxes in rows 1, 3 and 9 zoom into...' should refer to panels rather than rows, or should otherwise match the layout of the figure.","section":"Figure 1 notes"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a strong fit for an econometrics journal, and the supplementary code and notebooks are a clear asset. The issues I raise are fixable within the scope of a revision: the uniqueness wording in Condition 1 and Proposition 1 should be corrected, and the apparent contradiction between Section 4.4 and Section 5.1 about DoubleML weight extraction needs to be resolved. I do not see grounds for rejection; after these revisions the paper should be publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is worth a serious referee. It delivers the first outcome weights for six workhorse DML and generalized random forest estimators—instrumental forest, causal forest, PLR, PLR-IV, AIPW, and Wald-AIPW—under a single pseudo-IV (PIVE) framework. The algebra is simple, but it hasn't been assembled this way before, and the payoff is practical: standard covariate-balancing diagnostics can now be applied to these estimators, including to CATEs from a causal forest. The noiseless-outcome EMCS is a nice touch: it converts the theoretical normalization claims into testable predictions about software output, and the predictions are confirmed. The R package and supplementary notebooks back the claims, which is real evidence.\n\nThe soft spots are minor. Condition 1 asks for 'a unique smoother matrix S' such that SY = Ŷ. As stated, that is false: for any observed Y there are infinitely many matrices that map Y to the fitted values, so 'unique' cannot mean uniqueness as a matrix solution. The paper clearly intends 'the smoother matrix that the estimation algorithm actually uses,' and the weights are implementation-specific by design—that is a central message. But the wording should be corrected because, taken literally, it makes Proposition 1 impossible to instantiate. A sentence acknowledging that different algorithms, seeds, or software versions can produce different weights for the same estimator would also align the paper with its own message.\n\nThe second wrinkle is an apparent contradiction between Section 4.4, which says DoubleML outcome-weight extraction is 'currently not possible,' and Section 5.1, which presents covariate-balancing plots using DML outcome weights. The likely resolution is that the Section 5.1 implementation uses accessible smoothers (e.g., grf) rather than the DoubleML package internals, but as written it is confusing and a referee should ask for clarification.\n\nThe central argument holds. The stress-test concern that weights are implementation-specific is real but not a flaw in the framework—it is the paper's own headline finding. The derivation is exact given the extracted smoother matrix, and the Monte Carlo confirms it against actual software output.\n\nThis paper is for applied causal inference users and for methodologists who care about weighting representations. It deserves a serious peer review and will likely be accepted with a minor revision. I would cite it and bring it to the reading group.","headline":"A useful diagnostic paper that first derives outcome weights for six DML/GRF estimators; the uniqueness wording in Condition 1 needs fixing, but the contribution holds up.","tokens_in":25300,"tokens_out":3707,"would_cite":true,"duration_ms":33927,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G08","62P20","62G05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Every pseudo-IV estimator can be rewritten as a weighted sum of outcomes; the paper derives those weights for six machine-learning treatment-effect estimators.","keywords":["outcome weights","implied weights","pseudo-IV estimators","double machine learning","generalized random forests","causal forest","covariate balancing","treatment effect estimation"],"falsifier":"Take a DML or generalized-random-forest implementation and fit it on a dataset with outcome $Y_i = 1 + D_i$ and no noise; the theory predicts exactly one only for fully-normalized weights. If a standard causal-forest or PLR implementation produced exactly one in every bootstrap sample, or a modified implementation using one affine smoother for both outcome and treatment failed to produce exactly one, the normalization claims would be refuted. A second direct check is to estimate with Lasso or logistic-regression outcome models: finding a unique fixed outcome-weight vector that exactly reproduces the estimate would violate Proposition 1's scope condition.","tokens_in":24321,"feed_emoji":"⚖️","tokens_out":12002,"duration_ms":103078,"temperature":0.7,"pith_summary":"This paper establishes that any estimator defined by a pseudo-IV moment condition — a scalar pseudo-outcome, pseudo-treatment, and pseudo-instrument solving $E_N[(\\tilde Y_i - \\hat\\tau \\tilde D_i)\\tilde Z_i] = 0$ — can be written as a linear combination of observed outcomes, provided the pseudo-outcome is produced from outcomes by a unique transformation matrix. Applied to double machine learning and generalized random forests, this yields the first closed-form outcome weights for instrumental forests, causal forests, partially linear regression with and without instruments, AIPW, and Wald-AIPW, while recovering known weights for OLS, TSLS, IPW, and related estimators as special cases. The weights reveal that implementation details, not just the estimator's label, decide whether treated outcomes receive weights summing to one and untreated outcomes summing to minus one: standard implementations of partially linear regression based estimators and causal/instrumental forests are only scale-normalized, whereas AIPW with affine smoothers is fully normalized. Because these weights plug directly into established diagnostics such as covariate-balance plots and extreme-weight checks, the result gives applied researchers a concrete way to interrogate grey-box machine-learning estimates.","feed_headline":"Derive outcome weights for causal forests and double ML","feed_subtitle":"New closed-form weights turn grey-box machine-learning estimates into checkable weighting sums.","key_machinery":"The load-bearing object is the pseudo-IV representation together with the transformation matrix. A pseudo-IV estimator solves the empirical moment condition $E_N[(\\tilde Y_i - \\hat\\tau \\tilde D_i)\\tilde Z_i] = 0$, and Proposition 1 states that its outcome weights are $\\omega' = (\\tilde{\\mathbf{Z}}'\\tilde{\\mathbf{D}})^{-1}\\tilde{\\mathbf{Z}}'\\mathbf{T}$ whenever a unique $\\mathbf{T}$ with $\\mathbf{T}\\mathbf{Y}=\\tilde{\\mathbf{Y}}$ exists. In the applications $\\mathbf{T} = \\mathbf{I}_N-\\mathbf{S}$, where $\\mathbf{S}$ is the smoother matrix that generates the outcome predictions inside the estimator, so the framework turns every smoothed-prediction estimator into an explicit weighting estimator. The same machinery classifies weight properties through three shortcut identities: $\\mathbf{T}\\mathbf{1}_N = \\mathbf{0}_N$ implies normalized weights, $\\mathbf{T}\\mathbf{D}=\\tilde{\\mathbf{D}}$ implies treated weights summing to one, and $\\mathbf{T}(\\mathbf{1}_N-\\mathbf{D})=-\\tilde{\\mathbf{D}}$ implies untreated weights summing to minus one. This reduction lets the paper translate implementation choices into a small set of checkable conditions on the smoother matrix.","core_discovery":"The central claim is that outcome weights of any pseudo-IV estimator have the closed form $\\omega' = (\\tilde{\\mathbf{Z}}'\\tilde{\\mathbf{D}})^{-1}\\tilde{\\mathbf{Z}}'\\mathbf{T}$, where $\\tilde{\\mathbf{Z}}$ and $\\tilde{\\mathbf{D}}$ are the pseudo-instrument and pseudo-treatment vectors and $\\mathbf{T}$ is the unique matrix mapping observed outcomes $\\mathbf{Y}$ into the pseudo-outcome $\\tilde{\\mathbf{Y}}$. Whenever such a $\\mathbf{T}$ exists, the weighted representation is numerically identical to the original moment-condition estimator. In this paper's applications $\\mathbf{T} = \\mathbf{I}_N - \\mathbf{S}$ is a generalized residual-maker built from the smoother matrix $\\mathbf{S}$ whose rows contain the smoothing weights of the outcome predictor. From this identity the paper derives the first outcome weights for instrumental forest, causal forest, partially linear regression, PLR-IV, AIPW, and Wald-AIPW, and shows how Wald, TSLS, OLS, regression adjustment, IPW, difference-in-means, and Wald-RA/IPW arise as special cases. A second, related claim is that weight normalization is controlled by explicit implementation conditions: affine smoothers make the total weight sum zero, and full normalization (treated weights summing to one, untreated weights summing to minus one) additionally requires that the same smoother predicting the outcome also predicts the treatment, or, for AIPW, that treatment groups are smoothed separately. The headline diagnostic result is that standard software implementations of PLR, causal forest, instrumental forest, PLR-IV, and Wald-AIPW are scale-normalized but not fully normalized.","pith_inferences":["The shortcut conditions suggest a practical tuning criterion the paper does not pursue: among similar-performing configurations, prefer implementations whose implied weights are closer to fully normalized, which in the paper's empirical illustration aligns with hyperparameter tuning.","Because adaptive smoothers make the outcome weights functions of the outcome itself, standard inference procedures that treat weights as fixed will not directly apply; sample-splitting the weight construction from the outcome evaluation is the natural remedy.","Since any estimator with a linear score can be rewritten as a pseudo-IV estimator, the closed-form proposition is a template for deriving implied weights for other moment-based or machine-learning estimators beyond the fourteen considered here.","The framework also reframes a design question for future estimators: if full normalization is desirable, one could deliberately share one affine smoother across outcome and treatment predictions and check whether finite-sample performance improves."],"forward_implications":["Researchers can now construct covariate-balance plots, common-support diagnostics, and extreme-weight checks for causal forests and DML estimators just as they do for classical weighting estimators.","Outcome weights are available exactly when the outcome nuisance is estimated with a smoother; the treatment and instrument nuisance models do not affect availability.","Under standard implementations, PLR, causal forest, instrumental forest, PLR-IV, and Wald-AIPW have scale-normalized but not fully normalized weights, so in a noiseless dataset with outcome $Y_i=1+D_i$ they will not reproduce the true effect of one exactly.","Forcing the outcome smoother to also predict the treatment (Condition 5a or 5b) makes those estimators fully normalized, and doing so is a user-controlled implementation choice.","AIPW with affine smoothers and group-specific outcome models is fully normalized by construction, regardless of whether the inverse-probability weights are themselves normalized."],"supporting_citations":[{"why":"Supplies the generalized random forest framework, including the instrumental and causal forests whose outcome weights are derived.","marker":"(Athey et al., 2019)"},{"why":"Supplies the double machine learning estimators PLR, PLR-IV, AIPW, and Wald-AIPW that the framework applies to.","marker":"(Chernozhukov et al., 2018)"},{"why":"Supplies the partially linear regression model at the core of the PLR and PLR-IV estimators.","marker":"(Robinson, 1988)"},{"why":"Introduces the Wald-AIPW estimator whose outcome weights are derived in Corollary 3.","marker":"(Tan, 2006)"},{"why":"Provides the normalization taxonomy and Abadie-kappa estimators that the paper recovers and classifies within the pseudo-IV framework.","marker":"(Słoczyński et al., 2024)"},{"why":"Establishes the tradition of outcome weights and the assumed desirability of weights summing to (minus) one within groups.","marker":"(Imbens & Rubin, 2015)"},{"why":"Supplies the implied weights for OLS that the framework recovers as a special case.","marker":"(Chattopadhyay & Zubizarreta, 2023)"},{"why":"Provides the IPW normalization whose known properties are recovered as a special case under Condition 6.","marker":"(Hájek, 1971)"},{"why":"Motivates the reverse direction by showing weighting estimators as regression estimators, complementing the regression-as-weighting result.","marker":"(Bruns-Smith et al., 2023)"}],"fun_headline_variants":["Closed-form outcome weights for causal forests and double ML","Treatment effect estimators are weighted outcomes: unified framework","Standard causal forest weights fail full normalization","New closed-form weights make ML effect estimates explicable","Outcome weights exist for causal forests, double ML, and more"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire framework depends on the existence of a unique smoother matrix $\\mathbf{S}$ such that $\\mathbf{S}\\mathbf{Y}$ equals the outcome predictions inside the estimator; for outcome models that are not linear smoothers of $\\mathbf{Y}$—Lasso, logistic regression, or many neural networks—the closed-form outcome weights do not exist under this framework.","fun_headline_variants_meta":{"raw":{"variants":["Closed-form outcome weights for causal forests and double ML","Treatment effect estimators are weighted outcomes: unified framework","Standard causal forest weights fail full normalization","New closed-form weights make ML effect estimates explicable","Outcome weights exist for causal forests, double ML, and more"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000809,"raw_usage":{"total_tokens":3587,"prompt_tokens":1017,"completion_tokens":2570,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":2495}},"tokens_in":633,"tokens_out":2570,"duration_ms":18641,"temperature":1.0,"reasoning_tokens":2495,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:21:57.025447+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a DML or generalized-random-forest implementation and fit it on a dataset with outcome $Y_i = 1 + D_i$ and no noise; the theory predicts exactly one only for fully-normalized weights. If a standard causal-forest or PLR implementation produced exactly one in every bootstrap sample, or a modified implementation using one affine smoother for both outcome and treatment failed to produce exactly one, the normalization claims would be refuted. A second direct check is to estimate with Lasso or logistic-regression outcome models: finding a unique fixed outcome-weight vector that exactly reproduces the estimate would violate Proposition 1's scope condition.","supporting_citations":[{"cited_title":", Chetverikov, D","cited_arxiv_id":null,"evidence_quote":"Supplies the double machine learning estimators PLR, PLR-IV, AIPW, and Wald-AIPW that the framework applies to."},{"cited_title":"APACrefauthors \\ 1988","cited_arxiv_id":null,"evidence_quote":"Supplies the partially linear regression model at the core of the PLR and PLR-IV estimators."},{"cited_title":"APACrefauthors \\ 2006 12","cited_arxiv_id":null,"evidence_quote":"Introduces the Wald-AIPW estimator whose outcome weights are derived in Corollary 3."},{"cited_title":"\\ Zubizarreta, J R","cited_arxiv_id":null,"evidence_quote":"Supplies the implied weights for OLS that the framework recovers as a special case."},{"cited_title":"Augmented balancing weights as linear regression","cited_arxiv_id":"2304.14545","evidence_quote":"Motivates the reverse direction by showing weighting estimators as regression estimators, complementing the regression-as-weighting result."}],"review_version":1}