Testing for Spillovers in Resource Conservation: Evidence from a Natural Field Experiment

Lorenz Goette; Zhi Hao Lim

arxiv: 2508.04371 · v2 · submitted 2025-08-06 · 💰 econ.GN · q-fin.EC

Testing for Spillovers in Resource Conservation: Evidence from a Natural Field Experiment

Lorenz Goette , Zhi Hao Lim This is my paper

Pith reviewed 2026-05-19 01:07 UTC · model grok-4.3

classification 💰 econ.GN q-fin.EC

keywords behavioral interventionsspillover effectsresource conservationnatural field experimentwater useenergy consumptionnudges

0 comments

The pith

Water-saving interventions reduce shower use with no spillovers to air-conditioning consumption.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether nudges that encourage conservation in one resource domain automatically affect behavior in another. It deploys a natural field experiment with over 2,000 households that randomly assigns real-time feedback and social-comparison messages either to shower water use or to air-conditioning energy use. Direct effects appear only for the water interventions, which lower shower consumption, while the energy intervention produces no measurable change in air-conditioning. Cross-domain spillovers are estimated at precise zero in both directions. A sympathetic reader cares because the result bears on whether conservation programs can be designed narrowly or must anticipate unintended transfers between resource categories.

Core claim

The authors report significant reductions in shower use from both water-saving interventions, but no direct effect of the energy-saving intervention on air-conditioning use. They estimate precise null effects of water-saving interventions on air-conditioning use and of the energy-saving intervention on shower use.

What carries the argument

Random assignment of three distinct interventions across households, with separate measurement of shower water and air-conditioning electricity consumption to isolate direct versus spillover channels.

Load-bearing premise

The three interventions were randomly assigned to households and there are no unmeasured factors that simultaneously affect both shower and air-conditioning consumption independently of the treatments.

What would settle it

A statistically significant change in air-conditioning use after a water-saving intervention, or in shower use after an energy-saving intervention, in a replication with similar sample size would falsify the reported null spillovers.

read the original abstract

This paper studies whether behavioral interventions designed to promote resource conservation in one domain generate spillovers in another. Using a natural field experiment involving over 2,000 residents, we identify the direct and spillover effects of real-time feedback and social comparisons on water and energy consumption. We implement three interventions: two targeting shower use and one targeting air-conditioning use. We find significant reductions in shower use from both water-saving interventions, but no direct effect of the energy-saving intervention on air-conditioning use. For spillovers, we estimate precise null effects of water-saving interventions on air-conditioning use, and of the energy-saving intervention on shower use.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Water interventions cut shower use with precise null spillovers to AC, but the energy arm's null direct effect makes its spillover null less informative.

read the letter

The main takeaway is that this experiment finds reductions in shower use from water-saving nudges but no spillovers to air conditioning use, while the energy nudge shows neither direct nor spillover effects. The water results give the cleaner test. The paper does a solid job setting up a natural field experiment with over 2,000 residents and three separate interventions. It measures both direct impacts and cross-domain spillovers using real consumption data. The precise null spillover estimates are new for this water-energy combination and could inform whether utilities can rely on one nudge to affect multiple resources. What stands out is the honest reporting of the null direct effect for the energy intervention. That said, this creates a soft spot for the spillover claim from energy to showers. If the treatment did not change air conditioning behavior, then the lack of change in showers does not strongly rule out spillovers; it might just mean the nudge was not effective in that domain. The authors need to address why the energy feedback produced no movement, perhaps through compliance checks or heterogeneity analysis. Overall, this is useful for people working on behavioral conservation programs. It shows that spillovers are not automatic, at least from the water side. A reader focused on policy design or spillover mechanisms would get something out of it. I think it deserves peer review. The field experiment design is straightforward and the questions are relevant enough that referees can help sharpen the interpretation around the uneven direct effects.

Referee Report

2 major / 1 minor

Summary. The paper reports results from a natural field experiment with over 2,000 residents testing direct and spillover effects of behavioral interventions on resource conservation. Two interventions target shower use (real-time feedback and social comparisons) while one targets air-conditioning use; the central claims are significant direct reductions in shower use from the water interventions, no direct effect of the energy intervention on AC use, and precise null spillover effects in both directions.

Significance. If the experimental design and null results hold after addressing the issues below, the study would provide field-experimental evidence against behavioral spillovers between water and energy domains, informing whether domain-specific conservation policies generate unintended cross-effects. The large sample and use of real consumption data are strengths that enhance credibility over lab or survey designs.

major comments (2)

[Abstract] Abstract: The claim of a precise null spillover from the energy-saving intervention onto shower use is undermined by the reported absence of any direct effect on air-conditioning use. Without evidence that the energy treatment shifted its targeted outcome (e.g., via compliance rates, feedback intensity, or arm size), this arm tests a non-manipulation rather than the presence or absence of spillovers; the water-intervention arms supply the stronger test, so the overall 'no spillovers' conclusion rests partly on an uninformative component.
[Experimental Design / Results] Experimental Design / Results sections: The manuscript provides no per-arm sample sizes, randomization balance tables, or pre-analysis plan, making it impossible to verify that the precise null spillover estimates are adequately powered or that exclusion restrictions hold; these details are load-bearing for interpreting the null effects as evidence against spillovers rather than under-powered tests.

minor comments (1)

[Abstract] Abstract: Specify the exact content and delivery mechanism of the 'real-time feedback and social comparisons' messages to allow readers to assess domain salience and replicability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed comments on our manuscript. These suggestions help clarify the interpretation of our results on the absence of cross-domain spillovers. We respond to each major comment below and indicate the revisions we will make.

read point-by-point responses

Referee: [Abstract] The claim of a precise null spillover from the energy-saving intervention onto shower use is undermined by the reported absence of any direct effect on air-conditioning use. Without evidence that the energy treatment shifted its targeted outcome (e.g., via compliance rates, feedback intensity, or arm size), this arm tests a non-manipulation rather than the presence or absence of spillovers; the water-intervention arms supply the stronger test, so the overall 'no spillovers' conclusion rests partly on an uninformative component.

Authors: We agree that the null direct effect of the energy intervention on air-conditioning use limits the informativeness of the spillover test in that direction. The energy arm was designed to mirror the structure of the water interventions (real-time feedback and social comparisons), but behavioral change in air-conditioning proved more resistant in this field setting. In the revision we will update the abstract and discussion to emphasize that the primary and stronger evidence against spillovers derives from the water interventions, which produced clear direct reductions in shower use together with precise null effects on air-conditioning. We will also add available details on implementation intensity and any compliance indicators for the energy arm to better contextualize its null direct effect, while retaining the bidirectional design as supplementary evidence. revision: partial
Referee: [Experimental Design / Results] The manuscript provides no per-arm sample sizes, randomization balance tables, or pre-analysis plan, making it impossible to verify that the precise null spillover estimates are adequately powered or that exclusion restrictions hold; these details are load-bearing for interpreting the null effects as evidence against spillovers rather than under-powered tests.

Authors: We acknowledge the importance of these details for assessing the credibility of the null spillover results. In the revised manuscript we will add a table reporting per-arm sample sizes, a randomization balance table with baseline covariates across arms, and an expanded methods section that describes our pre-specified analysis plan (including variable definitions, regression specifications, and any power considerations). These additions will allow readers to evaluate the precision of the spillover estimates and the plausibility of the exclusion restriction. revision: yes

Circularity Check

0 steps flagged

No circularity: identification from randomized field experiment

full rationale

The paper is a primary data collection study reporting results from a natural field experiment with random assignment of three interventions (two water-saving, one energy-saving) to over 2,000 households. Direct effects and spillover effects are identified from the experimental design and observed consumption data rather than from any derivation chain, fitted parameters, or equations. No self-definitional relations, predictions that reduce to inputs by construction, uniqueness theorems, or ansatzes appear in the reported analysis. The central claims rest on the random assignment and the observed null and positive effects, which are externally falsifiable. This matches the default expectation for an empirical RCT paper with no load-bearing self-citation or renaming of known results.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The paper relies on standard assumptions of randomized field experiments (random assignment, no interference, stable unit treatment value assumption) and conventional econometric inference for null effects.

axioms (2)

domain assumption Random assignment of interventions to households
Required for causal identification of direct and spillover effects
domain assumption No unmeasured confounders affecting both water and energy use
Needed to interpret null spillovers as absence of behavioral cross-effects

pith-pipeline@v0.9.0 · 5627 in / 1235 out tokens · 36608 ms · 2026-05-19T01:07:58.416082+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

We estimate the average treatment effects (ATE) of being assigned to real-time feedback (RTF) and/or social comparisons (SC-S for shower usage or SC-A for aircon usage) using a difference-in-differences regression model of the form: yit = αi + λt + β1RTFit + … (eq. 1)
IndisputableMonolith/Foundation/ArrowOfTime.lean z_monotone_absolute unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Result 1: No evidence of attentional spillovers. The RTF intervention reduces shower water usage (target behavior), but does not impact air conditioning usage (non-target behavior), on average.

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.