{"id":"d298121d-91e6-4b6b-94e6-7f582a24d724","arxiv_id":"2505.08323","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A randomized survey experiment with 490 Japanese budget officers shows that a subsidy, a nudge, and their combination each increase assessed budgets, while only the nudge increases self-financed allocations.","lead":"Budget officers in Japanese municipalities were randomly assigned to see a 50% national subsidy, a nudge about peer behavior and potential losses, both, or neither when assessing an HPV vaccination awareness campaign. All three interventions raised the total budget they would allocate, and only the nudge raised the portion funded from the municipality's own revenue.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The financial-incentive arm's total-budget effect is largely an accounting artifact: the outcome includes the 50% national grant, and self-financed spending is unchanged (Table 3, Cols. 3-4), so the headline claim conflates accepting a subsidy with increasing local budget allocation.","rationale":"The reader's formally stated weakest assumption was external validity (hypothetical scenarios), which the authors themselves acknowledge in Section 7. However, the reader's rationale also mentioned that the incentive arm's total-budget effect is driven by the inclusion of the subsidy in the outcome. I consider this latter point the most load-bearing concern because it affects the internal interpretation of the central claim, not just its generalization. The paper is transparent in reporting Table 3 Columns 3-4 and Figure 1, so the statistical results are not in question. The issue is that the headline 'all three treatments significantly increased assessed budget amounts' is misleading for the incentive and combination arms: the increase in those arms is essentially the national grant added to an unchanged local share. The only treatment that raised the amount the local government itself would fund is the nudge. This does not invalidate the experiment, but it means the abstract should be reframed to distinguish 'total program size including the grant' from 'local budget allocation.' The reader's CONDITIONAL verdict already captures the need for careful interpretation, and my concern reinforces rather than overturns it. Therefore I recommend no change to the verdict, while urging the reframing and a replication package that includes the raw data so readers can verify the self-financed decomposition. I did not find a more serious internal inconsistency: randomization checks, pre-registration, and cluster-robust inference are appropriate, and the nudge effect is robust on self-financed amounts. The external-validity caveat stands but was already the reader's primary conditional reason.","tokens_in":16072,"tokens_out":6366,"duration_ms":69714,"concrete_test":"Re-estimate the main regression with the dependent variable set to the self-financed amount (total minus national grant) for all four arms, using the same specification as Table 3 Columns 3-4 but for the full sample. If the incentive and combination coefficients remain near zero and insignificant, the total-budget effect is fully attributable to the embedded subsidy. Additionally, for the incentive arm, compute the ratio of total to self-financed amounts among non-zero responses; if the ratio clusters near 2, respondents treated the total as their unchanged local share plus the mechanical grant. A supplementary check: re-run the main analysis excluding the grant-eligible arms and testing the nudge effect alone; this isolates the behavioral effect that survives the outcome-construction issue.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In the incentive and combination arms, the dependent variable 'total assessed budget amount' is explicitly defined to include the central government subsidy (Appendix 1: '国からの補助額を含めた総額を記入してください'). In the control and nudge arms no subsidy exists, so total equals self-financed amount. Thus the cross-arm comparison of total assessed amounts is not a comparison of the same decision variable. Table 3 Columns 3-4 show that relative to control, self-financed amounts change by -34,350 JPY (incentive) and +140,470 JPY (combination), both insignificant; the nudge increases self-financed by 66,538 JPY (p<.01). In Figure 1, the incentive arm's average total (1,436,070 JPY) is approximately twice its self-financed component (718,035 JPY), consistent with respondents leaving local spending unchanged and the subsidy mechanically doubling the reported total. The paper's central claim that 'both the financial incentive and the nudge independently increased assessments' is therefore true only under the definition that counts the national grant as part of the officer's assessment. What the officer actually decided to allocate from local resources did not change in either incentive arm. This is not a statistical error, but it changes the policy interpretation: the incentive did not make budget officers value the campaign more; it merely added a matching grant to an unchanged local contribution. The claim 'only the nudge raised the self-financed portion' (abstract) is the honest finding, but the headline 'increased assessed budget amounts' for the incentive arms is potentially misleading.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a mail-based survey experiment with 490 Japanese municipal budget officers, using a 2x2 factorial design to test the effects of a 50% national subsidy (financial incentive) and a loss-framing/peer-information nudge on assessed budget amounts for an HPV vaccination awareness campaign. The authors find that all three treatments raise the total assessed budget amount, with the combination having the largest effect, and that only the nudge raises the self-financed portion. The paper interprets this as evidence that low-cost nudges can match financial incentives in changing budget assessments.","tokens_in":16338,"tokens_out":4966,"duration_ms":44170,"significance":"The experimental design is a clear strength: the study is pre-registered, uses stratified randomization, reports balance tests, employs cluster-robust standard errors, and samples real budget officers from all Japanese municipalities. If the main finding were correctly stated, the paper would make a useful contribution to behavioral public administration and public budgeting. However, the central interpretation is undermined by the non-comparability of the total-budget outcome across treatment arms, as detailed in the major comments. The nudge finding on the self-financed margin is novel and policy-relevant, but the framing of the incentive effects needs substantial revision.","major_comments":[{"comment":"The total assessed budget amount is defined differently across treatment arms: in the control and nudge arms, total equals the self-financed amount, while in the incentive and combination arms, total includes the central government grant (see the instruction '国からの補助額を含めた総額を記入してください' in Appendix 1). Table 3, Columns 3-4 show that relative to control, self-financed amounts change by -34,350 JPY (Incentive) and +140,470 JPY (Combination), both statistically insignificant, while the nudge increases self-financed by 66,538 JPY (p<.01). The incentive arms' total increases of 678,350 and 1,066,810 JPY are therefore largely or entirely mechanical, reflecting the inclusion of the grant rather than an increase in the local budget decision. The abstract's claim that 'Both the financial incentive and the nudge independently increased assessments' is thus misleading: the incentive did not increase the amount the officer chose to allocate from local resources. The authors should either make the self-financed amount the primary outcome or clearly frame the total as total project cost, and adjust the abstract and policy conclusions accordingly.","section":"Appendix 1, Table 3 (Columns 3-4), Abstract"},{"comment":"The statement that the financial incentive and the nudge had 'comparable' effects holds only for the total-budget outcome. For the self-financed outcome (Columns 3-4 of Table 3), the nudge is significant while the incentive is not. Presenting the two interventions as equivalent on a decision-relevant margin is inaccurate; the only intervention that increased the local government's own contribution is the nudge. This is not a cosmetic wording issue, because it reverses the policy conclusion about the incentive's effectiveness relative to the nudge.","section":"Section 5.2, Abstract"},{"comment":"Using the far-sighted group's control-group mean as a normative benchmark to claim that the treatments 'corrected' undervaluation is a post-hoc interpretation that was not pre-registered. The argument that the combination treatment may have led to overestimation is speculative, and the subsequent claim that the 'appropriate level' might exceed the far-sighted group's mean is ad hoc. This section should be labeled as exploratory, and the subgroup analysis should not be given a causal interpretation because time-discounting tendency was not randomly assigned.","section":"Section 6.3"}],"minor_comments":[{"comment":"The units of the budget amounts are not clearly stated in the main text; values such as 187.246 in Table 2 and 111.522 in Table 3 appear to be in units of 10,000 JPY, but this should be stated explicitly for readers.","section":"Table 2 and Table 3"},{"comment":"The figure caption should define 'National grant' and 'Self-financed' and explain how the self-financed amount is computed for the incentive arms (total minus the 50% grant), so the reader can follow the decomposition without consulting the appendix.","section":"Figure 1"},{"comment":"The manipulation check reports sensitivity to 'projected funding for FY2024' and 'performance targets for FY2024,' but it is not immediately clear which items in the survey (E1-3) correspond to these constructs; please map the figures to the specific survey questions.","section":"Section 6.1"}],"recommendation":"major_revision","confidential_remarks":"The experimental design is strong, but the framing of the primary outcome needs to be revised before the paper can be accepted. I would encourage the editor to request a resubmission in which the self-financed amount is treated as the primary outcome (or the total is described strictly as total project cost), and the abstract and conclusions are rewritten to avoid the misleading suggestion that the financial incentive increased local budget allocations. The nudge-only finding is the paper's most credible and novel contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my read of arXiv:2505.08323. The headline result to know before you get to the paper: the financial-incentive arm's increase in total assessed budget is real but mechanical—the outcome includes the national grant, and the self-financed portion is unchanged (Table 3, Cols. 3–4). The nudge is the only treatment that raises self-financed spending. The paper reports this split clearly in Section 5.2, but the abstract and policy conclusions lead with the total-budget framing, which can easily be misread as the incentive making officers value the campaign more. That's the main weakness and it's addressable through reframing, not new data. What's genuinely new is the 2x2 design itself, applied to real budget officers in 490 Japanese municipalities: a 50% central-government matching grant, a peer-information/loss-framing nudge, and the combination. The prior literature, including the authors' own nudge-only field experiment, hadn't compared these head-to-head. The execution is solid. Stratification by municipality type, balance checks, pre-registration (AEA RCT Registry), cluster-robust standard errors at the prefecture level, and covariate-adjusted estimates are all there. The effect sizes are meaningful: roughly 1.1M JPY total for the combination and 0.66M for each alone. The internal validity holds up. The central claim in the abstract—'all three treatments significantly increased assessed budget amounts'—is true under the definition used, and the paper goes one step further and tells you the self-financed numbers, which is where the policy story actually lives. That split isn't a statistical error; it is the honest finding that only the nudge moves the locals' own money. The 2x2 also gives a sensible heterogeneous-types story for why the combination adds up. Smaller soft spots: the myopic/far-sighted classification rests on a single hypothetical time-discounting question, and the external validity of hypothetical budget assessments is an acknowledged limitation. No replication package is posted, which is a miss for a paper with pre-registration. Who should read this: people working on behavioral public budgeting, fiscal federalism, or nudge-vs-incentive comparisons in government. It's a legitimate extension and the data-collection effort is above the norm for this literature. My recommendation: send it to peer review. It deserves a serious referee. The right referee will push the authors to de-emphasize the total-budget framing for the incentive arms and make the self-financed result the headline, and to clarify that the incentive adds a grant without changing local contributions.","headline":"Solid pre-registered experiment showing only the nudge moves self-financed budgets; the incentive's headline effect is the subsidy itself, and the authors should reframe the story.","tokens_in":16902,"tokens_out":4480,"would_cite":true,"duration_ms":38082,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 50% national subsidy and a loss-framed peer nudge each raise municipal budget assessments by about ¥650,000, and combining them adds roughly ¥1.1 million; only the nudge lifts the locally funded share.","keywords":["budget allocations","nudges","financial incentives","survey experiment","local government","HPV vaccination","loss framing","peer information"],"falsifier":"Run a field version of the intervention in an actual budget-request cycle, sending the nudge and subsidy information to a randomly assigned set of municipalities and comparing final approved budgets to a control group; if the real increases fall far below the 650,000 to 1,100,000 JPY effects found in the survey, or do not appear at all, the claim that these interventions improve real budget assessments lacks support.","tokens_in":15849,"feed_emoji":"💰","tokens_out":13756,"duration_ms":114849,"temperature":0.7,"pith_summary":"Budget officers in local governments tend to underfund programs whose benefits arrive in the future, such as vaccination awareness campaigns. In a nationwide mail experiment with 490 Japanese municipal budget officers, the paper shows that a 50% central-government subsidy and a non-financial nudge—telling officers that neighboring municipalities have already launched similar campaigns and that failure to act will keep vaccination rates low—each raise the assessed budget by roughly ¥650,000 to ¥670,000 relative to no intervention. Combining the two yields the largest increase, about ¥1.1 million. The nudge is the only treatment that raises the self-financed portion of the budget, suggesting it works through psychological mechanisms rather than simply providing money. These findings imply that low-cost behavioral messages can be as effective as fiscal transfers in encouraging forward-looking budget decisions, at least in a hypothetical scenario.","feed_headline":"A nudge alone lifts budget assessments as much as a 50% subsidy","feed_subtitle":"A low-cost nudge matches a 50% subsidy in raising assessed budgets, and it alone increases locally funded spending.","key_machinery":"The engine of the experiment is a 2×2 between-subjects design in which two kinds of information are appended to an otherwise identical hypothetical budget-request scenario: a financial incentive (a 50% central-government subsidy) and a non-financial nudge (peer information that neighboring municipalities have already launched similar campaigns, combined with a loss-framed statement about the consequences of inaction). The outcome is the assessed budget amount, recorded both as the officer's individual assessment and as the amount after consultation with a manager, and separately as the total amount and the amount excluding the national grant. The loss-framed message targets time discounting and psychological hesitation; a time-discounting question (choosing ¥100,000 today versus ¥110,000 in one year) supports a heterogeneity analysis of whether the treatments correct myopic undervaluation.","core_discovery":"The paper reports a 2×2 between-subjects field-based survey experiment embedded in a realistic hypothetical budget request for an HPV vaccination awareness campaign. Across 490 valid responses from budget officers in Japanese municipalities, the total assessed budget (including any national grant) rose by 678,350 JPY for the financial-incentive group, 661,960 JPY for the non-financial-nudge group, and 1,066,810 JPY for the combination group, each relative to the control and significant at the 1% level. When the national-grant component is removed, only the nudge group shows a significant increase in the self-financed amount (661,960 JPY); the incentive and combination groups show no significant change in what the municipality itself pays. The authors interpret this as evidence that the nudge changes the intrinsic willingness to allocate local resources, whereas the subsidy expands the total budget without affecting the local contribution. The paper also finds that the treatment effects concentrate among budget officers classified as myopic, moving their assessments close to the level of far-sighted officers.","pith_inferences":["A natural extension not tested here is cost-effectiveness: because the nudge is nearly costless to administer, its effect per yen spent likely exceeds that of a 50% subsidy, and a follow-up could compare the marginal cost of each intervention.","The combination result—that the nudge's self-financed effect disappears when a subsidy is present—is consistent with motivation crowding-out, and it suggests that financial incentives can blunt the benefit of social-comparison framings in other intrinsic-motivation settings.","The HPV case is one instance of a broader class of delayed-benefit policies; the same 2×2 design could be applied to climate adaptation or preventive health, though the peer-information and loss-framing messages would need cultural adaptation.","The external-validity caveat points to a sharper test: linking survey responses to actual budget execution records for the same municipalities in subsequent fiscal years would reveal whether stated assessments predict real allocations."],"forward_implications":["If stated assessments translate into real budgets, a peer-information and loss-framing nudge can raise total funding for an undervalued campaign by roughly the same amount as a 50% national subsidy, at near-zero fiscal cost to the central government.","Combining the subsidy with the nudge yields a larger total increase than either alone (about ¥1.1 million over the control group), suggesting the two interventions operate through partly separate channels.","Only the nudge increases the self-financed local share; the subsidy alone or combined with the nudge leaves the municipality's own contribution unchanged, so policymakers who value local fiscal autonomy should prefer the nudge.","The treatment effects appear mainly among budget officers who discount future outcomes, moving their assessments up to the level of more far-sighted officers, which indicates the interventions correct undervaluation rather than simply inflating budgets.","Managerial review adds little except in the incentive-only group, since 0.867 JPY of every 1 JPY of the initial individual assessment survives the review process."],"supporting_citations":[{"why":"Prior field-based survey experiment on nudging budget officers; supplies the nudge design and the comparison point for effect size.","marker":"Kuroki and Sasaki (2023a)"},{"why":"Natural field experiment on matching subsidies in charitable giving; informs the prediction that a subsidy can either increase total giving or leave it unchanged.","marker":"Karlan and List (2007)"},{"why":"Review of matching-subsidy experiments; provides the theoretical framing for how subsidies affect total versus self-financed contributions.","marker":"Epperson and Reif (2019)"},{"why":"Large-scale evidence that nudge effects are often modest on average; motivates testing whether a nudge can match a subsidy.","marker":"DellaVigna and Linos (2022)"},{"why":"Motivation-crowding theory; used to explain why the nudge raises self-financed budgets alone but not when combined with the subsidy.","marker":"Frey and Jegen (2021)"},{"why":"Evidence of budget spillovers and fiscal interdependence among local governments; supports the peer-information nudge mechanism.","marker":"Case et al. (1993)"},{"why":"Classic study of policy diffusion; further support for the peer-information mechanism.","marker":"Walker (1969)"},{"why":"Projection that Japan's cumulative HPV vaccination rate will plateau below target; establishes the policy need and scenario realism.","marker":"Yagi et al. (2024)"}],"fun_headline_variants":["Nudge matches subsidy in budget boosts, but only nudge raises local share","Low-cost nudge rivals 50% subsidy for budget assessments","Nudge alone ups local budget share; subsidy only inflates total","For myopic budget officers, nudge and subsidy both lift assessments"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Budget officers' written assessments in a hypothetical mail survey reflect the amounts they would actually allocate in real budget decisions.","fun_headline_variants_meta":{"raw":{"variants":["Nudge matches subsidy in budget boosts, but only nudge raises local share","Low-cost nudge rivals 50% subsidy for budget assessments","Nudge alone ups local budget share; subsidy only inflates total","For myopic budget officers, nudge and subsidy both lift assessments"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000614,"raw_usage":{"total_tokens":2878,"prompt_tokens":996,"completion_tokens":1882,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":1806}},"tokens_in":612,"tokens_out":1882,"duration_ms":13311,"temperature":1.0,"reasoning_tokens":1806,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:57:32.208756+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a field version of the intervention in an actual budget-request cycle, sending the nudge and subsidy information to a randomly assigned set of municipalities and comparing final approved budgets to a control group; if the real increases fall far below the 650,000 to 1,100,000 JPY effects found in the survey, or do not appear at all, the claim that these interventions improve real budget assessments lacks support.","supporting_citations":[],"review_version":1}