{"id":"dcf50874-cdb8-4ab1-9f6e-2e1c480efaf8","arxiv_id":"2606.25688","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Develops a sensitivity-based criterion for selecting calibration-estimation partitions in structural models to minimize worst-case local bias in target objects from calibration errors.","lead":"This paper introduces a sensitivity statistic to choose which parameters to calibrate versus estimate in structural economic models, selecting the partition that minimizes local bias in target outputs like policy effects. A smart generalist might read it to see a systematic way to make model-based conclusions less dependent on arbitrary calibration choices.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"The sensitivity statistic uses a first-order derivative approximation whose accuracy for finite calibration errors is unverified in the NK application.","rationale":"The reader's weakest assumption directly identifies the load-bearing step; the local-linear construction is necessary for the worst-case claim to hold beyond infinitesimal errors, and the paper's NK results rest on it without reported validation.","tokens_in":1638,"tokens_out":293,"duration_ms":12403,"concrete_test":"For the selected versus next-best partition in the NK model, replace the local derivative with a finite-difference perturbation of calibrated parameters at the magnitudes shown in the paper's figures (e.g., 10-20% shifts); recompute the actual change in the target object and test whether the minimal-sensitivity partition still produces the smallest realized bias.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that the partition minimizing the sensitivity statistic also minimizes worst-case local bias from calibration errors. The statistic is built from local derivatives of the target (policy effect, welfare, etc.) w.r.t. calibrated parameters. This yields an exact bound on bias only in the infinitesimal limit. For the finite miscalibrations illustrated in the Nakamura-Steinsson NK example, higher-order terms or curvature could reverse the ranking. The abstract and procedure description give no analytic or numerical check that the linear ranking survives at the error magnitudes the paper deems \"sizeable.\"","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper treats the choice of which parameters to calibrate versus estimate in structural models as a partition-selection problem. For each admissible partition it constructs a scalar sensitivity statistic from the local derivatives of a target object (policy effect, welfare, impulse response, etc.) with respect to the calibrated parameters. The partition that minimizes this statistic is selected on the grounds that it minimizes worst-case local bias from calibration errors. The procedure is illustrated in two canonical examples and then applied to the New Keynesian model of Nakamura and Steinsson (2018), where the authors report that some partitions remain reliable under sizeable miscalibrations while others generate large bias from small calibration errors. The method uses only local derivatives and avoids repeated re-estimation.","tokens_in":1750,"tokens_out":470,"duration_ms":17486,"significance":"If the local linear approximation is shown to be reliable at the error magnitudes the authors deem relevant, the procedure supplies a computationally light, systematic criterion for partition choice that could improve the credibility of structural-model results, especially in applications where the target object is a policy or welfare quantity. The explicit demonstration that partition choice can materially affect robustness in the NK setting is a useful illustration. The fact that the statistic is obtained from local derivatives without re-estimation is a practical strength.","major_comments":[{"comment":"Abstract and Nakamura-Steinsson application: the central claim is that the partition minimizing the sensitivity statistic also minimizes worst-case local bias from calibration errors. The statistic is constructed from first-order derivatives, so the bound on bias is exact only in the infinitesimal limit. The manuscript provides no analytic derivation or numerical check confirming that the linear ranking survives at the finite miscalibration sizes illustrated in the NK example.","section":"Abstract; Nakamura-Steinsson (2018) application"},{"comment":"The weakest assumption is that the local linear approximation via derivatives of the target with respect to calibrated parameters accurately reflects the impact of calibration errors. Higher-order terms or curvature could reverse the ranking for the \"sizeable\" miscalibrations the paper considers; no verification of this approximation is reported.","section":"Abstract; Nakamura-Steinsson (2018) application"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the thoughtful and constructive report. The two major comments raise a valid point about the scope of the local approximation. We address them point by point below and commit to revisions that directly respond to the concern.","responses":[{"response":"We agree that the first-order sensitivity statistic delivers an exact bound on bias only in the infinitesimal limit; the paper states this explicitly by referring to 'local bias' and 'local derivatives.' The NK application is intended to illustrate that partitions can differ dramatically in their local sensitivity, not to claim that the local ranking is necessarily preserved for every finite perturbation. In the revision we will add a new subsection that numerically evaluates the actual (non-local) bias for the finite miscalibration magnitudes shown in the application and reports whether the ordering of partitions induced by the sensitivity statistic is preserved.","revision_made":"yes","referee_comment":"[Abstract; Nakamura-Steinsson (2018) application] Abstract and Nakamura-Steinsson application: the central claim is that the partition minimizing the sensitivity statistic also minimizes worst-case local bias from calibration errors. The statistic is constructed from first-order derivatives, so the bound on bias is exact only in the infinitesimal limit. The manuscript provides no analytic derivation or numerical check confirming that the linear ranking survives at the finite miscalibration sizes illustrated in the NK example."},{"response":"The referee is correct that higher-order terms or curvature could, in principle, reverse the local ranking for the finite miscalibrations examined. Because the procedure is deliberately local and derivative-based, it cannot automatically guarantee global robustness. We will therefore include, in the revised manuscript, a direct numerical check within the Nakamura-Steinsson example that compares the realized bias under the reported finite perturbations across the candidate partitions and discusses any discrepancies with the local ranking.","revision_made":"yes","referee_comment":"[Abstract; Nakamura-Steinsson (2018) application] The weakest assumption is that the local linear approximation via derivatives of the target with respect to calibrated parameters accurately reflects the impact of calibration errors. Higher-order terms or curvature could reverse the ranking for the \"sizeable\" miscalibrations the paper considers; no verification of this approximation is reported."}],"tokens_in":1393,"tokens_out":480,"duration_ms":21157,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core idea is to treat the split as a selection problem and choose the partition that minimizes a scalar sensitivity statistic built from the local derivatives of the target with respect to the calibrated parameters.\n\nIt does this in two simple examples first, then applies it to the Nakamura-Steinsson NK model. There the choice of partition turns out to matter a lot: some splits stay reliable even with sizeable miscalibrations while others blow up from small errors. The method only needs derivatives, so it avoids re-estimating the model many times.\n\nWhat is new is the explicit partition-selection rule tied to the target object. That is a practical step beyond the usual convention-driven choices.\n\nThe soft spot is the first-order nature of the statistic. It gives an exact local bound, but the NK application talks about finite miscalibrations. Nothing in the write-up checks whether the ranking of partitions survives once higher-order terms or curvature enter at those error sizes. If the linear approximation does not hold, the selected partition may not actually deliver the smallest bias.\n\nThis is aimed at structural econometricians who run policy or welfare calculations and want a systematic way to defend their calibration choices. A reader who already works with these models will see immediately how to adapt the statistic.\n\nIt is worth sending to referees. The practical payoff is clear even if the local approximation needs more scrutiny in revision.","headline":"The paper gives a clean derivative-based procedure for picking the calibration-estimation split that keeps a target like a policy effect least sensitive to calibration mistakes.","tokens_in":2204,"tokens_out":357,"would_cite":false,"duration_ms":18316,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A sensitivity statistic selects the calibration-estimation split in structural models that minimizes worst-case local bias in targets such as policy effects.","keywords":["calibration","estimation partition","sensitivity statistic","structural models","local bias","New Keynesian model","parameter selection"],"falsifier":"In a Monte Carlo exercise where true parameter values are known, compute the actual bias in the target object under calibrated values drawn from a plausible error distribution and verify whether the partition chosen by the statistic produces smaller bias than the next-best partitions.","tokens_in":2523,"feed_emoji":"","tokens_out":674,"duration_ms":15618,"temperature":0.7,"pith_summary":"Structural models conventionally fix some parameters by calibration and estimate the rest, but the paper treats the split itself as a selection problem. For each admissible partition it builds a scalar sensitivity statistic that records the local response of a chosen target object to small changes in the calibrated parameters. The procedure then keeps the partition whose statistic is smallest, which directly bounds the largest local bias that calibration errors can produce. Only first derivatives are required, so the method avoids repeated full re-estimation and applies to any structural model whose target object can be differentiated with respect to parameters. An application to the Nakamura-Steinsson New Keynesian model shows that the choice of partition can turn a reliable result into one that is highly sensitive to small calibration mistakes.","feed_headline":"Sensitivity statistic picks calibration split that bounds local bias","feed_subtitle":"Minimizing the local response of policy effects and welfare measures to calibrated-parameter errors yields the most robust partition without","key_machinery":"the sensitivity statistic, a scalar that records the local response of the target object to perturbations of the calibrated parameters","core_discovery":"For any structural model and target object the paper defines, for every admissible calibration-estimation partition, a scalar sensitivity statistic equal to the local derivative response of the target to perturbations in the calibrated parameters; the partition that produces the smallest value of this statistic is selected because it minimizes the worst-case local bias that can arise from errors in the calibrated values.","pith_inferences":["The same statistic could be used to rank which parameters are most worth estimating when data are scarce.","If the local linear approximation proves inadequate, the framework could be extended to higher-order or global sensitivity measures.","The approach supplies a quantitative criterion that could be added to existing robustness checks in applied structural work."],"forward_implications":["The procedure applies to any target object whose derivatives with respect to parameters exist, including policy effects, welfare measures, impulse responses, and treatment effects.","In the New Keynesian application some partitions remain reliable under large miscalibrations while others generate large bias from small errors.","The method requires only local derivatives and therefore scales to models where repeated re-estimation would be costly.","Partition choice is shown to have first-order consequences for the credibility of model-based conclusions."],"fun_headline_variants":["Sensitivity statistic selects minimal-bias calibration partition","Local sensitivity guides partition choice to curb calibration bias","Target response statistic picks optimal calibration split","Statistic minimizes worst-case bias from calibrated parameters","Sensitivity identifies robust calibration-estimation partition"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The local linear approximation given by the derivatives of the target with respect to calibrated parameters accurately captures the size of bias that calibration errors induce.","fun_headline_variants_meta":{"raw":{"variants":["Sensitivity statistic selects minimal-bias calibration partition","Local sensitivity guides partition choice to curb calibration bias","Target response statistic picks optimal calibration split","Statistic minimizes worst-case bias from calibrated parameters","Sensitivity identifies robust calibration-estimation partition"]},"model":"grok-4.3","cost_usd":0.002677,"raw_usage":{"total_tokens":1396,"prompt_tokens":597,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":26765500,"prompt_tokens_details":{"text_tokens":597,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":736,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":597,"tokens_out":63,"duration_ms":7098,"temperature":1.0,"reasoning_tokens":736,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-25T19:19:49.048844+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"In a Monte Carlo exercise where true parameter values are known, compute the actual bias in the target object under calibrated values drawn from a plausible error distribution and verify whether the partition chosen by the statistic produces smaller bias than the next-best partitions.","supporting_citations":[],"review_version":1}