{"id":"8728424d-71b4-4be5-b857-566c455fa01e","arxiv_id":"2412.08661","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Adding location-based weights to conformal prediction yields per-location uncertainty intervals; the authors test this on housing and temperature data but the coverage proof is incomplete.","lead":"GeoConformal Prediction adds location-based weights to conformal prediction so each spot on a map gets its own uncertainty interval, instead of one fixed interval. It could help geospatial AI models show where predictions are trustworthy and where they are not.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The coverage guarantee in Eq. (4) rests on Assumption 1, which is not derived and, as written, is not a normalized probability law; the local weighted exchangeability condition is assumed rather than established.","rationale":"The reader's weakest-assumption analysis identifies exactly the load-bearing point: Assumption 1 postulates localized weighted exchangeability rather than deriving it, the bandwidth is unreported, and Eq. (4) overclaims conditional coverage. My stress test agrees and adds a sharper observation: the formula in Assumption 1 is not even a normalized probability statement. For A equal to the full sample space, the RHS is the expectation of a ratio of products to sums of kernel weights, which is generically less than 1 and can be undefined if any weight vanishes. Hence Theorem 2 has no valid premise. This is not a matter of disagreeing with the consensus or approximating an intractable quantity; the proof of the central guarantee is internally incomplete. I still credit the empirical direction: the simulation design, the comparison with bootstrap, and the spatial-interpolation analyses are useful heuristics. The proposed fix would be substantial: state a correct weighted-exchangeability condition, justify that the geographic kernel is the likelihood ratio (or derive coverage under a weaker condition), report bandwidth selection, and temper Eq. (4) to a marginal-coverage statement. Because these changes would affect the main theoretical claim and the headline comparison, the reader's REJECT verdict stands; no adjustment is needed.","tokens_in":20153,"tokens_out":7178,"duration_ms":67994,"concrete_test":"Check the internal consistency of Assumption 1 by instantiating it with n=1 and A equal to the whole sample space: the left-hand side must be 1. If the right-hand side is not identically 1, no data-generating process can satisfy the assumption, and Theorem 2 is vacuous. A corrected version of the assumption should then be tested by simulating a spatial covariate-shift setting with known likelihood ratio q/p and checking whether GeoCP's empirical coverage reaches 1-epsilon only for bandwidths that make the kernel proportional to q/p.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is Eq. (4): conditional on the test location, the GeoCP interval has 1-epsilon coverage. Appendix C attempts to ground this in Assumption 1, but the assumption is doing all the theoretical work and is not derived from Tobler's law or from any spatial data-generating process. As printed, it is not even a valid probability law: for a fixed permutation, the right-hand side is E[(prod_{i=1}^{n+1} w(X_i) / sum_{i=1}^{n+1} w(X_i)) 1_A]. Taking A to be the full sample space gives E[prod_i w_i / sum_i w_i], which is generically less than 1 and is undefined whenever any w_i = 0, so the required probability 1 is not recovered. Thus Theorem 2 cannot be verified from the stated assumption. Even after correcting the formula to the standard weighted-exchangeability condition of Tibshirani et al. (2019), the geographic kernel K(|l_i - l_test| / b) would need to equal the true covariate likelihood ratio for coverage to hold. The bandwidth b is neither estimated nor reported, and no sensitivity analysis is given. The finite-sample coverage guarantee that Eq. (4) advertises is therefore unsupported; the empirical coverage rates do not repair this because they are computed on one data split and cannot validate a conditional guarantee.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GeoConformal Prediction (GeoCP), a split-conformal method that replaces the equally weighted empirical distribution of calibration nonconformity scores with a geographically weighted distribution, using a distance-decay kernel (e.g., Gaussian) between the test location and calibration locations. The authors claim a conditional coverage guarantee in Eq. (4), supported by an appendix theorem under a 'localized weighted exchangeability' assumption. The empirical sections apply GeoCP to a Seattle housing-price regression with XGBoost and to Los Angeles temperature interpolation with Ordinary Kriging and a deep spatial interpolation model, reporting coverage rates, correlations with bootstrap errors, and analyses of how spatial features affect uncertainty. The paper also releases code and data on GitHub.","tokens_in":20503,"tokens_out":3921,"duration_ms":49866,"significance":"If the coverage guarantee were valid, GeoCP would be a useful, model-agnostic tool for spatially varying uncertainty quantification, filling a genuine gap in GeoAI. The simulation framework that constructs biased models with known uncertainty is a thoughtful way to benchmark uncertainty estimates, and the empirical explorations of spatial feature effects on uncertainty are interesting. The paper's strengths include the clear demonstration that original conformal prediction gives constant intervals, the reported negative results for kriging variance as a spatial uncertainty measure, and the public release of code and data. However, the central advertised property, the finite-sample coverage guarantee of Eq. (4), is not established by the manuscript: the stated assumption is not a valid probability law, the theorem is truncated, and the empirical coverage claims rest on an unfair comparison with a confidence interval rather than a prediction interval. These are load-bearing issues for the paper's main contribution.","major_comments":[{"comment":"Assumption 1 as written is not a valid probability law. Taking A to be the full sample space, the right-hand side equals E[∏_{i=1}^{n+1} w(X_{π(i)}) / ∑_{i=1}^{n+1} w(X_{π(i)})], which is generically less than 1 and can be undefined whenever a weight is zero; it does not reduce to P((Z_1,...,Z_{n+1}) ∈ Ω) = 1. Thus the weighted exchangeability condition is not correctly formulated, and Theorem 2 cannot be verified from the stated assumption. Moreover, the sentence 'This holds under covariate shift or when sampling is locally randomized around the test point' is asserted without proof; the assumption is not derived from Tobler's law or from any explicit spatial data-generating process.","section":"Appendix C, Assumption 1"},{"comment":"Theorem 2 is incomplete: the conclusion reads 'P(Y_{n+1} ∈ C(X_{n+1})) ≥ 1 −' with the miscoverage level missing, and no proof is provided. In addition, the theorem as stated would give at best a marginal coverage guarantee, whereas Eq. (4) claims the stronger conditional guarantee P[y_test ∈ C_geo(X_test) | (u_test, v_test)] ≥ 1 − ε. Conditional coverage at each test location is a much stronger property and is not a consequence of the (even corrected) weighted exchangeability result. The central coverage claim of the paper is therefore unsupported.","section":"Appendix C, Theorem 2; Eq. (4)"},{"comment":"The headline comparison between GeoCP and bootstrapping is not apples-to-apples. The main text describes computing bootstrap intervals by taking the 5th and 95th percentiles of 2000 fitted-model predictions at each location, which is a confidence interval for the mean prediction. Appendix A itself notes that prediction intervals are wider than confidence intervals. Although the text says a prediction interval was computed per Appendix B, the reported procedure omits the residual resampling step described there. As a result, the maximum bootstrap coverage of 81.00% is expected to be below the 90% target even if the bootstrap method were working correctly, and the claimed 93.67% versus 81.00% advantage is not a valid demonstration of GeoCP's superiority.","section":"Section on 'Evaluation of coverage ratio' (GeoCP in spatial regression)"},{"comment":"The geographic weights depend on a kernel K and a bandwidth b, but the bandwidth is never estimated, reported, or subjected to sensitivity analysis. The algorithms list a 'decay function β' as input without specifying b. Since the coverage guarantee, if it existed, would require the weights to match the true covariate likelihood ratio, and since the empirical intervals are sensitive to b, the absence of bandwidth reporting makes the experimental results non-reproducible and the claimed guarantee vacuous in practice.","section":"Algorithm 1 and Algorithm 2 (GeoQuantile)"}],"minor_comments":[{"comment":"The text refers to 'boxplot in Figure 8b' when discussing GeoCP in the spatial interpolation section; this appears to be a reference to Figure 12b, and the figure numbering should be corrected.","section":"Section 'GeoCP in spatial interpolation', Figure 12"},{"comment":"The notation Quantile_{1−ε}((1/m)∑δ_{α_i}) is nonstandard; it should be stated explicitly that this is the (1−ε)-quantile of the empirical distribution of calibration scores, and similarly for the weighted empirical distribution in Eq. (6).","section":"Eq. (2) and Eq. (6)"},{"comment":"The abstract uses 'GeoConformal Prediction' while the rest of the paper uses 'GeoCP'; the terminology should be unified throughout.","section":"Introduction/abstract terminology"},{"comment":"When describing 'DGSI with local features', the model uses the average of k-nearest neighbors' target values as an input; the authors should clarify how this is handled to avoid train/test leakage in the interpolation evaluation.","section":"Section 'GeoCP in spatial interpolation'"}],"recommendation":"reject","confidential_remarks":"The empirical study is substantial and the code release is commendable, but the theoretical foundation is the paper's main selling point and it is not sound: the assumption in Appendix C is effectively the conclusion, Theorem 2 is truncated, and the empirical coverage comparison is biased toward GeoCP. These issues are load-bearing and would require a substantial re-derivation (e.g., properly grounding the method in weighted conformal prediction with likelihood-ratio weights and deriving conditions under which a geographic kernel provides valid weights) rather than local edits. If the authors resubmit with a correct coverage theorem, reported bandwidth choices, and a fairer bootstrap comparison, the paper could be suitable for a methods-oriented journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, the empirical core is genuinely useful: GeoCP is weighted split conformal prediction with a geographic kernel, and the authors validate it with a simulation study, a Seattle housing-price regression, and a temperature interpolation task. They also ship code and data. That is real work, and the qualitative finding that local features reduce prediction uncertainty is plausible and well illustrated. Second, the central theoretical claim does not hold as stated. Equation (4) promises conditional coverage given the test location, but the proof in Appendix C rests on Assumption 1, which simply assumes localized weighted exchangeability. That is the exact condition needed to get the result; it is not derived from Tobler's law or from any spatial generative process. Worse, as printed, Assumption 1 is not a normalized probability law: for A equal to the full sample space, the right-hand side is E[prod w_i / sum w_i], which is generically less than 1. Theorem 2 is also truncated mid-sentence, ending at “≥ 1 −”. So the finite-sample coverage guarantee advertised in the abstract is unsupported. This is a load-bearing flaw, not a cosmetic one. The empirical coverage rates do not repair it, since they come from a single data split and cannot validate a conditional guarantee. The bandwidth b is never reported, nor is there sensitivity analysis, so the kernel weighting is effectively a free parameter. The bootstrap comparison is also weakened by the usual confidence-interval versus prediction-interval mismatch, though the authors at least acknowledge the distinction in an appendix. What the paper does well: it frames spatial heterogeneity as covariate shift, connects to Tibshirani et al.’s weighted conformal prediction, and demonstrates a computationally cheap way to get spatially varying intervals. The simulation framework with known uncertainty is a nice contribution on its own. The citation pattern is fine; they engage the relevant conformal and spatial-statistics literature. My take: this is a promising applied paper with a broken theoretical core. It deserves a serious referee, not a desk reject, because the method is a special case of existing weighted conformal prediction and the fix is clear: either correct the assumption, prove the guarantee under explicit and verifiable conditions, or honestly reposition GeoCP as a heuristic and support it with more thorough empirical validation, including bandwidth sensitivity. As written, I would not accept it. But I would send it to review and ask the authors to address the theory head-on.","headline":"A useful empirical package for spatially varying conformal intervals, but the advertised coverage guarantee is assumed rather than proven, so the paper needs major revision before it deserves publication.","tokens_in":20930,"tokens_out":1274,"would_cite":false,"duration_ms":56243,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62M30","62G15","62G05"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that replacing the equal weights of conformal prediction with distance-decayed geographic weights yields prediction intervals with local, location-specific coverage guarantees that hold for any spatial prediction model.","keywords":["GeoConformal Prediction","spatial uncertainty","conformal prediction","spatial prediction","covariate shift","Kriging","GeoAI","prediction interval"],"falsifier":"Run GeoCP on a synthetic field with a sharp spatial discontinuity, such as two distinct regimes separated by a boundary, using the paper's Gaussian weight with a moderate bandwidth; if coverage at test points near the boundary falls substantially below 1 - epsilon, then the localized weighted exchangeability assumption, rather than the underlying predictor, is the failing component.","tokens_in":19971,"feed_emoji":"🗺️","tokens_out":3654,"duration_ms":32940,"temperature":0.7,"pith_summary":"GeoConformal Prediction (GeoCP) claims to fix a blind spot in spatial prediction: most models report accuracy but not trustworthy uncertainty, and standard conformal prediction gives one global interval that ignores location. The paper's proposal is to weight each calibration point's nonconformity score by its geographic proximity to the test point, producing a prediction interval that varies across space while retaining a coverage guarantee. In a housing-price regression, GeoCP reached 93.67% coverage at the 90% level, whereas bootstrapping topped out at 81.00% even after 2000 resamples, with far less computation. The authors also show the uncertainty aligns with Kriging variance in interpolation and that adding explicit local features to a GeoAI model shrinks uncertainty most where local spatial dependence is strong.","feed_headline":"93.67% coverage: geographic weighting upgrades conformal prediction","feed_subtitle":"A model-agnostic wrapper gives every location its own prediction interval; bootstrapping caps out near 81%.","key_machinery":"The carrying object is the geographic weight function w(X_i) = K(|l_i - l_test| / b), a kernel that decays with distance, applied to nonconformity scores in the weighted conformal prediction quantile. The paper uses a Gaussian decay kernel throughout. The geographically weighted quantile (GeoQuantile) computes the (1 - epsilon) quantile of the weighted empirical distribution of calibration scores, and this threshold becomes the local prediction interval radius; Appendix C states a localized weighted exchangeability assumption under which this yields the local coverage theorem.","core_discovery":"The central claim is equation (4): conditional on the test point's location (u_test, v_test), the probability that the true value lies in the GeoCP interval is at least 1 - epsilon. This is achieved by replacing the empirical quantile of calibration nonconformity scores with a geographically weighted quantile, GeoQuantile, where each calibration score is weighted by a kernel of distance to the test point. The authors argue this converts conformal prediction from a marginal, dataset-level guarantee into a local, place-specific one, without modifying the underlying predictor.","pith_inferences":["The coverage guarantee depends on the kernel bandwidth b matching the true covariate shift; because the paper does not estimate or report b, a misspecified bandwidth is the main practical risk to equation (4).","Distance-only weighting may mislead when the distribution shift is driven by features correlated with location but not smoothly distance-decayed, such as a sharp urban-rural boundary.","A natural extension is to calibrate the bandwidth adaptively per test point, for example by cross-validating coverage on held-out spatial data.","The simulation result that recovery degrades as the spatial autocorrelation of true uncertainty approaches zero suggests a floor beyond which no location-based method can reconstruct locally varying uncertainty."],"forward_implications":["Any regression or classification spatial model can be wrapped with GeoCP without retraining or changing its internal structure.","Uncertainty becomes mappable: prediction intervals shrink or grow across space instead of being a single fixed width.","GeoCP enables direct comparison of uncertainty across different spatial models fitted to the same data.","Adding explicit spatial features to a GeoAI model reduces prediction uncertainty, with the largest reductions where local spatial dependence is strongest.","GeoCP uncertainty, as the range of the error distribution at each location, can serve as a basis for measuring geographic bias."],"supporting_citations":[{"why":"Supplies the weighted conformal prediction framework under covariate shift that GeoCP adapts to geographic locations.","marker":"(Tibshirani et al. 2019)"},{"why":"Provides the coverage-validity theory for conformal prediction beyond exchangeability, which the localized weighted theorem extends.","marker":"(Barber et al. 2023)"},{"why":"A prior spatial conformal prediction method; also cited for the empirical finding that Gaussian kernels yield lower interval scores.","marker":"(Mao, Martin, and Reich 2024)"},{"why":"Foundational distribution-free predictive inference results that split conformal prediction builds on.","marker":"(Lei et al. 2018)"},{"why":"Establishes the conformal prediction framework that GeoCP extends with geographic weighting.","marker":"(Shafer and Vovk 2008)"},{"why":"Defines the DGSI deep geometric spatial interpolation model used as the GeoAI baseline in the interpolation experiments.","marker":"(Zhang et al. 2022)"},{"why":"Provides the underlying algorithmic theory of conformal prediction and nonconformity measures.","marker":"(Vovk, Gammerman, and Shafer 2005)"}],"fun_headline_variants":["GeoConformal: local prediction intervals from geographic weighting","93.67% coverage: geographic weighting beats bootstrap's 81%","Model-agnostic spatial uncertainty goes local with GeoConformal","Place-specific conformal prediction: from global to local coverage","GeoCP: geographic weighting upgrades conformal prediction to 93.67%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The coverage guarantee rests on the premise that a location-based kernel weight captures the actual distribution shift from calibration to test; the paper assumes this localized weighted exchangeability and does not estimate or report the bandwidth that would make it true.","fun_headline_variants_meta":{"raw":{"variants":["GeoConformal: local prediction intervals from geographic weighting","93.67% coverage: geographic weighting beats bootstrap's 81%","Model-agnostic spatial uncertainty goes local with GeoConformal","Place-specific conformal prediction: from global to local coverage","GeoCP: geographic weighting upgrades conformal prediction to 93.67%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000802,"raw_usage":{"total_tokens":3535,"prompt_tokens":963,"completion_tokens":2572,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":2484}},"tokens_in":579,"tokens_out":2572,"duration_ms":21033,"temperature":1.0,"reasoning_tokens":2484,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:24:03.378399+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GeoCP on a synthetic field with a sharp spatial discontinuity, such as two distinct regimes separated by a boundary, using the paper's Gaussian weight with a moderate bandwidth; if coverage at test points near the boundary falls substantially below 1 - epsilon, then the localized weighted exchangeability assumption, rather than the underlying predictor, is the failing component.","supporting_citations":[],"review_version":1}