The reviewed record of science sign in
Pith

arxiv: 2606.23453 · v1 · pith:LAZ3XDKD · submitted 2026-06-22 · cs.LG

Do Location Encoders Capture Spatial Effects? A GeoShapley Benchmark Across Scales

Daniel Kiv , Shaowen Wang This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-26 09:17 UTCgrok-4.3pith:LAZ3XDKDrecord.jsonopen to challenge →

classification cs.LG
keywords location encodersGeoShapleyspatial effectsgeographic embeddingsmachine learning interpretabilitysynthetic benchmarksscale dependencecoefficient recovery
0
0 comments X

The pith

Location encoders recover primary spatially varying coefficients consistently across scales, but secondary recovery is scale-dependent and raw coordinates remain competitive.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper evaluates whether location encoders produce embeddings that capture interpretable spatial effects by applying GeoShapley explanations to models trained on synthetic data with known coefficients. It measures recovery success as the correlation between explained and true coefficients, testing eleven encoders at grid, county, and global scales under trained and untrained conditions, with and without raw coordinates. Recovery of the main coefficient stays high for all encoders, while the secondary coefficient shows greater variation especially at the global scale. The simple raw-coordinate baseline matches or approaches the performance of the more complex embeddings. This setup directly tests whether the embeddings add value beyond coordinates for recovering spatial structure.

Core claim

Using a synthetic data-generating process with known spatially varying coefficients, the benchmark shows that GeoShapley attributions recover the primary coefficient at high correlation levels across all eleven location encoders and scales. Recovery of the secondary coefficient is lower and more sensitive to scale, with the largest differences appearing at the global scale. The raw-coordinate baseline produces comparable recovery rates throughout, including under both untrained and contrastively trained encoder conditions.

What carries the argument

GeoShapley, a game-theoretic explainer that treats all location features as a single joint player to attribute model predictions and recover the underlying spatially varying coefficients via correlation with ground truth.

If this is right

  • Primary spatial effects can be extracted reliably from encoder-based models at grid, county, and global scales.
  • Secondary spatial effects require explicit scale checks, particularly when operating at global extents.
  • Raw geographic coordinates serve as a competitive baseline that often matches embedding performance for coefficient recovery.
  • Both untrained and contrastively trained encoders yield similar recovery patterns in the benchmark.
  • The joint treatment of location features in the explainer enables direct comparison of embedding utility against simpler inputs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Practitioners may default to raw coordinates for many spatial tasks unless a specific encoder demonstrates clear gains on secondary effects at the target scale.
  • The benchmark approach could be extended to test recovery when multiple interacting spatial processes are present simultaneously.
  • If real-world spatial variation deviates from the synthetic process, validation against held-out data with partial ground truth would be needed before trusting attributions.
  • This suggests potential value in hybrid models that combine embeddings with explicit coordinate terms when secondary effects matter.

Load-bearing premise

The synthetic data-generating process with known spatially varying coefficients is representative of the spatial effects that location encoders are intended to capture in real applications.

What would settle it

Running the same GeoShapley recovery test on a real geographic dataset where independent estimates of the true spatially varying coefficients can be obtained from domain knowledge or additional measurements, then comparing the resulting correlations to the synthetic benchmark results.

Figures

Figures reproduced from arXiv: 2606.23453 by Daniel Kiv, Shaowen Wang.

Figure 1
Figure 1. Figure 1: 𝛽2 recovery (Pearson 𝑟) per encoder across the three scales and the baseline (BL), embedding+coordinates (EC), and embedding-only (EO) conditions defined in the panel (MLP, untrained). ∗ marks values significantly above the coords-only baseline (paired Wilcoxon, 𝑝 < 0.05); 𝛽1 (𝑟 > 0.98 in nearly all conditions) is omitted ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Global 𝛽2 surfaces for representative encoders (top row: ground truth): the coordinates-only baseline, and Sphere2Vec￾dfs, Space2Vec-grid, and tile_ffn in the embedding-only condition [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
read the original abstract

Location encoders transform geographic coordinates into high dimensional embeddings for downstream machine learning, but it is unclear how well these representations capture interpretable spatial effects. We benchmark whether GeoShapley, a game-theoretic explainer that treats all location features as a single joint player, can recover spatially varying coefficients from models built on location-encoder embeddings. Eleven encoders from the TorchSpatial framework are evaluated against a synthetic process with known coefficients, across three scales (grid, county, global), with and without raw coordinates alongside the embedding, and under untrained and contrastively trained conditions. Measuring recovery as the correlation between estimated and true coefficients, we report how it varies with scale and encoder architecture and compare the embeddings against a raw-coordinate baseline. Recovery of the primary coefficient is consistently high across encoders, whereas recovery of a secondary coefficient is more scale-dependent, differing most at the global scale; the raw-coordinate baseline remains competitive throughout.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims to benchmark whether location encoders capture spatial effects by using GeoShapley to recover known spatially varying coefficients from models trained on embeddings. Eleven encoders from TorchSpatial are tested on a synthetic process across grid, county, and global scales, with and without raw coordinates and under untrained/contrastively trained conditions; recovery is measured by correlation between estimated and true coefficients. The headline results are consistently high recovery of the primary coefficient across encoders, more scale-dependent recovery of the secondary coefficient (differing most at global scale), and competitive performance from the raw-coordinate baseline.

Significance. If the results hold, the work supplies a reproducible empirical protocol for probing the spatial interpretability of location embeddings via game-theoretic attribution. The multi-scale design, explicit comparison to a raw-coordinate baseline, and evaluation under both untrained and trained regimes are concrete strengths that could guide practical encoder choice in geospatial tasks. The significance is limited by dependence on a single synthetic DGP whose structure must match the spatial heterogeneity encoders are intended to model.

major comments (2)
  1. [Methods] Methods section (synthetic data generation): the abstract and experimental description provide no functional form for the spatially varying coefficients, no specification of how scale is instantiated in the DGP, and no justification for why this particular process was chosen. Because all recovery correlations are measured exclusively on this DGP, its representativeness is load-bearing for any claim that the benchmark informs what encoders capture in real applications.
  2. [Results] Results section: the manuscript states that recovery 'is consistently high' and 'more scale-dependent' but supplies neither the precise correlation formula used, confidence intervals, nor any statistical test for differences across encoders or scales. Without these, the quantitative support for the primary vs. secondary and scale-dependent claims cannot be verified.
minor comments (2)
  1. Add a brief description or reference table for the eleven TorchSpatial encoders so that readers can map architectural differences to the reported recovery patterns.
  2. Clarify in the figure captions or text whether the reported correlations are computed per coefficient, per location, or aggregated, and whether any regularization or post-processing is applied to the GeoShapley values.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major comment below and will make the requested revisions to improve clarity and rigor.

read point-by-point responses

  1. Referee: [Methods] Methods section (synthetic data generation): the abstract and experimental description provide no functional form for the spatially varying coefficients, no specification of how scale is instantiated in the DGP, and no justification for why this particular process was chosen. Because all recovery correlations are measured exclusively on this DGP, its representativeness is load-bearing for any claim that the benchmark informs what encoders capture in real applications.

    Authors: We agree that the functional form of the spatially varying coefficients, the precise instantiation of scales in the DGP, and the justification for selecting this process must be stated explicitly, as the benchmark's value depends on the DGP's representativeness. Although these details appear in the full methods, they were insufficiently highlighted in the abstract and experimental description. In revision we will add the exact equations for the primary and secondary coefficients, describe how grid, county, and global scales are operationalized (coordinate ranges, sampling, boundaries), and justify the DGP by reference to standard spatial heterogeneity models used in geospatial ML. revision: yes

  2. Referee: [Results] Results section: the manuscript states that recovery 'is consistently high' and 'more scale-dependent' but supplies neither the precise correlation formula used, confidence intervals, nor any statistical test for differences across encoders or scales. Without these, the quantitative support for the primary vs. secondary and scale-dependent claims cannot be verified.

    Authors: We agree that the results lack the necessary statistical details. We will revise to define the recovery metric explicitly as the Pearson correlation between estimated and true coefficients, add bootstrap confidence intervals, and include statistical tests (e.g., paired comparisons or ANOVA) for differences across encoders and scales. These changes will supply verifiable quantitative support for the claims on primary versus secondary recovery and scale dependence. revision: yes

Circularity Check

0 steps flagged

Empirical benchmark with no derivation chain or self-referential reduction

full rationale

The paper conducts an empirical benchmark: it generates synthetic data with known spatially varying coefficients, trains models on location-encoder embeddings (plus raw-coordinate baseline), applies GeoShapley to recover coefficients, and reports Pearson correlations between estimated and true values across scales and encoders. No equations derive a quantity from itself, no fitted parameter is relabeled as a prediction, and no uniqueness theorem or ansatz is imported via self-citation to force the result. The central claims are direct measurements on the synthetic process; the representativeness of that process for real data is an external-validity question, not a circularity in the reported recovery statistics.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper is an empirical benchmark and introduces no free parameters, domain axioms, or invented entities beyond standard supervised learning assumptions.

pith-pipeline@v0.9.1-grok · 5682 in / 1039 out tokens · 22958 ms · 2026-06-26T09:17:27.832808+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 16 canonical work pages

  1. [1]

    1988.Spatial Econometrics: Methods and Models

    Luc Anselin. 1988.Spatial Econometrics: Methods and Models. Studies in Operational Regional Science, Vol

    1988

  2. [2]

    doi:10.1007/978-94-015- 7799-1 Oisin Mac Aodha, Elijah Cole, and Pietro Perona

    Springer Netherlands, Dordrecht. doi:10.1007/978-94-015- 7799-1 Oisin Mac Aodha, Elijah Cole, and Pietro Perona

    work page doi:10.1007/978-94-015-

  3. [3]

    In2019 IEEE/CVF International Conference on Computer Vision (ICCV)

    Presence-Only Geographi- cal Priors for Fine-Grained Image Classification. In2019 IEEE/CVF International Conference on Computer Vision (ICCV). 9595–9605. doi:10.1109/ICCV.2019.00969 Daniel Cai and Randall Balestriero

    work page doi:10.1109/iccv.2019.00969 2019

  4. [4]

    arXiv:2502.06831 [cs] doi:10.48550/arXiv.2502.06831 Ian Covert and Su-In Lee

    No Location Left Behind: Measur- ing and Improving the Fairness of Implicit Representations for Earth Data. arXiv:2502.06831 [cs] doi:10.48550/arXiv.2502.06831 Ian Covert and Su-In Lee

    work page doi:10.48550/arxiv.2502.06831

  5. [5]

    2017), 1247–1265

    Multiscale Geographically Weighted Regression (MGWR).Annals of the American Association of Geographers 107, 6 (Nov. 2017), 1247–1265. doi:10.1080/24694452.2017.1352480 Chris Harris, Richard Pymar, and Colin Rowat

    work page doi:10.1080/24694452.2017.1352480 2017

  6. [6]

    arXiv:2107.11357 [stat.ML] doi:10.48550/arXiv.2107.11357 Ziqi Li

    Joint Shapley Values: A Measure of Joint Feature Importance. arXiv:2107.11357 [stat.ML] doi:10.48550/arXiv.2107.11357 Ziqi Li

    work page doi:10.48550/arxiv.2107.11357

  7. [7]

    2022), 101845

    Extracting Spatial Effects from Machine Learning Model Using Local Interpretation Method: An Example of SHAP and XGBoost.Computers, Environment and Urban Systems96 (Sept. 2022), 101845. doi:10.1016/j.compenvurbsys.2022. 101845 Ziqi Li

    work page doi:10.1016/j.compenvurbsys.2022 2022

  8. [8]

    Hou et al

    GeoShapley: A Game Theory Approach to Measuring Spatial Effects in Machine Learning Models.Annals of the American Association of Geographers114, 7 (Aug. 2024), 1365–1385. doi:10.1080/24694452.2024.2350982 Ziqi Li and Zhan Peng

    work page doi:10.1080/24694452.2024.2350982 2024

  9. [9]

    2025), 602–615

    Can Moran Eigenvectors Improve Machine Learning of Spatial Data? Insights From Synthetic Data Validation.Geographical Analysis57, 4 (Oct. 2025), 602–615. doi:10.1111/gean.70011 Lingbo Liu

    work page doi:10.1111/gean.70011 2025

  10. [10]

    2024), 104036

    An Ensemble Framework for Explainable Geospatial Machine Learn- ing Models.International Journal of Applied Earth Observation and Geoinformation 132 (Aug. 2024), 104036. doi:10.1016/j.jag.2024.104036 Gengchen Mai, Krzysztof Janowicz, Yingjie Hu, Song Gao, Bo Yan, Rui Zhu, Ling Cai, and Ni Lao

    work page doi:10.1016/j.jag.2024.104036 2024

  11. [11]

    doi:10.1080/13658816.2021.2004602 Gengchen Mai, Krzysztof Janowicz, Bo Yan, Rui Zhu, Ling Cai, and Ni Lao

    A Review of Location Encoding for GeoAI: Methods and Applications.International Journal of Geographical Information Science36, 4 (April 2022), 639–673. doi:10.1080/13658816.2021.2004602 Gengchen Mai, Krzysztof Janowicz, Bo Yan, Rui Zhu, Ling Cai, and Ni Lao

    work page doi:10.1080/13658816.2021.2004602 2022

  12. [12]

    doi:10.1016/j.isprsjprs.2023.06.016 Ben Mildenhall, Pratul P

    Sphere2Vec: A General-Purpose Location Representation Learning over a Spherical Surface for Large-Scale Geospatial Pre- dictions.ISPRS Journal of Photogrammetry and Remote Sensing202 (2023), 439–462. doi:10.1016/j.isprsjprs.2023.06.016 Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ra- mamoorthi, and Ren Ng

    work page doi:10.1016/j.isprsjprs.2023.06.016 2023

  13. [13]

    Srinivasan, Matthew Tancik, Jonathan T

    NeRF.Commun. ACM65, 1 (Dec. 2021), 99–106. doi:10.1145/3503250 F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay

    work page doi:10.1145/3503250 2021

  14. [14]

    Ali Rahimi and Benjamin Recht

    Scikit-Learn: Machine Learning in Python.Journal of Machine Learning Research12 (2011), 2825–2830. Ali Rahimi and Benjamin Recht

    2011

  15. [15]

    arXiv:2511.02101 [cs.LG] doi:10

    Measuring the Intrinsic Dimension of Earth Representations. arXiv:2511.02101 [cs.LG] doi:10. 48550/arXiv.2511.02101 Marc Rußwurm, Konstantin Klemmer, Esther Rolf, Robin Zbinden, and Devis Tuia

    work page arXiv

  16. [16]

    Noam Shazeer

    Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks. arXiv:2310.06743 [cs] doi:10.48550/arXiv.2310.06743 Nripsuta Ani Saxena, Wenbin Zhang, and Cyrus Shahabi

    work page doi:10.48550/arxiv.2310.06743

  17. [17]

    arXiv:2403.14040 [cs] doi:10.48550/arXiv.2403.14040 Kevin Tang, Manohar Paluri, Li Fei-Fei, Rob Fergus, and Lubomir Bourdev

    Spatial Fairness: The Case for Its Importance, Limitations of Existing Work, and Guidelines for Future Research. arXiv:2403.14040 [cs] doi:10.48550/arXiv.2403.14040 Kevin Tang, Manohar Paluri, Li Fei-Fei, Rob Fergus, and Lubomir Bourdev

    work page doi:10.48550/arxiv.2403.14040

  18. [18]

    InProceedings of the 2015 IEEE International Conference on Computer Vision (ICCV) (ICCV ’15)

    Improving Image Classification with Location Context. InProceedings of the 2015 IEEE International Conference on Computer Vision (ICCV) (ICCV ’15). IEEE Computer Society, USA, 1008–1016. Nemin Wu, Qian Cao, Zhangyu Wang, Zeping Liu, Yanlin Qi, Jielu Zhang, Joshua Ni, Xiaobai Yao, Hongxu Ma, Lan Mu, Stefano Ermon, Tanuja Ganu, Akshay Nambi, Ni Lao, and Gen...

    2015

  19. [19]

    arXiv:2406.15658 [cs] doi:10

    TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning. arXiv:2406.15658 [cs] doi:10. 48550/arXiv.2406.15658

    work page arXiv