{"id":"c29af68c-ab35-490e-8216-872d37ef71db","arxiv_id":"2505.03368","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LLM internal representations of placenames show measurable spatial autocorrelation, and sparse autoencoders extract a small number of geospatially interpretable features.","lead":"This chapter introduces a way to inspect what a large language model knows about places by looking at its internal neuron activations and using spatial statistics to find geographic patterns. It applies this to placenames in the UK, Italy, and the US, finding that some internal representations are geographically structured.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The geospatial claim is confounded by administrative region tokens repeated in every prompt; without a no-qualifier or mismatched-region control, spatial autocorrelation may reflect token identity rather than geographic location.","rationale":"The reader's weakest-assumption analysis correctly identifies the administrative-token confound as the central threat to the paper's claim. My reading of the full text confirms that the prompt design in Section 2.3.1 embeds a repeated region token in every input, and the paper's own Figure 2(a) shows a neuron whose pattern matches those tokens. The authors acknowledge the issue in words but do not provide a quantitative control, so the headline result is ambiguous between two interpretations: (a) the model encodes genuine geographic structure associated with placenames, or (b) neurons are simply sensitive to the repeated 'England', 'Scotland', 'New York', or province tokens. The latter is a trivial and less interesting mechanism, and it would not justify the conclusion that the model represents spatial relationships among places. The proposed control—prompts without the qualifier, or with deliberately mismatched qualifiers—would directly separate these possibilities. I do not think the paper should be rejected, because the framework and exploratory analyses are valuable and the code is promised to be available; however, the central claim should remain conditional until the control is run. Therefore the reader's CONDITIONAL verdict stands unchanged.","tokens_in":13596,"tokens_out":2006,"duration_ms":24160,"concrete_test":"Recompute the global and local Moran's I analysis for the same model, layers, and placenames using two control prompt conditions: (1) '[placename]' with no administrative qualifier, and (2) '[placename], [randomly assigned region]' where region is drawn from the same set but mismatched to the placename's true location. If the percentage of neurons with Moran's I >= 0.3 at p < .01 drops to near the nominal false-positive rate, or if the spatial clusters disappear, the autocorrelation is driven by the administrative token rather than geographic location. If significant autocorrelation persists in condition (1) and vanishes only when the qualifier is mismatched, the geospatial interpretation is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that internal representations of placenames display spatial patterns 'related to their geographic location.' The experimental prompts in Section 2.3.1 always append an administrative qualifier: '[placename], [country/province/state]'. This means every placename in England is presented with the token 'England', every placename in New York State with 'New York', and so on. A neuron that simply tracks the administrative token will therefore produce identical activations for all placenames in the same admin area, and spatial autocorrelation will follow by construction. The paper explicitly acknowledges this risk in Section 2.3.1, and Figure 2(a) even displays a neuron whose high values align with the admin-area names included in prompts. Yet no control condition is run to separate the contribution of the placename's geographic location from the contribution of the repeated administrative token. The reported statistic—1,841 neurons (14.98%) with Moran's I >= 0.3 at p < .01—is compared only to an implicit chance level, not to a token-identity baseline. Because the central interpretation depends on the activations encoding geographic structure beyond the administrative labels, this confound is load-bearing: if the effect survives only when the admin token is present, the conclusion that LLMs 'think geographically' about placenames is not supported by the presented evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes a framework for geospatial mechanistic interpretability of large language models. The authors extract internal activations from Mistral-7B-Instruct-v0.2 for placenames in the UK, Italy, and four US states, using prompts that append a country/province/state qualifier. They compute global and local Moran's I on per-neuron activations and report that 1,841 of 12,288 neurons (14.98%) show significant spatial autocorrelation. They then train a sparse autoencoder on layer-15 activations to decompose the representations into 32,768 features, of which 67 are reported to show significant spatial autocorrelation, and present examples of features that appear to correspond to regions such as Wales, southern Italy, and 'city'. The central claim is that these results show LLMs encode geographic information in a spatially structured, largely polysemantic way, and that sparse autoencoders can disentangle this structure into more monosemantic features.","tokens_in":13811,"tokens_out":9438,"duration_ms":86338,"significance":"If the claims were supported, the paper would offer a novel methodological bridge between geographic information science and mechanistic interpretability, providing a concrete toolset (Moran's I on internal activations, SAE decomposition) applicable to questions about bias, diversity, and knowledge representation in LLMs. The paper is clearly written, situates the work well in both literatures, and makes code and data available. However, the significance is conditional: the central geospatial interpretation is confounded by the administrative tokens always present in the prompts, and the SAE statistical summary is misleading. With appropriate controls and corrected statistics, the contribution could be solid and of interest to the geoAI and interpretability communities.","major_comments":[{"comment":"The prompt design confounds geographic location with administrative-token identity. Every prompt is of the form '[placename], [country/province/state]', and the reported activations are mean-pooled across all tokens in the prompt (Section 2.3.1). Consequently, a neuron that simply responds to the token 'England' will produce near-identical activations for all English placenames, yielding spatial autocorrelation that mirrors administrative boundaries by construction. The manuscript acknowledges this risk explicitly (Section 2.3.1) but never controls for it; Figure 2(a) even displays a neuron whose high values align with the admin-area tokens in the prompts. To support the claim that the spatial patterns are 'related to their geographic location' (abstract), the authors should run a control condition, e.g., prompts with the qualifier omitted, prompts with mismatched qualifiers (e.g., 'Liverpool, California'), or a neuron-level baseline obtained by scrambling the qualifier across places. Without such a control, the 1,841-neuron statistic cannot distinguish encoding of geographic location from encoding of token identity.","section":"Section 2.3.1 (prompt design); Section 2.3.2 (results)"},{"comment":"The statistical interpretation is problematic in two ways. First, the 1,841 significant neurons out of 12,288 are assessed at p < .01 without any multiple-comparison correction or permutation test; at that threshold roughly 123 false positives are expected, and the paper does not report how many neurons survive a corrected threshold, so the strength of the rejection of the a-spatial null is not quantified. Second, the SAE result is mis-stated: the paper reports 'only 67 of the 32,768 features (0.2%) displayed a significant spatial autocorrelation', but 99.53% of features are always zero (Section 3.2.2), so only about 154 features are active. The meaningful rate is therefore 67/154 ≈ 43%, far above the ~1.5 false positives expected among active features at p < .01. The paper's conclusion that geospatial information is 'sparse and diffuse' is the opposite of what this calculation suggests; the correct denominator should be active features, not all 32,768.","section":"Section 2.3.2 and Section 3.2.2 (statistical summarization)"},{"comment":"The sparse autoencoder is trained on the full set of pooled activations, and k = 2,048 is selected based on the final training loss on that same dataset. The subsequent Moran's I analysis is then performed on the very representations used for training, with no held-out split, seed averaging, or stability analysis. This does not make the spatial analysis circular (the SAE objective is reconstruction, not spatial autocorrelation), but it does mean the reported counts of spatially autocorrelated features are not protected against overfitting to the training representations, and the quantitative claims may not generalize. The paper should either validate the SAE on a held-out set or report that the results are stable across random initializations.","section":"Section 3.2.1 (SAE training and selection)"}],"minor_comments":[{"comment":"The spatial weights matrix used for the global and local Moran's I (e.g., k-nearest neighbors or distance band, row standardization) is not specified, which makes the reported p-values and Moran's I values impossible to verify from the text alone.","section":"Section 2.3.1 (methodology)"},{"comment":"The maps in Figures 2 and 3 would be easier to interpret with color bars, explicit layer/neuron or feature labels, and the corresponding Moran's I values for the displayed examples.","section":"Figures 2 and 3"},{"comment":"Training hyperparameters for the sparse autoencoder (learning rate, optimizer, batch size, normalization) are not reported beyond the 300 epochs and k values; these details should be given or referenced to the code repository.","section":"Section 3.2.1 (training details)"},{"comment":"The phrase 'an a-spatial manner' is awkward; consider 'aspatial' or 'non-spatial' for clarity.","section":"Section 1"},{"comment":"Reference [65] is cited for 'a new research question'; the intended noun is likely 'questions'.","section":"Section 4"},{"comment":"The interpretation of the feature in Figure 3(c) as relating to ports and industrial development is speculative and not backed by a validation experiment; it should be flagged as such in the text.","section":"Section 3.2.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a book chapter and is accordingly exploratory, but the central empirical claim is stated as a general finding. For journal-level acceptance, the administrative-token confound must be addressed; the authors have the code and the prompt templates, so a control condition is entirely feasible. Note that the corrected SAE statistic (43% of active features significant) would strengthen the paper, so the revision is within scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the paper does something genuinely new: it brings spatial autocorrelation (Moran's I) to bear on LLM internal activations and sparse autoencoder features for placenames, asking whether nearby places produce similar internal states. That is a reasonable way to look for geographic structure. Second, the headline empirical result—that 14.98% of neurons show significant spatial autocorrelation—is not trustworthy in the form presented, because every prompt appends the administrative region name ('[placename], England', '[placename], New York', etc.). A neuron that simply tracks that token will produce identical activations for all places in the same region, and Moran's I will follow by construction. The authors explicitly mention this risk but never control for it. Given that one of their own example neurons (Figure 2a) aligns with the admin-area names, the confound is load-bearing, not a side concern.\n\nWhere they do well: the framework is clearly explained, the chapter gives a solid overview of probing and mechanistic interpretability, and they are honest that these are preliminary experiments. The code and data are promised on GitHub, which is good practice. The observation that geographic information seems distributed across polysemantic neurons, with a few somewhat cleaner features after SAE, is worth exploring further.\n\nThe soft spots are real. Besides the prompt confound, there is no multiple-comparison correction across the tens of thousands of neurons tested, so a chunk of the 14.98% is likely false positives. The SAE training picks k based on training loss on the full dataset, and 99.53% of features are always zero—a sign the SAE is not really doing useful decomposition. They acknowledge this but don't connect it to the validity of the geospatial features they then interpret. The 'monosemantic' features are judged visually, not quantitatively.\n\nThe bottom line: the framework deserves to be developed, and the paper is worth discussing in a reading group for its approach and its flaws. But the central claim that LLMs 'think geographically' about placenames is not supported by the evidence as analyzed. A control condition (e.g., no qualifiers, or swapped region labels) plus proper multiple-testing control would tell you if there is a real effect. Yes, I'd send it to peer review—the idea is worth referee time, and the confound is fixable in revision.","headline":"A fresh framework for spatial interpretability of LLMs, but the main empirical claim is confounded by the always-present admin region token in the prompts.","tokens_in":14354,"tokens_out":2809,"would_cite":false,"duration_ms":28280,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Spatial autocorrelation exposes geospatial structure inside LLM place-name representations.","keywords":["mechanistic interpretability","geospatial analysis","spatial autocorrelation","sparse autoencoders","large language models","internal representations","geospatial AI","probing"],"falsifier":"Take the same placenames and run the identical spatial-autocorrelation analysis with prompts that omit the country, province, or state suffix, or with those administrative-region tokens removed from the activation vector; if the clustered spatial patterns (Moran's I >= 0.3) disappear or drop sharply, the spatial structure is an artifact of the shared region token rather than evidence of encoded geographic knowledge.","tokens_in":13367,"feed_emoji":"🗺️","tokens_out":5813,"duration_ms":52234,"temperature":0.7,"pith_summary":"This paper tries to establish that the internal representations a large language model forms for place names are geographically structured, and that this structure can be reverse-engineered with spatial statistics. Using spatial autocorrelation on post-attention activations, it argues that nearby places evoke similar internal patterns, while distant or different kinds of places diverge. It further claims that most geographic encoding is polysemantic, meaning spread across many neurons that each encode several concepts, and that sparse autoencoders can disentangle a small number of monosemantic, spatially interpretable features such as 'southern Italy' and 'city'. If right, this gives geography a window into the model's hidden geography: a way to inspect, rather than merely query, what LLMs know about place.","feed_headline":"LLM neurons reveal geographic patterns for place names","feed_subtitle":"Internal activations cluster by location, and sparse autoencoders isolate features like 'southern Italy' or 'big city'.","key_machinery":"The central mechanism is spatial autocorrelation, measured with global and local Moran's I, applied to vectors that summarize the model's state: 4,096 post-attention, mean-pooled activations per layer for each placename, and then 32,768 sparse features obtained from a sparse autoencoder with a TopK/ReLU activation that retains the 2,048 largest embeddings per input. Autocorrelation tests whether high values for one placename tend to be surrounded by high values for nearby placenames; local Moran's I then maps significant high-high and low-low clusters. The sparse autoencoder is the disentangling instrument: it projects the 4,096-dimensional activation space into a wider, sparsely activated feature space in which individual directions can become interpretable, and the same autocorrelation test is applied to those feature directions to see whether they are geospatially coherent. The whole argument thus rides on Moran's I as a way to turn a causal-sounding question, 'does the model encode geography?', into a measurable spatial-statistical one.","core_discovery":"The paper claims that internal activations evoked by placenames are not geographically inert: neurons and sparse-autoencoder features carrying high values for nearby places cluster in space, consistent with Tobler's first law operating inside the model. Across layers 7, 15, and 31 of a 7-billion-parameter instruction-tuned model, about 15% of neurons show significant spatial autocorrelation (p < .01, Moran's I >= 0.3), with individual neurons activating for multiple regions at once, such as New York State and Northern Ireland, which supports a polysemantic, superposition-style encoding. A sparse autoencoder trained on layer-15 activations decomposes these into 32,768 sparse features, of which 0.2% show significant spatial autocorrelation; some of these are clearly monosemantic, such as a feature for southern Italy that does not fire for northern Italy, the UK, or the US, and a 'city' feature firing for London, Manchester, Milan, New York, and Philadelphia. The small count of spatially coherent features is read as evidence that geographic information is distributed sparsely and diffusely across the representation, entangled with non-geographic concepts.","pith_inferences":["A natural control experiment follows from the paper's own setup: hold the administrative suffix constant but shuffle placenames across coordinates, or drop the suffix entirely; if autocorrelation disappears, the spatial signal lives in the token, not in the place.","The same spatial-autocorrelation test could be applied to features from larger models, multilingual models, or different sparse-autoencoder variants to map how geographic scale and cultural context shift the features that fire.","The 'city' feature hints at a path toward geospatial feature atlases, dictionaries that connect model-internal directions to regions, urbanity, and other geographic concepts, which could make model geographies auditable for bias and diversity.","The paper's observation of a feature shared by northeastern Italy and northwestern England suggests feature-level analysis could reveal non-cartographic, relational geographies that coordinate probes miss."],"forward_implications":["Placenames that are geographic neighbors tend to have similar internal representations, so geographic structure is present in LLM hidden states, not only in outputs.","Geographic knowledge is mostly distributed across polysemantic neurons; single-neuron readings will mislead, and superposition is a real obstacle for geographic interpretability.","Sparse autoencoders can recover a small number of monosemantic geographic features such as 'southern Italy' and 'city', suggesting that disentangled geographic directions exist in representation space.","Because only 0.2% of sparse features show spatial autocorrelation, geographic content is encoded sparsely and diffusely; future work must improve feature extraction to surface more of it.","The framework opens a research path on geospatial mechanistic interpretability for foundation models beyond the specific model and three test regions."],"supporting_citations":[{"why":"Supplies the motivating principle, Tobler's first law, that nearby places should have similar internal representations.","marker":"[35]"},{"why":"Establishes the probing baseline linking city- and country-level representations to coordinates, population, and neighbours.","marker":"[36]"},{"why":"Shows linear probes can predict geographic coordinates from LLM internal representations, grounding the coordinate-probing expectation.","marker":"[37]"},{"why":"Extends coordinate probing across model scales and connects representation quality to geographic content in training data.","marker":"[38]"},{"why":"Provides non-linear probing of spatial representations and links internal-representation quality to geospatial task performance.","marker":"[39]"},{"why":"Supplies the sparse-autoencoder scaling methodology and monosemantic feature extraction approach the experiments adapt.","marker":"[32]"},{"why":"Introduces dictionary-learning sparse autoencoders as the tool for decomposing polysemantic representations.","marker":"[62]"},{"why":"Proposes the superposition hypothesis that motivates interpreting polysemantic neurons as superimposed features.","marker":"[57]"},{"why":"Provides the TopK/ReLU activation used to train the sparse autoencoder with controlled sparsity.","marker":"[64]"}],"fun_headline_variants":["LLMs map places internally: neurons cluster by location","Sparse features expose LLM's mental map of geography","Inside LLMs: place-name activations show spatial order","LLM neurons reveal a hidden geographic coordinate system"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The analysis assumes the spatial autocorrelation in activations reflects the model's geographic knowledge rather than the administrative region name (for example 'England', a province, or a US state) that is appended to every placename in the prompt; because every placename from the same region shares that token, a neuron could cluster spatially simply by tracking that token, and the paper does not test this alternative.","fun_headline_variants_meta":{"raw":{"variants":["LLMs map places internally: neurons cluster by location","Sparse features expose LLM's mental map of geography","Inside LLMs: place-name activations show spatial order","LLM neurons reveal a hidden geographic coordinate system"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000576,"raw_usage":{"total_tokens":2773,"prompt_tokens":1057,"completion_tokens":1716,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":673,"completion_tokens_details":{"reasoning_tokens":1652}},"tokens_in":673,"tokens_out":1716,"duration_ms":11967,"temperature":1.0,"reasoning_tokens":1652,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:52:40.641162+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same placenames and run the identical spatial-autocorrelation analysis with prompts that omit the country, province, or state suffix, or with those administrative-region tokens removed from the activation vector; if the clustered spatial patterns (Moran's I >= 0.3) disappear or drop sharply, the spatial structure is an artifact of the shared region token rather than evidence of encoded geographic knowledge.","supporting_citations":[{"cited_title":"A Computer Movie Simulating Urban Growth in the Detroit Region","cited_arxiv_id":null,"evidence_quote":"Supplies the motivating principle, Tobler's first law, that nearby places should have similar internal representations."},{"cited_title":"Do Language Models Know the Way to Rome? arXiv preprint arXiv:210907971","cited_arxiv_id":null,"evidence_quote":"Establishes the probing baseline linking city- and country-level representations to coordinates, population, and neighbours."},{"cited_title":"Language Models Represent Space and Time","cited_arxiv_id":null,"evidence_quote":"Shows linear probes can predict geographic coordinates from LLM internal representations, grounding the coordinate-probing expectation."},{"cited_title":"On the Scaling Laws of Geographical Representation in Language Models","cited_arxiv_id":null,"evidence_quote":"Extends coordinate probing across model scales and connects representation quality to geographic content in training data."},{"cited_title":"Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet","cited_arxiv_id":null,"evidence_quote":"Supplies the sparse-autoencoder scaling methodology and monosemantic feature extraction approach the experiments adapt."},{"cited_title":"Towards Monosemantic- ity: Decomposing Language Models With Dictionary Learning","cited_arxiv_id":null,"evidence_quote":"Introduces dictionary-learning sparse autoencoders as the tool for decomposing polysemantic representations."},{"cited_title":"Toy Models of Superposition","cited_arxiv_id":null,"evidence_quote":"Proposes the superposition hypothesis that motivates interpreting polysemantic neurons as superimposed features."}],"review_version":1}