{"id":"00d6a282-ef3c-4ef5-845c-594d32720a7f","arxiv_id":"2501.10573","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Token-level intrinsic dimension of internal representations correlates with next-token cross-entropy loss across layers in three LLMs; higher-loss prompts live in higher-dimensional token manifolds.","lead":"This paper measures how the geometry of token representations inside large language models changes across layers, and finds it correlates with prediction difficulty. Prompts that the model finds hard, as measured by higher next-token loss, tend to have tokens represented in higher-dimensional spaces, suggesting a possible unsupervised diagnostic.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The ID–loss correlation rests on unvalidated GRIDE/TWO-NN estimates in a 1024-point, 4096-dimensional regime; the central claim needs independent-estimator confirmation to rule out estimator artifacts.","rationale":"The paper is a good-faith empirical study: the same qualitative ID peak appears across three models, shuffling changes the metrics in a consistent direction, and the authors are transparent about limitations and provide a repository link. The strongest evidence is the repeated correlation between ID and loss, but that correlation is only as meaningful as the ID estimator on which it is built. The estimator's local-uniformity and independence assumptions are not validated for N = 1024 points in a 4096-dimensional residual stream, and no synthetic benchmark is reported. Because the central claim is about true manifold dimension, an estimator that responds to cloud dispersion, anisotropy, or density can produce the observed correlation even if the underlying geometric dimension is unrelated to loss. My proposed check is targeted: reproduce the main correlation with an independent estimator and, in parallel, measure GRIDE's recovery on known manifolds in the same finite-sample/high-ambient regime. If both pass, the concern is resolved; if not, the conclusion should be weakened to a statement about GRIDE-estimated dimension. These are addressable empirical checks, so the reader's CONDITIONAL verdict is appropriate; no change is needed.","tokens_in":17902,"tokens_out":5992,"duration_ms":71513,"concrete_test":"Recompute the Fig. 6 layerwise ID–loss correlation on the same 2244 prompt residual streams using a second, independent intrinsic-dimension estimator (e.g., ESS as in [19], or DANCo), and run a synthetic recovery check for GRIDE on N = 1024 points drawn from known d-dimensional manifolds embedded in R^4096 with controlled noise and anisotropy. If the correlation survives with the independent estimator and GRIDE recovers the known dimensions, the concern is settled; if the correlation is estimator-specific or recovery fails, the central claim should be downgraded to a property of GRIDE-estimated dimension rather than intrinsic dimension.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Sec. 5) is that token-cloud intrinsic dimension correlates with next-token cross-entropy loss, and that higher loss means higher-dimensional token representations. The load-bearing premise is that GRIDE/TWO-NN, as applied here, estimates the manifold dimension of the cloud. Equation (3) is exact only under local uniform density and independent nearest-neighbor ratios, and the paper applies it to N = 1024 points in a 4096-dimensional residual stream without any synthetic validation in this regime. At second-neighbor scale in high ambient dimension, Euclidean-distance ratios are sensitive to distance concentration, residual anisotropy, and local density; the estimated ID can then track estimator-driven properties rather than true manifold dimension. Since Fig. 6 correlates this estimated ID (log-transformed) with loss, an estimator artifact could produce the observed layerwise correlation even if true geometric dimension were unrelated to loss. The shuffling control shows that shuffling changes both the ID profile and loss, but it does not separate a genuine dimensional change from the estimator's response to a changed point-cloud distribution. The linked repository could not be checked, so there is no independent confirmation that the pipeline is unbiased. The central claim is plausible but not yet secured.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript studies token-level geometry of internal representations in three decoder-only LLMs (Llama 3 8B, Mistral 7B, Pythia 6.9B) through the lens of empirical measures. It computes intrinsic dimension (GRIDE/TWO-NN), neighborhood overlap, and cosine similarity across layers for 2244 Pile-10K prompts of 1024 tokens, compares structured prompts with block-shuffled versions, and reports a layerwise Pearson correlation between log intrinsic dimension and average next-token cross-entropy loss. Section 5.1 proposes an explanatory chain from last-layer ID to logits ID to contextual entropy to loss, supported by a toy softmax model. The abstract and conclusions additionally suggest that ID could be a metric for evaluating model performance across models.","tokens_in":18155,"tokens_out":8818,"duration_ms":79617,"significance":"If the central correlation is real, the paper makes a meaningful empirical contribution: it extends prior prompt-level studies (e.g., Cheng et al., 2023, 2024) to token-level point clouds inside a prompt and shows a consistent layerwise association across three independently trained models, with p-values mostly below 0.01. The analysis is direct rather than circular: the main quantity is a measured correlation, not a fitted parameter, and the authors are careful to label the softmax-box and Dirichlet calculations as toy examples. The scale analysis in Appendix C and the comparison with ESS-based prompt-level correlations in Table 1 are useful consistency checks. The reproducibility statement gives a code repository. However, the result is currently gated by two issues: the ID estimator is not validated in the specific regime used, and the pooled correlation is not shown to be robust to the Pile-10K source-domain structure. These are the main reasons the central claim is not yet secured.","major_comments":[{"comment":"The load-bearing premise that GRIDE/TWO-NN estimates the true local manifold dimension in this regime is not validated. The distribution in Eq. (2) requires local uniform density and independence of the neighbor ratios across points, and the paper applies it to N=1024 points in d=4096 without any synthetic test at this sample size and ambient dimension. The footnote asserting that local homogeneity 'is generally true' is not evidence. Because Fig. 6 correlates the log of this ID estimate with loss, an estimator artifact—say, sensitivity to distance concentration or to non-uniform density at second-neighbor scale in high dimension—could produce the reported layerwise correlations even if the true geometric dimension is unrelated to loss. The shuffling contrasts do not resolve this, since shuffling changes the point-cloud distribution in ways that could alter the estimator's bias. Required: validate GRIDE on synthetic manifolds of known dimension embedded in 4096 dimensions with N=1024, including non-uniform and anisotropic densities, and/or reproduce the ID profiles and correlations with an independent estimator (e.g., ESS, correlation dimension, or local PCA).","section":"Section 3, Eq. (2)-(3)"},{"comment":"The toy model connects softmax entropy to the number of active logits D_M, not to the GRIDE estimate of the intrinsic dimension of the logit point cloud. The identification between the estimated ID and the parameter D_M is not established; in the empirical analysis, ID is estimated from a point cloud of logits in a vocabulary-sized ambient space, whereas the toy model presumes an explicit coordinate box of dimension D_M. The reported rho=0.43 between log ID and contextual entropy is therefore consistent with mechanisms other than the toy model. A concrete test is to generate synthetic logit clouds with known D_M at the relevant ambient dimension and sample size, and verify that GRIDE recovers D_M; alternatively, the text should explicitly state that the toy model is only an analogy and not a derivation of the observed correlation.","section":"Section 5.1, Eqs. (11)-(15)"},{"comment":"The Pearson correlation is computed on 2244 prompts pooled over the 22 source domains of Pile-10K. If both the average loss and the ID estimates vary systematically by source, the pooled correlation can be inflated by a domain-level confound, and the statement that 'prompts with a higher cross-entropy loss have token representations lying in higher dimensional manifolds' would not follow within domains. Please report source-stratified correlations or a partial correlation controlling for the Pile source, and show that the correlation holds for at least the most frequent sources.","section":"Section 5, Fig. 6"}],"minor_comments":[{"comment":"The Pearson coefficient rho=0.96 between the log ID of the last layer and the log ID of the logits is stated without a scatter plot, confidence interval, or model breakdown; please provide the supporting figure or at least the per-model values.","section":"Section 5.1, item 1"},{"comment":"The text says Pythia has a lower ID peak than the other models 'though the significance is low'; because this is an explicit statement of low significance, either provide a formal significance test (e.g., a permutation test over prompts) or refrain from treating the difference as a model-level property.","section":"Section B.2"},{"comment":"The sentence 'analysis of the correlation between the logits ID at scaling = 2 ... and the contextual entropy to the average contextual entropy' is grammatically garbled; please state explicitly which quantity is plotted on each axis in each panel.","section":"Figure 7 caption"},{"comment":"The variables x_{i,k} and r_{i,k} are used in the 'Distribution of tokens at the ID peak' paragraph but defined only in the footnote of the following paragraph; move the definition to the first occurrence.","section":"Section 4.2"},{"comment":"The repository URL 'https://github.com/RitAreaSciencePark/token geometry' contains a space and could not be resolved during review; please ensure the link is valid and includes a README with the exact model revisions, filtering code, and GRIDE implementation needed to reproduce all figures.","section":"Reproducibility section"},{"comment":"The claim that ID 'could be an important metric for evaluating model performance across different models' goes beyond the within-model, across-prompt correlation reported in Fig. 6; either add a cross-model analysis (e.g., relating peak ID to per-model loss while controlling for model identity) or soften the conclusion to the within-model setting actually tested.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a serious empirical study with honest limitations and useful comparisons to prior work. The main risk is not circularity—the correlation is measured directly—but estimator validity and domain confounding. I would encourage the authors to add the synthetic validation and alternative-estimator analysis described in Major Comment 1, and the domain control in Major Comment 3, before publication. If those checks pass, the paper would be a solid contribution to the interpretability literature. The cross-model claim in the conclusion should be either supported or scaled back."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Tom—\n\nHere's my read on 2501.10573. The headline result—token-level intrinsic dimension in the residual stream correlates with next-token cross-entropy loss across layers, with shuffled prompts showing higher ID peaks—is real as an empirical observation, and it's new relative to the prompt-level ID studies (Valeriani et al., Cheng et al.). The paper does a lot right: three models, 2244 prompts, clear methodology, honest discussion of limitations, and a reproducibility link. The shuffling contrast is a good idea, and the fact that the effect reverses at the token level compared to the prompt level is interesting and deserves attention.\n\nBut the soft spot the stress-test flags is real: TWO-NN/GRIDE is applied to 1024 points in a 4096-dimensional space without any synthetic validation. The estimator's assumptions (local uniformity, independent neighbor ratios) are from a different regime. The footnote claiming that the assumptions hold 'generally on a wide range of datasets' is an assertion, not a check. Without an independent estimator—say, a different ID method or a synthetic test with known manifold dimension embedded in 4096 dimensions—I can't rule out that the correlation is an artifact of the estimator's response to distance concentration or anisotropy that also tracks loss. The fact that the correlation strengthens at larger range scalings helps a little, but it doesn't settle it.\n\nTwo other gaps: the shuffled-vs-unshuffled differences in ID and NO are shown with error bars but never given a significance test, so it's hard to know how much of the contrast is systematic. And the explanatory chain in Section 5.1 has a weak link: the logits-ID-to-contextual-entropy correlation is only rho=0.43, and the toy model (softmax entropy ~ log D) is too idealized to carry the argument. It's honest to call it a toy model, but it doesn't make the mechanism convincing.\n\nThe Pile-10K domain structure is a potential confound for the ID-loss correlation, but I don't think it's fatal; the correlation is consistent across models, and the shuffling control partly addresses it.\n\nBottom line: this is a solid empirical paper that deserves a serious referee. The central claim is plausible but not yet secured. The right fix is a validation appendix—apply TWO-NN/GRIDE to synthetic manifolds of known dimension in the same ambient dimension, and run at least one alternative ID estimator on the real prompts—plus significance tests for the shuffling contrasts. If I were editing, I'd send it to review and ask for those experiments in the first round. I'd bring it to the reading group as a good example of how to do a probe-based study with honest caveats, but I wouldn't cite the ID-loss correlation in my own work until the estimator question is closed.\n\n—M.","headline":"A solid, honest token-level study of how intrinsic dimension tracks next-token loss, but the ID estimator is unvalidated at this scale, so the headline correlation needs an independent sanity check before I'd fully trust it.","tokens_in":18676,"tokens_out":3374,"would_cite":false,"duration_ms":32786,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Token geometry carries the signature of prediction difficulty: intrinsic dimension correlates with next-token loss.","keywords":["intrinsic dimension","token representations","cross-entropy loss","next-token prediction","empirical measure","transformer interpretability","neighborhood overlap","GRIDE"],"falsifier":"Run the same GRIDE/TWO-NN estimator on synthetic point clouds of known intrinsic dimension, using 1024 points embedded in 4096 dimensions with realistic anisotropy, correlated directions, and nonuniform density; if the estimate systematically misses the true dimension in this regime, the quantitative claims lose their support. A second check is to compare prompts matched for cross-entropy loss but with different syntactic or semantic structure: if their token-level intrinsic-dimension profiles differ substantially despite equal loss, then the intrinsic-dimension-loss correlation is not a stable signature of prediction difficulty.","tokens_in":17707,"feed_emoji":"📐","tokens_out":10065,"duration_ms":89417,"temperature":0.7,"pith_summary":"This paper tries to show that the geometry of a prompt's tokens inside a large language model encodes how hard the next-token prediction is. By measuring the intrinsic dimension of 1024-token point clouds in the residual stream, the vector representation the model carries from layer to layer, the authors find that prompts with higher average cross-entropy loss are represented in higher-dimensional token manifolds, with the strongest correlation in the early-to-middle layers. They support this with a chain of observations: the intrinsic dimension of the last-layer representations tracks the intrinsic dimension of the logits, the logits' intrinsic dimension tracks the entropy of the softmax distribution, and that contextual entropy approximates the cross-entropy loss. If the claim holds, intrinsic dimension becomes an unsupervised geometric proxy for prediction difficulty and model performance, measurable from the activations alone.","feed_headline":"Harder prompts live in higher-dimensional token spaces","feed_subtitle":"Across three large language models, next-token loss tracks the intrinsic dimension of token clouds.","key_machinery":"The central object is the empirical measure of the token cloud at each layer: the probability distribution that puts equal mass on every token's position in the residual stream. To probe that measure the paper relies on the GRIDE intrinsic-dimension estimator, a likelihood-based nearest-neighbor method whose range-scaling-2 special case is the TWO-NN estimator, which converts the ratios of second-to-first nearest-neighbor distances into a local dimension estimate. That estimate carries the argument: the intrinsic-dimension profile across layers, the higher peak for shuffled prompts, and the correlation between intrinsic dimension and loss are all computed with it. Neighborhood overlap between adjacent layers and cosine similarity serve as complementary probes of how coherent token neighborhoods are and how aligned the token vectors become, and a chain through logits and softmax entropy connects the geometric quantity to the information-theoretic loss.","core_discovery":"On the paper's own account, the central discovery is stated in Section 5: the intrinsic dimension of token representations across hidden layers is correlated with the average cross-entropy loss of the next-token probability distribution for a given prompt. Across the three decoder-only models studied, the Pearson correlation between $\\log(\\mathrm{ID})$ and loss is positive and significant, especially around the early-to-middle-layer intrinsic-dimension peak. The paper also establishes a layer-by-layer chain: the last-layer token representation is linearly unembedded into logits, the intrinsic dimension of the logits tracks the intrinsic dimension of the last layer ($\\rho = 0.96$), the logits' intrinsic dimension correlates with the contextual entropy of the softmax output ($\\rho = 0.43$ for one of the models), and the contextual entropy, averaged over a long prompt, is nearly the cross-entropy loss. Toy calculations with logits supported on a $D_M$-dimensional unit box or probability simplex give $\\langle S\\rangle \\sim \\log D_M$, suggesting that softmax entropy grows logarithmically with the intrinsic dimension of the logit manifold.","pith_inferences":["A natural extension the paper leaves implicit is a per-token version of the correlation: if prompt-level loss is tracked by token-cloud dimensionality, then per-token difficulty maps could be read off hidden states alone, flagging high-surprisal tokens without computing the output distribution.","The shuffling contrast suggests that the geometry of the empirical measure acts as a structure detector that should transfer to non-text token streams, such as protein or image-patch sequences, and could flag distribution shift or out-of-domain inputs.","The $\\langle S\\rangle \\sim \\log D_M$ calculation hints that constraining the logit manifold to a lower-dimensional subspace would reduce predictive entropy, a testable prediction about confidence and calibration that the paper does not make.","Because the correlation is strongest near the early-to-middle-layer intrinsic-dimension peak, the layer at which the peak occurs could serve as a diagnostic of where a model commits to its prediction, connecting this work to layerwise latent-prediction analyses."],"forward_implications":["If the central claim is right, intrinsic dimension can serve as an unsupervised metric for evaluating model performance: prompts with higher loss are represented in higher-dimensional manifolds, and the correlation holds across three different model families.","The shuffle experiments imply that natural syntactic and semantic structure compresses token representations: disrupting that structure raises the intrinsic-dimension peak, increases cosine alignment among tokens, and lowers neighborhood overlap around the peak.","Because last-layer intrinsic dimension correlates with logit intrinsic dimension ($\\rho = 0.96$), the correlation can be read off the residual stream without evaluating the full softmax distribution.","The toy-model relation $\\langle S\\rangle \\sim \\log D_M$ suggests the softmax entropy is bounded by the intrinsic dimension of the logit manifold, giving a concrete geometric mechanism for the loss correlation."],"supporting_citations":[{"why":"Supplies the TWO-NN estimator formula (Equation 3) that turns nearest-neighbor distance ratios into an intrinsic-dimension estimate.","marker":"[41]"},{"why":"Supplies the GRIDE likelihood-based estimator and the range-scaling scheme used for the intrinsic-dimension profiles.","marker":"[42]"},{"why":"Provides the mean-field empirical-measure perspective in which the output token measure encodes the next-token probability distribution.","marker":"[2]"},{"why":"Establishes that unembedding the residual stream produces logits, the basis for the last-layer-to-logits intrinsic-dimension step.","marker":"[13]"},{"why":"Supplies the prior token-level intrinsic-dimension and neighborhood-overlap analysis that this work extends to in-prompt token clouds.","marker":"[18]"},{"why":"Provides the earlier prompt-level correlation between peak intrinsic dimension and perplexity that this paper refines to a per-prompt, layerwise correlation.","marker":"[19]"},{"why":"Supplies the prompt-level high-dimensional abstraction phase and training-time intrinsic-dimension behavior used for comparison with the token-level results.","marker":"[20]"},{"why":"Defines the neighborhood-overlap metric used to measure how consistently token neighborhoods persist across adjacent layers.","marker":"[43]"}],"fun_headline_variants":["Token geometry predicts next-token difficulty","Higher token dimension, higher loss","Intrinsic dimension tracks next-token loss","Loss grows with token space dimensionality","Token clouds hint at prompt loss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the GRIDE/TWO-NN intrinsic-dimension estimate, computed from 1024 tokens in a 4096-dimensional residual stream, genuinely measures the local manifold dimension of the token representations; the estimator assumes locally uniform density and independent nearest-neighbor ratios, and if those assumptions fail in this regime, the intrinsic-dimension peak, the shuffle contrasts, and the intrinsic-dimension-loss correlation could be artifacts of the estimator rather than properties of the representations.","fun_headline_variants_meta":{"raw":{"variants":["Token geometry predicts next-token difficulty","Higher token dimension, higher loss","Intrinsic dimension tracks next-token loss","Loss grows with token space dimensionality","Token clouds hint at prompt loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000239,"raw_usage":{"total_tokens":1483,"prompt_tokens":883,"completion_tokens":600,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":541}},"tokens_in":499,"tokens_out":600,"duration_ms":6922,"temperature":1.0,"reasoning_tokens":541,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:06:31.840406+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same GRIDE/TWO-NN estimator on synthetic point clouds of known intrinsic dimension, using 1024 points embedded in 4096 dimensions with realistic anisotropy, correlated directions, and nonuniform density; if the estimate systematically misses the true dimension in this regime, the quantitative claims lose their support. A second check is to compare prompts matched for cross-entropy loss but with different syntactic or semantic structure: if their token-level intrinsic-dimension profiles differ substantially despite equal loss, then the intrinsic-dimension-loss correlation is not a stable signature of prediction difficulty.","supporting_citations":[{"cited_title":"Estimating the intrinsic dimension of datasets by a minimal neighborhood information,","cited_arxiv_id":null,"evidence_quote":"Supplies the TWO-NN estimator formula (Equation 3) that turns nearest-neighbor distance ratios into an intrinsic-dimension estimate."},{"cited_title":"Distributional Results for Model-Based Intrinsic Dimension Estimators","cited_arxiv_id":"2104.13832","evidence_quote":"Supplies the GRIDE likelihood-based estimator and the range-scaling scheme used for the intrinsic-dimension profiles."},{"cited_title":"interpreting gpt: the logit lens,","cited_arxiv_id":null,"evidence_quote":"Establishes that unembedding the residual stream produces logits, the basis for the last-layer-to-logits intrinsic-dimension step."},{"cited_title":"The geometry of hidden representations of large transformer models,","cited_arxiv_id":null,"evidence_quote":"Supplies the prior token-level intrinsic-dimension and neighborhood-overlap analysis that this work extends to in-prompt token clouds."},{"cited_title":"Bridging information-theoretic and geometric compression in language models,","cited_arxiv_id":null,"evidence_quote":"Provides the earlier prompt-level correlation between peak intrinsic dimension and perplexity that this paper refines to a per-prompt, layerwise correlation."},{"cited_title":"Hierarchical nucleation in deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Defines the neighborhood-overlap metric used to measure how consistently token neighborhoods persist across adjacent layers."}],"review_version":1}