{"id":"e812002f-632e-43b7-b6b2-556bb1f8b018","arxiv_id":"2412.06245","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Across four decoder LLMs and eight tasks, in-context learning with five or more demonstrations consistently yields higher intrinsic dimensions in hidden representations than LoRA fine-tuning, while fine-tuning achieves higher accuracy.","lead":"This paper measures the intrinsic dimension of LLM hidden states during in-context learning and fine-tuning. It finds that in-context learning produces higher-dimensional representations than fine-tuning, even when fine-tuning gives better task accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central ICL-vs-SFT comparison confounds learning paradigm with input length: ICL-k≥5 prompts are much longer than SFT/0-shot prompts, and no length-matched control is reported.","rationale":"The reader's weakest assumption concerns TwoNN estimator bias and missing error bars. That is a valid secondary concern, and the paper's MLE sanity check (Appendix F) with r=0.70 is only moderate. However, the more load-bearing issue is that the headline comparison is confounded by input length. Even a perfectly unbiased ID estimator would not resolve whether the observed ID gap reflects the learning paradigm or simply the fact that ICL-k≥5 prompts contain many more tokens. The paper reports no control for prompt length or for task-irrelevant context, so the central claim is not yet supported as a statement about representational geometry. Since this is a missing control rather than a demonstrated contradiction, the appropriate verdict remains conditional pending that experiment. The reader's own rationale already calls for additional robustness analysis, so the recommendation is unchanged, but for a different primary reason than the reader identified.","tokens_in":25116,"tokens_out":4824,"duration_ms":53969,"concrete_test":"Run the Section 7 comparison with a length-matched control: construct ICL-style prompts with k=5 task-irrelevant demonstrations (e.g., demonstrations from an unrelated task, or matched-length filler text) and compute ID curves for the unfine-tuned model. If the normalized AUC rises to the ICL-5 level, the higher ID is driven by prompt length/content volume rather than by ICL's task demonstrations. Additionally, evaluate the SFT checkpoint on the same ICL-5 prompts used for the base model; if its ID increases to match ICL-5, the SFT-vs-ICL gap is a property of the prompt, not a representational consequence of the learning paradigm.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Section 7.2) is that ICL with k≥5 consistently induces higher intrinsic dimensions than SFT or 0-shot prompts. But the comparison is not apples-to-apples in input length. In Section 4.1, ID is computed from last-token hidden states, which depend on the entire preceding context. ICL prompts contain k demonstrations and are systematically longer than the SFT/0-shot prompts, which contain no demonstrations. The paper's own k vs. AUC results (Section 6.2) show ID rising from k=0 to k≈5 and then plateauing or decreasing, a pattern consistent with a length/context-volume effect that saturates. No control condition separates 'ICL as a learning paradigm' from 'more tokens in context.' For example, the SFT model is never evaluated on long prompts, and the zero-shot model is never evaluated on prompts padded with task-irrelevant text to match ICL-5 length. The abstract's interpretation that 'representations generated during ICL reside in higher dimensional manifolds' therefore rests on an uncontrolled variable. This is especially consequential because Doimo et al. (2024), cited in Section 3.3, report the opposite ordering; a simple length confound could explain the discrepancy without any difference in how the two paradigms shape the representation manifold.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents an empirical study of the intrinsic dimension (ID) of LLM hidden representations under supervised fine-tuning (SFT) and in-context learning (ICL). Using the TwoNN estimator on last-token hidden states from four open-weight LLMs (7B–13B) across eight datasets, the authors report three main findings: (i) ID generally increases over the course of LoRA fine-tuning, with occasional initial decreases; (ii) ID initially rises with the number of ICL demonstrations k and then plateaus or decreases; and (iii) ICL with k≥5 consistently yields higher ID than SFT or zero-shot prompts, which they interpret as evidence that ICL representations inhabit higher-dimensional manifolds. They also propose the k value at peak ID as a practical heuristic for choosing demonstration count.","tokens_in":25419,"tokens_out":5514,"duration_ms":51104,"significance":"If the central comparison were robust, this would be a useful geometric characterization of two major adaptation paradigms, and the demonstration-count heuristic could save computational resources. The paper has genuine strengths: it is systematic, covering 8 datasets and 4 models; it uses an established estimator with a sanity check against MLE (Appendix F); it reports code availability; and it includes a check against demonstration-reuse artifacts (Appendix D). However, the claim that ICL induces higher ID than SFT is currently undermined by an uncontrolled input-length confound and by the absence of any uncertainty quantification, both of which are load-bearing. The significance is therefore conditional on these issues being resolved.","major_comments":[{"comment":"The central comparison confounds learning paradigm with input length. In Section 4.1, ID is measured on the last-token hidden state of each input sequence, so the estimate reflects the entire preceding context. ICL prompts with k≥5 are systematically longer than SFT prompts (which contain no demonstrations) and zero-shot prompts. The paper reports no control in which SFT or zero-shot models are evaluated on long prompts, e.g., padded with task-irrelevant text to match ICL-5 length. The k-vs-AUC results in Section 6.2, where ID rises from k=0 to k≈5 and then plateaus or decreases, are precisely the pattern one would expect from a saturating context-length effect. Without such a control, the abstract's conclusion that 'representations generated during ICL reside in higher dimensional manifolds' is not warranted by the data. This is the load-bearing point of the paper and needs an additional experiment.","section":"Section 7.2 (and Section 4.1)"},{"comment":"No measure of estimator variability is reported. ID estimates are point values with no error bars, confidence intervals, or repeated runs. The consistency claim 'across all datasets and models' in Section 7.2 depends on differences between paradigms that may be within estimator noise. The only reliability check, the TwoNN-versus-MLE correlation in Appendix F (r=0.70), is moderate. The authors should provide bootstrap confidence intervals for the ID estimates or repeated independent runs, and verify that the ordering ICL-k≥5 > SFT and ICL-k≥5 > ICL-0 holds after accounting for this variability.","section":"All experimental figures (e.g., Figures 4 and 5)"},{"comment":"The proposed heuristic that the peak of the k-vs-AUC curve indicates the optimal number of demonstrations is tested on the same six experiments from which it is derived, and the match is reported as 5 out of 6. This is an in-sample evaluation with a small sample; no held-out validation or out-of-sample test is provided. The heuristic therefore should be framed as a preliminary observation, not as a practical indicator, unless the authors add a validation experiment or cross-validation.","section":"Section 6.2"}],"minor_comments":[{"comment":"The displayed formula for TwoNN suggests a closed-form per-point estimate, but the estimator actually computes the slope of a linear fit to the empirical CDF; please clarify the estimation procedure.","section":"Section 2.2"},{"comment":"The first sentence contains a subject-verb agreement error: 'ICL prompts with k ≥ 5 consistently induces higher intrinsic dimensions' should be 'ICL prompts with k ≥ 5 consistently induce higher intrinsic dimensions'.","section":"Section 7.2"},{"comment":"Figure 36 is cited in Section 7.3.1 but is located in Appendix C; please add explicit cross-references to appendices for all cited figures.","section":"Section 7.3.1 and Appendix C"},{"comment":"Table 1 reports accuracy without any variance; because demonstrations are sampled randomly and fine-tuning has stochasticity, standard errors or a note on single-run reporting should be included.","section":"Table 1"},{"comment":"Appendix G appears to contain dataset prompts but the text refers to 'details of dataset creation' in Appendix G; add a section heading and describe the split generation procedure used for CommonsenseQA and MMLU (which are drawn from train/test splits rather than a validation split).","section":"Appendix G"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution but the central comparison currently has a confound and no uncertainty analysis. For this journal, I would require the authors to add a length-matched control and uncertainty quantification before it can be considered. The relationship with Doimo et al. (2024), who report the opposite SFT/ICL ordering, should be directly discussed in light of the proposed control."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2412.06245. First, it's a systematic and reasonably careful empirical description of how intrinsic dimension (ID) of hidden representations changes during SFT and ICL across four models and eight datasets. The genuinely new observation is that ID in ICL rises with the number of demonstrations up to about k=5 and then plateaus or decreases—Doimo et al. only went to k=5 and reported a monotone increase. That is worth having. Second, the paper's headline claim—that ICL induces higher ID than SFT—is not actually established, because the comparison confounds learning paradigm with input length. ICL prompts with k>=5 demonstrations are much longer than the SFT/zero-shot prompts, and ID is computed from last-token hidden states, which depend on the whole context. There is no control condition that separates 'more demonstrations' from 'more tokens in context.' The authors never evaluate SFT on long prompts or ICL on prompts padded with task-irrelevant text. Their own k-vs-AUC curves, which rise and then saturate, are exactly what a length effect would look like. So the abstract's 'higher dimensional manifolds' interpretation rests on an uncontrolled variable.\n\nWhat the paper does well: the sweep is broad—4 models, 8 datasets—and the results are presented transparently with per-layer curves in appendices. They also ran a useful control for demonstration reuse across prompts (Appendix D) and report a sanity check correlating TwoNN with MLE (r=0.70). That is honest empirical work. The limitations section is candid about model sizes and LoRA-only fine-tuning.\n\nThe soft spots, in order of importance. (1) The length confound above. It is fixable, but as it stands the central comparison is not apples-to-apples. (2) There are no error bars, confidence intervals, or repeated runs anywhere in the paper. The ID estimates are treated as exact numbers, and the TwoNN-MLE correlation of 0.70 leaves room for estimator variability that could change the ordering. (3) The k-selection heuristic is based on six (model, dataset) pairs and is presented more strongly than the evidence warrants. (4) Minor: the code is mentioned but no link appears in the text.\n\nWho is this for? People working on mechanistic interpretability or model selection. It is a useful descriptive map of ID behavior, but not yet a reliable statement about how the two learning paradigms shape the representation manifold. I'd send it to peer review with a request for length-matched control experiments and uncertainty quantification. With those, the plateau finding and the systematic comparison would be a solid contribution.","headline":"A useful empirical survey of ID in LLM representations, but the headline ICL-vs-SFT comparison is confounded by input length and needs a control before the central claim can be believed.","tokens_in":25923,"tokens_out":3025,"would_cite":false,"duration_ms":29287,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Across four models and eight datasets, in-context learning with five or more demonstrations yields higher intrinsic dimension in hidden representations than supervised fine-tuning or zero-shot prompting.","keywords":["intrinsic dimension","in-context learning","supervised fine-tuning","large language models","hidden representations","TwoNN estimator","representation geometry","few-shot learning"],"falsifier":"Run the same protocol many times with fresh random seeds and subsamples of the validation set, then form bootstrap confidence intervals on the Normalized AUC difference between ICL-5 and SFT; if the gap is within noise, the ordering claim fails. A second check is to pad fine-tuning prompts with neutral filler text to match ICL input lengths: if the higher ID follows longer prompts rather than the learning paradigm itself, the central comparison collapses.","tokens_in":24929,"feed_emoji":"🧠","tokens_out":6971,"duration_ms":62599,"temperature":0.7,"pith_summary":"The paper argues that the two main ways of teaching a large language model a new task—fine-tuning its weights or giving it examples in the prompt—leave different geometric footprints in the hidden representations, and that this difference shows up as a measurable quantity called intrinsic dimension (ID), the number of effective degrees of freedom in the set of hidden states. Across four open-weight models and eight datasets, prompts with five or more demonstrations consistently produce higher ID at every layer than both fine-tuned models and zero-shot prompts. Fine-tuned models, meanwhile, score highest on accuracy while staying closest to the zero-shot baseline in ID. The paper also finds that ID first rises and then plateaus or falls as demonstrations are added, and that the ID peak tends to line up with the accuracy plateau, so ID could be a cheap guide for choosing how many demonstrations to use.","feed_headline":"ICL pushes LLM representations to higher dimensions than fine-tuning","feed_subtitle":"With five or more demonstrations, hidden states spread out; fine-tuning keeps them near the zero-shot baseline.","key_machinery":"The load-bearing object is the Intrinsic Dimension Curve: collect the last-token hidden-state vectors from all validation prompts at each transformer layer, estimate the intrinsic dimension of each layer's point cloud, and plot ID against layer index. The estimate comes from the TwoNN estimator, which takes each point's ratio μ = r2/r1 of second-nearest to nearest-neighbor distance and reads the dimension off the slope relating log μ to the empirical survival probability, under the local Poisson assumption. The per-layer curve is then compressed into a Normalized AUC (trapezoidal average across layers), which lets models of different depths be compared on a single number. This curve and its AUC are the metric that carries the whole argument.","core_discovery":"On its own terms, the central claim is that in-context learning induces a higher intrinsic dimension in LLM hidden representations than supervised fine-tuning does. In every (model, dataset) pair tested, ICL prompts with k ≥ 5 demonstrations yield ID curves above both the zero-shot baseline and the LoRA fine-tuned model at every layer, which the authors read as evidence that ICL representations live on higher-dimensional manifolds in embedding space. A second claim is that ID responds non-monotonically to demonstration count: it grows through small k, then plateaus or declines for larger k, and in most experiments the k with peak ID also reaches peak or near-peak accuracy. A third claim is that the ID of fine-tuned models tracks the zero-shot baseline closely even while accuracy rises, so ID is not a proxy for task performance. The paper contrasts its SFT-versus-ICL ordering with a concurrent study reporting the opposite ordering.","pith_inferences":["Editorial inference: the higher ID of ICL could partly be a prompt-length effect, since longer demonstration contexts change the distribution of last-token embeddings; a controlled comparison that pads fine-tuning prompts to the same length would show whether the paradigm or the sequence length drives the gap.","Editorial inference: if the geometry reflects mechanism, then parameter-efficient methods that interpolate between ICL and SFT, such as prefix tuning or attention-only adaptation, should produce ID values between the two paradigms, which is testable.","Editorial inference: the paper's own control experiments only rule out demonstration reuse as the cause of the ID plateau; an equally cheap check would be fixing k while shuffling or corrupting demonstrations, which should lower ID if the task signal, not the context length, is what expands the manifold.","Editorial inference: because the TwoNN estimator is sensitive to sampling density, the 5,000-prompt validation set size may itself set a ceiling on detectable ID differences; re-running with larger pools could push the reported gap up or down."],"forward_implications":["With five or more demonstrations, ICL can be expected to show higher intrinsic dimension than SFT across models of different families and sizes, not just the four models tested here.","The k value at which ID peaks can serve as a practical stopping point for choosing the number of demonstrations, since accuracy has usually plateaued by that point.","Intrinsic dimension is not a proxy for accuracy: fine-tuned models have the lowest ID and the highest accuracy, so a low ID should not be read as poor task performance.","Fine-tuned representations remain geometrically close to the zero-shot baseline, suggesting that LoRA fine-tuning on 1,000 examples reshapes the representation space less than adding demonstrations does.","The absence of a universal hunchback ID curve in LLMs, unlike CNNs and image transformers, points to a more diverse layer-wise organization of language representations."],"supporting_citations":[{"why":"Provides the TwoNN estimator that all ID measurements in the paper are built on.","marker":"Facco et al. (2017)"},{"why":"Concurrent study on MMLU whose opposite SFT-versus-ICL ordering is the main comparison the paper directly counters.","marker":"Doimo et al. (2024)"},{"why":"Prior result linking intrinsic dimension to validation performance and truthfulness; the paper tests and does not reproduce the validation trend, motivating its heuristic claims.","marker":"Yin et al. (2024)"},{"why":"Implicit Bayesian inference account of ICL that the paper uses to explain why ID plateaus as demonstrations increase.","marker":"Xie et al. (2021)"},{"why":"Shows ID values are consistent across model scales and correlate with fine-tuning ease, supporting the paper's cross-model comparisons.","marker":"Cheng et al. (2023)"},{"why":"Maximum likelihood ID estimator used as a sanity check, with a reported correlation r = 0.7 against TwoNN.","marker":"Levina and Bickel (2004)"},{"why":"Reports the hunchback ID pattern in CNNs that the paper uses as a baseline when arguing LLM ID curves are dataset-dependent.","marker":"Ansuini et al. (2019)"},{"why":"LoRA adaptation method used for all fine-tuning experiments, so the SFT-ID results are specific to low-rank updates.","marker":"Hu et al. (2021)"}],"fun_headline_variants":["ICL lifts LLM hidden states to higher intrinsic dimensions than SFT","In-context learning expands LLM representations more than fine-tuning","Fine-tuning keeps LLM ID near baseline; ICL pushes it higher","ICL representations live on higher-dimensional manifolds than SFT","Demonstration count non-monotonically shifts LLM intrinsic dimension"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire comparison rests on the assumption that the intrinsic-dimension estimator is unbiased for these hidden-state sets and that the ICL-versus-SFT gap is larger than the estimator's own variability, yet the paper reports no repeated runs, error bars, or confidence intervals to show that.","fun_headline_variants_meta":{"raw":{"variants":["ICL lifts LLM hidden states to higher intrinsic dimensions than SFT","In-context learning expands LLM representations more than fine-tuning","Fine-tuning keeps LLM ID near baseline; ICL pushes it higher","ICL representations live on higher-dimensional manifolds than SFT","Demonstration count non-monotonically shifts LLM intrinsic dimension"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000402,"raw_usage":{"total_tokens":2075,"prompt_tokens":902,"completion_tokens":1173,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1083}},"tokens_in":518,"tokens_out":1173,"duration_ms":10002,"temperature":1.0,"reasoning_tokens":1083,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:51:57.857630+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same protocol many times with fresh random seeds and subsamples of the validation set, then form bootstrap confidence intervals on the Normalized AUC difference between ICL-5 and SFT; if the gap is within noise, the ordering claim fails. A second check is to pad fine-tuning prompts with neutral filler text to match ICL input lengths: if the higher ID follows longer prompts rather than the learning paradigm itself, the central comparison collapses.","supporting_citations":[],"review_version":1}