{"id":"2f245c72-6784-4fed-bd55-c43c2f02ccef","arxiv_id":"2606.07978","paper_version":1,"verdict":"UNVERDICTED","confidence":"UNKNOWN","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":2,"one_line_summary":"Factual knowledge crystallizes abruptly in final layers of LLMs (26.8%-93.4% of correct answers absent from top-10 until end), explaining why CAA outperforms DoLa on some models but not others.","lead":"This paper finds that factual knowledge in LLMs does not build gradually through layers but appears abruptly near the output. A smart generalist might read it to see why some editing methods reduce hallucinations more effectively than others.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.3","headline":"Tuned-lens top-10 may miss factual knowledge encoded non-linearly at intermediate layers","rationale":"The reader's weakest assumption directly identifies the measurement-validity bottleneck for the crystallization claim. No stronger internal inconsistency appears in the abstract; the concern is therefore load-bearing and already correctly flagged.","tokens_in":1847,"tokens_out":296,"duration_ms":11534,"concrete_test":"Train a 2-layer MLP decoder on the same intermediate activations and next-token objective used for the tuned lens; recompute the fraction of MMLU items whose correct answer never enters its top-10 before layer 80%. A drop of >15 percentage points relative to the linear tuned-lens numbers would indicate the original metric underestimates early presence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that non-appearance in tuned-lens top-10 at layers <80% depth means the factual knowledge has not yet crystallized. This interpretation is valid only if the affine transformation learned by the tuned lens recovers the correct token whenever the relevant information is present in the residual stream. If factual associations are instead represented via non-linear feature interactions or directions not aligned with the lens training objective, the metric will under-count early presence and produce an artifactual late-crystallization signal. The sentiment-classification control and cross-model results address probe choice but do not test whether the linear decoder itself is information-theoretically complete.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that factual knowledge in LLMs does not emerge gradually across layers but crystallizes abruptly in the final layers. Across five model families (0.5-14B), 26.8%-93.4% of correct answers never appear in tuned-lens top-10 predictions at intermediate layers, with >80% depth emergence consistent; this is specific to factual recall (vs. sentiment control) and affects intervention choice (CAA outperforms DoLa on moderate-crystallization models). Additional claims include a computability-memorization spectrum and a LayerNorm ablation showing intrinsic residual-stream effects. MechLens is released.","tokens_in":1971,"tokens_out":420,"duration_ms":19683,"significance":"If the measurement is valid, the late-crystallization finding supplies a mechanistic account of why certain interventions succeed and offers a practical editing principle (e.g., LN scaling). Cross-model and cross-benchmark consistency plus the public release of MechLens are concrete strengths that would aid reproducibility and follow-up work.","major_comments":[{"comment":"Abstract (quantification paragraph) and tuned-lens results: the central claim that non-appearance in top-10 at layers <80% depth means knowledge has not yet crystallized requires that the affine tuned lens recovers the correct token whenever the relevant information is present in the residual stream. The sentiment-classification control and cross-model results address probe choice but do not test whether non-linear feature interactions or directions misaligned with the lens objective could produce an artifactual late signal. A direct test (e.g., whether the lens achieves near-ceiling recovery on a set of facts known to be present early by other probes) is needed to support the interpretation.","section":"Abstract (quantification paragraph)"}],"minor_comments":[{"comment":"Abstract omits dataset sizes, number of prompts, error bars, and statistical-test details for the reported percentages and p-values.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for highlighting a key assumption in our measurement approach. We address the concern regarding the tuned lens below and outline a planned revision to strengthen the supporting evidence.","responses":[{"response":"We agree that the interpretation of late crystallization rests on the tuned lens being able to recover the correct token when relevant information is present in the residual stream. The tuned lens is an affine map trained to reconstruct the final-layer vocabulary distribution from each intermediate residual stream, which is the established method for this type of analysis. The sentiment-classification control (0.5–2.0% late emergence vs. 26.8–85.9% for facts) and cross-model/cross-scale consistency reduce the chance of a lens-specific artifact, as any systematic misalignment would be expected to appear across tasks. Nevertheless, we acknowledge that this does not directly rule out non-linear interactions or misaligned directions. We will add a direct validation in the revision: on a subset of facts, we will train an alternative linear probe on early-layer activations using ground-truth labels and compare its recovery rate to the tuned lens, reporting whether the lens achieves near-ceiling performance on facts independently shown to be present early.","revision_made":"yes","referee_comment":"[Abstract (quantification paragraph)] Abstract (quantification paragraph) and tuned-lens results: the central claim that non-appearance in top-10 at layers <80% depth means knowledge has not yet crystallized requires that the affine tuned lens recovers the correct token whenever the relevant information is present in the residual stream. The sentiment-classification control and cross-model results address probe choice but do not test whether non-linear feature interactions or directions misaligned with the lens objective could produce an artifactual late signal. A direct test (e.g., whether the lens achieves near-ceiling recovery on a set of facts known to be present early by other probes) is needed to support the interpretation."}],"tokens_in":1490,"tokens_out":411,"duration_ms":19854,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The one thing to know is that factual answers often stay out of tuned-lens top-10 until the final layers, with 26.8-93.4% never appearing earlier, and this split lines up with which intervention (CAA or DoLa) works better on different models. The computability-memorization difference and the simple LN scaling gain are the other concrete observations.\n\nThe paper does a solid job running the same check on five families and two scales, adding the sentiment control to show the effect is not generic, and releasing MechLens so others can replicate. The intervention results give a direct use case, and the LN adjustment is low-cost enough to try.\n\nThe soft spot is the measurement itself. The claim that knowledge has not crystallized rests on the tuned lens recovering the token if the information is present; if facts sit in non-linear combinations the affine map misses, the late signal is partly an artifact. The paper tests probe choice but not whether the decoder is information-theoretically complete. The 80% depth and top-10 cutoffs are free parameters whose effect on the percentages is not shown, and the abstract gives no error bars or dataset sizes.\n\nThis is for interpretability and editing researchers who want layer-wise patterns and a rule of thumb for method choice. It has enough new cross-model counts and a usable downstream claim to deserve referee time, even if the metric interpretation needs tightening.","headline":"Late crystallization is a measurable pattern in the tuned-lens data across models, but the linear top-10 metric leaves open whether knowledge is truly absent earlier or just encoded differently.","tokens_in":2434,"tokens_out":368,"would_cite":false,"duration_ms":18787,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Factual knowledge in language models crystallizes abruptly in the final layers rather than emerging gradually.","keywords":["late crystallization","factual knowledge","language models","knowledge interventions","residual stream","layer analysis","hallucination mitigation","model editing"],"falsifier":"Demonstrating a model family or scale in which the majority of factual answers from MMLU or similar benchmarks enter the top-10 predictions before 50% network depth would falsify the claim of consistent late crystallization.","tokens_in":2735,"feed_emoji":"🧠","tokens_out":714,"duration_ms":18399,"temperature":0.7,"pith_summary":"The paper establishes that factual knowledge does not accumulate steadily through a model's layers but instead appears suddenly near the end. Across multiple model families and scales, a substantial share of correct answers stays out of the top predictions until late depths, and this pattern holds specifically for factual recall rather than other tasks like sentiment classification. The timing of crystallization directly influences which editing methods succeed on a given model. Layer normalization is shown to be central to the process, and a distinction emerges between knowledge that can be computed and knowledge that is memorized.","feed_headline":"Factual knowledge crystallizes late in LLMs","feed_subtitle":"Most correct answers remain hidden until the final layers, which determines the best intervention for each model.","key_machinery":"Late Crystallization, the abrupt surfacing of factual knowledge in the final layers of the residual stream, which determines intervention success and is measured by when correct answers first enter top-10 predictions.","core_discovery":"Factual knowledge does not gradually emerge across layers but crystallizes abruptly at the final layers. Across five model families, 26.8%--93.4% of correct answers never enter top-10 predictions at any intermediate layer, with late emergence (>80% depth) consistent across architectures. Tuned lens rules out probe artifacts, and the pattern is far stronger for factual questions than for sentiment classification. This leads to a crystallization-guided intervention principle where method effectiveness varies by model, plus a computability-memorization spectrum and the finding that LayerNorm scaling improves accuracy at zero added cost.","pith_inferences":["Knowledge editing techniques could gain efficiency by targeting only the layers at which crystallization occurs for each model and task.","Training objectives might be designed to shift more knowledge toward earlier crystallization, potentially improving reliability on factual queries.","The same measurement approach could be applied to non-text modalities to test whether crystallization timing is a general property of transformer residual streams."],"forward_implications":["CAA outperforms DoLa on moderate-crystallization models such as Llama and Mistral.","On high-crystallization models such as Qwen the performance ordering reverses.","Scaling LayerNorm by a factor of 1.2 raises multiple-choice accuracy with no inference-time cost.","Computable knowledge crystallizes earlier (around layer 22 in a 28-layer model) than memorized facts (layer 28).","The late-crystallization pattern is specific to factual recall and does not appear in sentiment classification."],"fun_headline_variants":["Factual knowledge crystallizes abruptly in final layers","Late crystallization determines best interventions per model","Most facts remain hidden until end layers in LLMs","Crystallization is earlier for computable than memorized facts"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That the absence of a correct answer from top-10 predictions at intermediate layers means the knowledge is truly not yet present rather than stored in a form the metric does not capture.","fun_headline_variants_meta":{"raw":{"variants":["Factual knowledge crystallizes abruptly in final layers","Late crystallization determines best interventions per model","Most facts remain hidden until end layers in LLMs","Crystallization is earlier for computable than memorized facts"]},"model":"grok-4.3","cost_usd":0.006327,"raw_usage":{"total_tokens":3041,"prompt_tokens":805,"num_sources_used":0,"completion_tokens":58,"cost_in_usd_ticks":63274500,"prompt_tokens_details":{"text_tokens":805,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2178,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":805,"tokens_out":58,"duration_ms":13874,"temperature":1.0,"reasoning_tokens":2178,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T20:06:17.315108+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Demonstrating a model family or scale in which the majority of factual answers from MMLU or similar benchmarks enter the top-10 predictions before 50% network depth would falsify the claim of consistent late crystallization.","supporting_citations":[],"review_version":1}