{"id":"813e9f5f-c852-433b-81fc-91927681b962","arxiv_id":"2504.14751","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Open-world AI requires rich features, disentangled representations, and inference-time learning; the thesis presents techniques and large-scale experiments supporting these principles.","lead":"This PhD thesis argues that artificial intelligence for open-ended, real-world tasks cannot be built with today's train-on-one-distribution approach; instead it needs three learning principles: rich features, disentangled representations, and learning at inference time. The thesis supports each principle with experiments, including a new memory-based architecture trained on up to one trillion tokens.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2.1.6 only shows P-equivalence of expected-loss optima, not that SGD lacks rich features; §2.4.3.5's buried-features argument undercuts 'regardless of model size and data size.'","rationale":"The reader's verdict of CONDITIONAL is appropriate. The empirical contributions (Cat, Bonsai, very-large dropout, Memory Mosaics v2) are substantial and largely self-contained; the theoretical claim in §2.2, however, overreaches. The proof of Theorem 2.1.6 works for any global minimizer of expected loss and says nothing about which minimizers gradient-based training visits, nor about feature usefulness under a shifted distribution. The thesis itself acknowledges this in §2.1.3 and then relies on an implicit sparsity-bias premise in §2.1.4 that is not established for the settings tested. The internal tension with §2.4.3.5 strengthens the concern: the very-large dropout method presupposes that standard pretraining already buries OOD-useful features in hidden layers, which is hard to square with the 'regardless of model size and data size' claim. The concrete counterexample shows the theorem permits rich optimal representations, so the 'cannot' conclusion must rest on an unproved dynamic premise. This does not overturn the empirical findings; it means the necessity claim should be stated conditionally, matching the reader's verdict.","tokens_in":53468,"tokens_out":8690,"duration_ms":85880,"concrete_test":"Exhibit a concrete instance of problem (2.2): let x=(x1,x2)~N(0,I), y=sign(x1), with logistic loss. Both Φ1=x1 and Φ2=(x1,x2) are global minimizers and carry equivalent information under P by Theorem 2.1.6, but under a shifted Q where y=sign(x2), C*_Q(Φ1∪Φ2)<C*_Q(Φ1). Constructing this instance settles that the theorem alone does not imply absence of rich features; the 'cannot' conclusion requires an additional, currently unproved premise about SGD's implicit bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the inference from Theorem 2.1.6 to the claim in §2.2 that standard i.i.d. training 'lacks the motivation to construct a representation with rich features, regardless of model size and data size.' The theorem says only that all global minimizers of the expected loss (2.2) carry equivalent information under P, where information is defined via the optimal linear-probing cost C*_P. Because problem (2.2) is unconstrained, a representation that contains extra, P-redundant features can be a global minimizer alongside a sparse one; the theorem therefore does not rule out rich representations, and it says nothing about which minimizers SGD finds. The missing premise is an explicit argument that SGD's implicit sparsity bias discards exactly the P-redundant features that would be useful under a shifted distribution Q. §2.1.4 cites literature on sparsity but does not establish this premise for the architectures and objectives used here. The tension is visible inside the thesis: §2.4.3.5 states that fine-tuning with very-large dropout works precisely because i.i.d.-pretrained models contain rich features 'buried in their hidden layers,' and that modern foundation models make this condition 'relatively easy to achieve.' That reading directly contradicts the 'regardless of model size and data size' conclusion. What the theory actually establishes is that the P-objective is indifferent to Q-useful features, not that such features are absent; the strong 'cannot' claim is therefore unsupported, although the Cat, Bonsai, and very-large-dropout experiments remain meaningful evidence for the practical benefit of rich features.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This PhD thesis argues that closed-world i.i.d. training is insufficient for open-world AI and proposes three learning principles: rich features, disentangled representation, and inference-time learning. Chapter 2 develops a linear-probing framework to argue that i.i.d. training lacks the motivation to discover rich features, then supports this with transfer, few-shot, invariant-learning, and fine-tuning experiments using Cat ensembles, Bonsai, and very-large dropout. Chapter 3 introduces predictive disentanglement and the Memory Mosaics architecture, with experiments on a three-moons toy task, BabiStories language modeling, and the RegBench in-context learning benchmark. Chapter 4 presents Memory Mosaics v2, a 9.9B-parameter memory-based architecture, and evaluates it on persistent-knowledge benchmarks, RULER-style multi-document QA, and in-context classification tasks, reporting substantial gains over transformers on new-knowledge storage and retrieval.","tokens_in":53730,"tokens_out":3744,"duration_ms":35109,"significance":"If the thesis' central claims were fully established, the work would be significant: it challenges the prevailing foundation-model scaling paradigm and points to concrete alternatives (Cat ensembles, Bonsai initialization, very-large dropout, Memory Mosaics v2) that are tested on external benchmarks such as Camelyon17, RegBench, RULER, and standard language tasks. The strength of the manuscript is its breadth of empirical evidence and the reproducibility-oriented release of code. The theoretical framework in Section 2.1 is clean, and the observation that multiple training episodes yield features that are P-equivalent but Q-beneficial is a valuable contribution. However, the sharpest advertised conclusion—that i.i.d. training 'cannot' discover rich features—is not supported by the stated theorem, and the manuscript contains an internal tension with its own very-large-dropout analysis.","major_comments":[{"comment":"The claim that Section 2.1 'theoretically proves that this approach lacks the motivation to construct a representation with rich features, regardless of model size and data size' is an overstatement. Theorem 2.1.6 only establishes that all global minimizers of the expected-loss problem (2.2) carry equivalent information under P, where information is defined via optimal linear-probing cost. Because problem (2.2) is unconstrained, a global minimizer can include extra P-redundant features alongside a sparse one; the theorem does not rule out rich representations, and it says nothing about which minimizers SGD finds. The missing premise is an explicit argument that SGD's implicit sparsity bias discards exactly the P-redundant features that would be Q-useful. The literature cited in §2.1.4 suggests a connection but does not establish this premise. The conclusion should be weakened to 'provides no incentive' or 'may not discover' rather than 'cannot'.","section":"§2.2, Theorem 2.1.6"},{"comment":"This passage directly contradicts the 'regardless of model size and data size' conclusion. The text states that fine-tuning with very-large dropout works precisely because i.i.d.-pretrained models contain rich features 'buried in their hidden layers' and that modern foundation models make this condition 'relatively easy to achieve.' That reading implies i.i.d. training can and does produce rich features, at least in large models, which undermines the theoretical claim in §2.2. The manuscript needs to reconcile these positions, for example by distinguishing features that are linearly exploitable under P from features that are present but not linearly accessible, and by clarifying that the theory addresses the former while the dropout experiments exploit the latter.","section":"§2.4.3.5"},{"comment":"The chapter claims that predictive disentanglement drives the learning of disentangled representations, but the evidence does not directly measure disentanglement. The three-moons experiment shows a functional benefit of multiple memory heads, and the RegBench results show strong in-context learning, yet neither quantifies the degree of disentanglement, e.g., through a dimension-factor correspondence metric. Moreover, §3.6 concedes that the analogous effect in standard transformers is only a conjecture. The disentanglement principle would be on firmer ground if the paper either reported a direct measure of disentanglement in the trained Memory Mosaics or explicitly framed predictive disentanglement as a hypothesis supported by indirect evidence.","section":"§3.2.2 and §3.5"},{"comment":"The claim that Memory Mosaics v2 'substantially outperforms transformers (>10%)' is based primarily on the new-knowledge storage tasks (RULER multi-document QA) and the in-context classification tasks. On the 19 persistent-knowledge benchmarks in Table 4.1, Memory Mosaics v2 and transformers perform nearly identically (e.g., 52.2% vs 52.2% for large models). Since the thesis' third principle concerns learning new tasks with fewer examples, the evidence is relevant, but the scope should be stated precisely; the current phrasing implies a broader superiority than the benchmarks support.","section":"§4.2.3"}],"minor_comments":[{"comment":"There are frequent grammatical and spelling errors, including 'numerous successes of AI has been made,' 'less priori knowledge' (should be 'prior knowledge'), 'innovated techniques' (should be 'innovative'), and 'qick-learning' in the Section 3.2 heading. These should be corrected in a full revision.","section":"Abstract and throughout"},{"comment":"The method is referred to inconsistently as 'very large dropout' and 'very-large dropout'; one hyphenated form should be used throughout.","section":"§2.4"},{"comment":"The sentence 'This is nothing to supervise' appears to be a typo, likely for 'This is nothing to be surprised about' or similar; please rephrase.","section":"§4.2.3.1"},{"comment":"The text refers to 'Appendix Table C.1' without a corresponding in-text description of its contents; please either describe the table or move it into the main text.","section":"§4.2.3.2"},{"comment":"The captions read 'Single head network' and 'Three-heads network'; the latter should be 'Three-head network' for grammatical consistency.","section":"§3.3"},{"comment":"Some citations are abbreviated inconsistently (e.g., 'Chen et al. 2019' and 'Chen et al. 2020' for the same few-shot baseline work), and the bibliography should be checked for completeness.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a PhD thesis with three largely self-contained chapters, some based on already-published work. The main risk for the journal is that the advertised theoretical claim in Section 2.2 is stronger than what the theorem proves, and the internal contradiction with Section 2.4.3.5 needs to be resolved. The empirical contributions are substantial and likely worth publishing after a careful revision that aligns the claims with the evidence. The editor may also wish to consider whether the thesis format fits the journal's scope or whether a condensed research-article version would be more appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this thesis. First, it contains real, large-scale experimental work that deserves attention: the Cat ensembles, Bonsai initialization, very-large dropout, and especially Memory Mosaics v2 are concrete contributions with results on external benchmarks like Camelyon17, RegBench, RULER, and standard language tasks. Second, the headline theoretical claim—that i.i.d. training 'lacks the motivation to construct a representation with rich features, regardless of model size and data size'—is not actually established by the paper's own theorem, and one section of the thesis contradicts it outright.\n\nThe core idea is worth taking seriously: expected-loss minimization under a single distribution P has no incentive to keep features that are redundant for P but useful under a shifted distribution Q. Theorem 2.1.6 only shows that all global minimizers of the idealized expected loss carry equivalent information under P. That says nothing about whether SGD can or will accumulate P-redundant features, and it says nothing about which minimizers SGD finds. The step to 'regardless of model size and data size' is an unsupported leap. The tension becomes explicit in Section 2.4.3.5, which argues that very-large dropout works precisely because i.i.d.-pretrained models contain rich features 'buried in their hidden layers' and that modern foundation models make this condition 'relatively easy to achieve.' That is a direct counterexample to the strong 'cannot' claim. What the theory plausibly shows is that the P-objective is indifferent to Q-useful features, not that such features are absent.\n\nWhat is genuinely new and well done: Memory Mosaics v2—adaptive bandwidth, gated time-variant key extraction, three-level memory—plus the large-scale evaluations. The results are plausible: comparable performance on persistent-knowledge benchmarks, clear gains on long-context QA after fine-tuning (12–15% over transformers), and context-length extrapolation without fine-tuning. The authors also transferred transformer hyperparameters verbatim to Memory Mosaics v2, which is a fair baseline choice. The predictive disentanglement chapter is more suggestive than proven—the three-moons toy is nice but not a demonstration of language-scale disentanglement—but it is presented as a principle, not a fait accompli.\n\nThe thesis does lean heavily on the author's own prior algorithms to validate its own principles. That is not disqualifying given the external benchmarks, but it does mean the broad framing should carry a heavier burden of independent replication.\n\nWho should read it: people working on OOD generalization, memory-augmented architectures, and in-context learning. It deserves a serious referee, not a desk reject. If it goes to a journal, the right request is not rejection but revision: soften the theoretical overclaim, address the internal contradiction, and separate the empirical findings from the three-principles manifesto.\n\nMy recommendation: engage with the work, cite the empirical results, but treat the 'i.i.d. training cannot produce rich features' claim as an open hypothesis rather than a proven theorem.","headline":"A serious, experiment-heavy thesis on open-world learning with a load-bearing theoretical claim that outruns its own theorem and its own empirical story.","tokens_in":54334,"tokens_out":1910,"would_cite":true,"duration_ms":20092,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The thesis claims that standard i.i.d. training cannot by itself produce representations rich enough for open-world AI, and that open-world AI requires three learning principles: rich features, disentangled representation, and…","keywords":["open-world AI","rich features","predictive disentanglement","inference-time learning","out-of-distribution generalization","memory-based models","in-context learning","representation learning"],"falsifier":"Train a single network wide enough to match the in-distribution linear-probing error of a concatenation of several separately seeded networks of the same architecture, then compare both by linear probing on a battery of held-out distribution shifts; if the single wide network matches or beats the concatenation on every shift, the thesis's central claim that i.i.d. training cannot build rich features regardless of model size would be empirically contradicted.","tokens_in":53167,"feed_emoji":"🧩","tokens_out":7523,"duration_ms":69213,"temperature":0.7,"pith_summary":"The thesis sets out to establish that success on one well-specified task does not transfer to open-world competence, where a machine must learn many new tasks quickly from few examples. Its central claim is that ordinary i.i.d. training—optimizing expected loss on a single distribution—has no incentive to build representations rich enough for such open-world competence, and that this is a structural limitation, not a matter of scale. The thesis argues that open-world AI requires three learning principles: rich features (a large toolbox), disentangled representation (an organized toolbox), and inference-time learning (a tool-savvy hand). It supports these principles with a linear-probing information framework, large-scale experiments in transfer learning, invariant learning, few-shot learning, and out-of-distribution fine-tuning, and language-model tests of memory-based architectures. If correct, the thesis says current foundation-model scaling on a single distribution is necessary but not sufficient, and that representation diversity, disentanglement, and memory-based adaptation must be engineered explicitly.","feed_headline":"I.I.D. training alone cannot build open-world AI","feed_subtitle":"A thesis identifies rich features, disentangled representations, and inference-time memorization as the missing learning principles.","key_machinery":"The central object is the representation $\\Phi$, a set of features evaluated by the optimal expected loss $C^*_P(\\Phi)$ of a linear classifier under a convex loss. Theorem 2.1.6—any two global optima of the expected loss carry equivalent information in this linear-probing sense—is the load-bearing identity: it implies that optimizing one i.i.d. objective gives no incentive to accumulate features that are redundant in-distribution, regardless of model or data size. The second mechanism is predictive disentanglement, the pressure created by minimizing the sum (area under the curve) of prediction losses across context lengths, which pushes each associative memory unit to produce a good conditional expectation estimate after as few time steps as possible and thereby split the task into independently memorized subtasks. The third mechanism is the associative memory unit implemented by Gaussian kernel smoothing, whose retrieval is a conditional expectation $E(V|K)$; Memory Mosaics v2 layers short-term, long-term, and persistent memories to decouple context-dependent and persistent knowledge, enabling inference-time storage and retrieval.","core_discovery":"On its own terms, the thesis claims that a deep network trained by i.i.d. empirical risk minimization has no motivation to retain features that are redundant on the training distribution, even when those features would become useful under a different distribution. The framework defines a representation as a set of features judged by the optimal cost of a linear classifier, and Theorem 2.1.6 shows that any two globally optimal representations carry equivalent linearly exploitable information about the training distribution; hence adding extra features cannot improve in-distribution cost. The thesis then shows experimentally that representations obtained from several independently seeded training episodes, concatenated as Cat or distilled as Distill, transfer substantially better to new tasks and distributions than an equivalently sized single network, and that fine-tuning with very large dropout preserves this richness. In invariant learning, it argues that what matters is the initial rich feature set (Bonsai) rather than the invariance penalties themselves, which are either too strong to optimize or too weak to enforce the desired constraints. For disentanglement, it introduces predictive disentanglement, a training pressure that splits a prediction task into independently memorized subtasks, and grounds it in a reduction of sample complexity from linear to logarithmic dependence on representation dimension. For inference-time learning, it proposes memory-based architectures such as Memory Mosaics v2, which store and retrieve new knowledge at inference time, and reports that they outperform transformers on multi-document question answering, context-length extrapolation, and in-context classification while matching them on standard persistent-knowledge benchmarks.","pith_inferences":["Beyond the thesis, the argument predicts that out-of-distribution robustness will systematically trade off against i.i.d. efficiency whenever regularizers prune redundant features; a benchmark that scores both would test whether that trade is fundamental or an artifact of current optimizers.","The thesis's mechanism suggests a testable extension for standard transformers: if predictive disentanglement is a general training pressure, plotting per-position loss curves on long contexts should reveal whether attention heads specialize by context length, as Memory Mosaics do.","The context-length extrapolation of Memory Mosaics v2 without fine-tuning implies that position encoding, not attention itself, may be the main bottleneck for length generalization; removing or replacing positional information in transformer-like models is a direct next experiment.","Read as a research program, this thesis implies that open-world evaluation should emphasize new-knowledge storage and retrieval and few-shot in-context adaptation under scarce examples, rather than aggregate accuracy on benchmarks that mostly test persistent knowledge."],"forward_implications":["Simply scaling model and data under one i.i.d. distribution will not yield open-world competence; representation diversity must be explicitly produced, for example by combining independently trained networks or distilling them into one.","Fine-tuning a rich representation with a very high dropout rate (90% or more) becomes a practical tool for out-of-distribution robustness, outperforming ensembles and weight averaging in the reported settings.","In invariant learning, effort should shift from inventing invariance penalties to constructing rich initial features; a frozen rich representation can outperform trained invariance penalties.","Disentanglement lowers the number of examples needed to learn new tasks by reducing sample complexity from linear to logarithmic dependence on representation dimension.","Memory-based architectures that store and retrieve new knowledge at inference time can adapt to new tasks and long contexts without catastrophic forgetting and without position-encoding-dependent extrapolation failures."],"supporting_citations":[{"why":"Supplies the feature/representation framework and the theorem that i.i.d. training lacks motivation to build redundant features.","marker":"[Zhang and Bottou 2023]"},{"why":"Introduces the Bonsai algorithm for constructing rich features by adversarial discovery and synthesis, used throughout the invariant-learning case study.","marker":"[Zhang et al. 2022]"},{"why":"Introduces Memory Mosaics and predictive disentanglement, which form the basis of the disentangled representation and inference-time learning chapters.","marker":"[Zhang et al. 2025]"},{"why":"Provides the sample-complexity results that disentanglement reduces the examples needed from linear to logarithmic dependence on representation dimension.","marker":"[Ng 2004]"},{"why":"Provides the classic argument that memory-based or locally weighted learning avoids negative interference between old and new tasks, motivating inference-time learning.","marker":"[Atkeson et al. 1997]"},{"why":"Supplies the RegBench benchmark for in-context learning where Memory Mosaics outperform transformers and other architectures.","marker":"[Akyürek et al. 2024]"},{"why":"Provides the TinyStories approach that underlies the BabiStories language-model experimental setup.","marker":"[Eldan and Li 2023]"},{"why":"Supplies the WILDS Camelyon17 benchmark and evaluation protocol used for the real-world invariant-learning experiments.","marker":"[Koh et al. 2021a]"}],"fun_headline_variants":["Open-world AI needs more than i.i.d. training","Why i.i.d. training fails for open-world AI","Rich features and disentanglement unlock open-world AI","Inference-time learning: key principle for open-world AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a theorem about idealized optimizers—global optima of an expected convex loss probed by a linear classifier—describes what stochastic gradient descent actually does in deep networks, so that features redundant on the training distribution are precisely the features that help under a shifted distribution.","fun_headline_variants_meta":{"raw":{"variants":["Open-world AI needs more than i.i.d. training","Why i.i.d. training fails for open-world AI","Rich features and disentanglement unlock open-world AI","Inference-time learning: key principle for open-world AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000222,"raw_usage":{"total_tokens":1561,"prompt_tokens":1158,"completion_tokens":403,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":774,"completion_tokens_details":{"reasoning_tokens":338}},"tokens_in":774,"tokens_out":403,"duration_ms":3666,"temperature":1.0,"reasoning_tokens":338,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:41:52.867709+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a single network wide enough to match the in-distribution linear-probing error of a concatenation of several separately seeded networks of the same architecture, then compare both by linear probing on a battery of held-out distribution shifts; if the single wide network matches or beats the concatenation on every shift, the thesis's central claim that i.i.d. training cannot build rich features regardless of model size would be empirically contradicted.","supporting_citations":[{"cited_title":"Rich feature construction for the optimization-generalization dilemma","cited_arxiv_id":null,"evidence_quote":"Introduces the Bonsai algorithm for constructing rich features by adversarial discovery and synthesis, used throughout the invariant-learning case study."}],"review_version":1}