REVIEW 4 major objections 4 minor 20 references
Less is More: some Computational Principles based on Parcimony, and Limitations of Natural Intelligence
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Constraints in brains are catalytic: less data and energy can produce more capable intelligence.
desk verdict A well-written synthesis of established ideas making a plausible but under-supported 'less is more' argument; worth reading as a perspective, not as a research result. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The unifying mechanism is constraint-driven parsimony, expressed in three concrete computational vehicles. First, Shannon's source coding theorem links input complexity $R_X$ to low-dimensional codes $R_W$ via $\log R_X \approx k \log R_W$, making entropy maximization an internal drive toward compression. Second, chaotic itinerancy—weakly chaotic dynamics where inhibitory neurons mask currently attended memories—lets the system visit quasi-attractors described as Milnor attractors and flexibly chain memories without catastrophic forgetting. Third, reservoir computing uses fixed random projections to produce a high-dimensional nonlinear expansion of inputs, from which a linear readout selects useful combinations; the Johnson–Lindenstrauss lemma is invoked to justify why random projections preserve structure while being nearly free, enabling rapid generalization from small datasets. These three mechanisms are framed as complementary angles on the same 'less is more' principle.
What would settle it
A decisive observation would be a small-sample language task in which a data-rich transformer consistently improves as training data grows while a reservoir model plateaus, contradicting the claim that constraints drive efficiency. Alternatively, a demonstration that a reservoir with random projections cannot learn a simple compositional rule (e.g., the Hanoi Tower rule) from few examples would falsify the bootstrapping-abstraction prediction.
Extended reading notes
Core claim
The central claim is that constraints in natural intelligence are paradoxically catalytic: limited neural bandwidth, energy, and data drive the emergence of concise codes, hierarchical structure, chaotic itinerancy, and active embodied learning, which together enable rapid generalization from sparse experience. The authors establish, through a synthesis of prior work and their own experiments, that low-dimensional serial-order and hierarchical codes compress inputs while preserving structure; that chaotic itinerancy lets networks transition among memory attractors and maintain information over long timescales; and that reservoir computing's random projections act like a kernel trick that bootstraps abstraction with little data. The conclusion is prescriptive: AI should be designed to operate under genuine limits rather than through unbounded scaling, because those limits are what make natural intelligence fast, flexible, and energy-frugal.
Load-bearing premise
The load-bearing premise is that random projections in reservoir computing are enough to bootstrap fast, generalizable abstraction from very small datasets, and that this mechanism is general enough to serve as a principle of natural intelligence; the paper relies on a few experiments, several from the authors' own group, without specifying when random projections fail or which hyperparameter conditions are required.
Editorial extensions
If this is right
- If constraints are catalytic, AI systems that deliberately limit data, energy, or network capacity could match or beat data-hungry models on small-sample tasks, particularly in language acquisition.
- Reservoir computing could serve as an energy-efficient alternative to backpropagation-based training, especially for short high-dimensional time series and embodied agents.
- Incorporating active learning, intrinsic motivation, and caregiver-like scaffolding could make language-learning agents converge faster and acquire meaning rather than mere statistical associations.
- Low-dimensional serial-order codes and small 'mini-reservoirs' could be combined to enable compositional planning and rule learning without exponential data demands.
- The proposed 'reservoir map'—a predictive atlas linking hyperparameter regions and physical media to classes of tasks—could make physical reservoir computing practical for industrial applications.
Reading between the lines
- The paper's thesis implies a concrete research program: benchmark constrained architectures against scaled ones under matched data budgets, and look for Pareto improvements in efficiency where constraints help.
- A testable hypothesis follows from the bootstrapping-abstraction argument: on tasks with low-dimensional underlying rules and scarce data, a small reservoir with random projections should outperform a larger trained model.
- The philosophical discussion suggests a probeable idea in robotics: agents with finite energy and time budgets should develop more robust and self-directed behavior than agents with unlimited resources.
- The authors leave open whether 'less is more' breaks down for tasks that require broad world knowledge, so a boundary condition would be where scaling clearly outperforms constraint-based learning.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that biological constraints — energy, bandwidth, data scarcity, and embodiment — are not merely limitations but computational catalysts. It surveys four candidate mechanisms: efficient low-dimensional coding (§2.1), chaotic itinerancy (§2.2), reservoir computing based on random projections (§2.3), and active social learning (§2.4). It concludes by recommending that AI adopt 'less is more' principles such as energy constraints, parsimonious architectures, and real-world interaction, claiming this could lead to more efficient, interpretable, and biologically grounded artificial systems.
Significance. If the thesis is correct, it offers a principled alternative to the scaling paradigm and gives a concrete research direction for data-efficient, energy-efficient AI. The paper is a valuable interdisciplinary synthesis, and the 'Reservoir Map' proposal in §2.3 is a concrete, falsifiable roadmap. The cited examples are not fitted with bespoke parameters, which is a reproducibility strength. However, the central evidence is a small set of selected simulations, several from the authors' own prior work, with no reported failure cases and no formal derivation for the key information-theoretic relation. The paper therefore establishes a coherent and interesting hypothesis rather than a demonstrated general principle.
major comments (4)
- [§2.1, paragraph "Efficient coding by information suppression"] The relation log RX ≈ k log RW is presented as following from Shannon's source coding theorem and is used as the formal support for the claim that bandwidth constraints promote concise codes. No derivation is given, the symbols RX and RW are not precisely defined beyond 'data complexity' and 'information capacity', and k is not specified as a function of the code or of the approximation error. Because this equation is load-bearing for the efficient-coding argument, it needs either a real derivation with stated conditions or an explicit reframing as an analogy rather than a mathematical consequence.
- [§2.3, subsection "Bootstrapping abstraction"] The claim that random projections bootstrap rapid generalization from small data rests on three cited studies: cross-situational word learning with 1000 sentences (JH20, VH20), an RL preprocessing study (LHNHM24), and COVID-19 forecasting with 400 days and 400 features (FDH+24). No failure cases, hyperparameter ranges, reservoir sizes, spectral radii, or input scalings are reported, and the section itself concedes that 'we still lack precise mathematical explanations for the practical success of RC.' This makes the extrapolation to a general biological principle untested. To support the central 'less is more' thesis, the authors should specify the conditions under which the RC advantage disappears and compare against unconstrained baselines with matched data and compute.
- [§2.2, subsection "Superiority of Chaotic Itinerancy"] The paragraph claims that weak chaos maintains information over long time scales and that the proposed neural learning is highly efficient, requiring only a few hundred to 0.1 million neurons and hours of learning. No quantitative comparison with standard recurrent networks, reservoirs, or transformers is provided, and the claim that chaotic itinerancy supports flexible memory retrieval under uncertainty is asserted rather than demonstrated. The authors should either provide a controlled comparison or clearly label this as a hypothesis that is not yet supported by the evidence presented.
- [§2.4, subsection "Active learning during infant's language development"] The developmental claims — that intrinsic motivation and caregiver responsiveness accelerate language learning — are supported only by the authors' own simulations (COH18, LEM22, MAR23) without comparison to passive-exposure baselines or to the data regimes of current large language models. Since the paper recommends real-world interaction as a 'less is more' mechanism, the authors should report at least one controlled study that varies the presence or absence of scaffolding/intrinsic motivation, or soften the claim to a research program rather than a demonstrated principle.
minor comments (4)
- [Title and Abstract] The word 'Parcimony' is misspelled; it should be 'Parsimony.'
- [Throughout] There are several typos, including 'stastistical', 'constrast', 'circumbscribed', and 'standart'; a careful proofread is needed.
- [§3, Discussion] The philosophical and aesthetic passages (Bauhaus, Berque, Borges, the Hard Problem of life and consciousness) are not explicitly connected to the computational claims; either make the connection explicit or condense these passages to short remarks.
- [§2.3, subsection "Reservoirs of computations"] The 'Reservoir Map' roadmap is intriguing but remains a single paragraph; a small figure, pseudocode, or a concrete example task would make the proposal actionable and testable.
Circularity Check
No significant circularity: the thesis is a conceptual synthesis, and its supporting experiments (including self-cited ones) are external, falsifiable benchmarks rather than fitted inputs or definitional equivalences.
full rationale
The paper offers no quantitative derivation chain: it is a perspective and synthesis piece. The closest thing to a derivation is Section 2.1's efficient-coding discussion, where 'abstraction' is deliberately assimilated to information reduction and Shannon's source-coding bound log RX ≈ k log RW is quoted; this is a framing or interpretation of a known theorem, not a prediction derived from the paper's own fitted quantities. Section 2.3's claim that random projections bootstrap generalization is supported by prior experiments [JH20, VH20, LHNHM24, FDH+24], several from the same group; these are external, falsifiable benchmark results (cross-situational word learning, RL preprocessing, COVID-19 forecasting) and are not re-fit inside this paper, so they do not constitute fitted inputs renamed as predictions. The chaotic-itinerancy discussion invokes prior simulation work [WT20, YT21, TYW16] by an author, but again as empirical or simulation support, not as a self-referential uniqueness theorem. The paper even concedes 'we still lack precise mathematical explanations for the practical success of RC,' which is an acknowledged limitation rather than a circular step. Concerns about generalizability or the absence of failure cases are evidence-quality and correctness issues, not circularity. No equation in the paper reduces to its own input by construction, and no fitted parameter is renamed as a prediction.
Assumptions & free parameters
assumptions (6)
- domain assumption The brain operates under severe energy and bandwidth constraints that shape its computational strategy.
- domain assumption Entropy maximization is the coding objective that compresses information in neural populations.
- standard math Shannon's source coding theorem can be applied to neural codes, giving log RX ≈ k log RW.
- domain assumption Random projections preserve useful structure and can bootstrap generalization from few examples.
- domain assumption Chaotic itinerancy is a viable model for flexible memory retrieval and decision-making under uncertainty.
- domain assumption Infants acquire core knowledge and language through intrinsic motivation and responsive caregivers.
Cite this review
Pith. "Pith review of Less is More: some Computational Principles based on Parcimony, and Limitations of Natural Intelligence." pith.science (2026). https://pith.science/paper/6MPC4WTG
@misc{pith2026250607060,
author = {Pith},
title = {Pith review of: Less is More: some Computational Principles based on Parcimony, and Limitations of Natural Intelligence},
year = {2026},
howpublished = {\url{https://pith.science/paper/6MPC4WTG}},
note = {Machine review of arXiv:2506.07060}
}
read the original abstract
Natural intelligence (NI) consistently achieves more with less. Infants learn language, develop abstract concepts, and acquire sensorimotor skills from sparse data, all within tight neural and energy limits. In contrast, today's AI relies on virtually unlimited computational power, energy, and data to reach high performance. This paper argues that constraints in NI are paradoxically catalysts for efficiency, adaptability, and creativity. We first show how limited neural bandwidth promotes concise codes that still capture complex patterns. Spiking neurons, hierarchical structures, and symbolic-like representations emerge naturally from bandwidth constraints, enabling robust generalization. Next, we discuss chaotic itinerancy, illustrating how the brain transits among transient attractors to flexibly retrieve memories and manage uncertainty. We then highlight reservoir computing, where random projections facilitate rapid generalization from small datasets. Drawing on developmental perspectives, we emphasize how intrinsic motivation, along with responsive social environments, drives infant language learning and discovery of meaning. Such active, embodied processes are largely absent in current AI. Finally, we suggest that adopting 'less is more' principles -- energy constraints, parsimonious architectures, and real-world interaction -- can foster the emergence of more efficient, interpretable, and biologically grounded artificial systems.
Reference graph
Works this paper leans on
-
[9]
The neonate brain detects speech structure
[JOH84] Johnson, William B Extensions of Lipshitz mapping into Hilbert space, Contemporary mathematics, (26)189–206, 1984, [GMC+08] Judit Gervain, Francesco Macagno, Silvia Cogoi, Marcela Pe˜ na, and Jacques Mehler. The neonate brain detects speech structure. Proceedings of the National Academy of Sciences, 105(37):14222–14227,
work page 1984
-
[15]
Informational Embodiment: Computational role of information structure in codes and robots
arXiv:2408.12950 [cs]. 17 [PQL+22] Alexandre Pitti, Mathias Quoy, Catherine Lavandier, Sofiane Boucenna, Wassim Swaileh, and Claudio Weidmann. In Search of a Neural Model for Serial Order: a Brain Theory for Memory Development and Higher-Level Cognition. IEEE Transactions on Cognitive and Developmental Systems , 10.1109/TCDS.2022.3168046,
-
[16]
[PWL24] Alexandre Pitti, Claudio Weidmann, and Krzysztoff Lebodia
eprint: https://www.pnas.org/doi/pdf/10.1073/pnas.2115335119. [PWL24] Alexandre Pitti, Claudio Weidmann, and Krzysztoff Lebodia. Doing More with Less: Computational Role of Information Structure in Neural Networks based on Entropy Max- imization 1st Workshop on NeuroAI @ 38th Conference on Neural Information Processing Systems (NeurIPS 2024). , [RAM21] H....
-
[18]
[St21] Stahl, B. C. (2021). Artificial intelligence for a better future: An ecosystem perspective on the ethics of AI and emerging digital technologies. Springer. 18 [SZ75] J. Szent´ agothai. The ‘module-concept’ in cerebral cortex architecture. Brain Re., 95:475- 496,
work page 2021
-
[1983]
[Dag00] F. Dagognet. Le plus dans le moins . Les cahiers de m´ ediologie 9, no. 1 (2000): 5–11. https://doi.org/10.3917/cdm.009.0005 [DAJ95] Peter Dominey, Michael Arbib, and Jean-Paul Joseph. A model of corticostriatal plasticity for learning oculomotor associations and sequences. Journal of cognitive neuroscience , 7(3):311–336,
-
[1987]
p.52 [Ber92] A. Berque. L’´ ecosymbole du tatami. L’Homme et la soci´ et´ e 104, no. 2 (1992): 7–14.http: //doi.org/10.3406/homso.1992.2635.p.8 [Ber13] A. Berque. Aida et Ma: De Ce Que Sont Les Choses Dans La Spatialit´ e Japonaise . Pour Un Vocabulaire de La Spatialit´ e Japonaise 43 (2013): 67–74. https://doi.org/15055/ 00002227 13 [BJ89a] P Barone and ...
-
[1989]
[Kor98] A. Korzybski. Une Carte n’est Pas Le Territoire. Prol´ egom` enes Aux Syst` emes Non- Aristot´ eliciens et ` a La S´ emantique G´ en´ erale (1933). Paris:´Editions de l’´ eclat, 1998 [KOZ07] R. Kozma, H. Aghazarian, T. Huntsberger, E. Tunstel, and W. J. Freeman. Computa- tional aspects of cognition and consciousness in intelligent devices. IEEE Co...
work page 1933
-
[1991]
pp.309-317. [La94] B. Latour (1994). We have never been modern (3. print.). Harvard Univ. Press. [LDHD22] Yair Lakretz, Th´ eo Desbordes, Dieuwke Hupkes, and Stanislas Dehaene. Can Transform- ers Process Recursive Nested Constructions, Like Humans? In Proceedings of the 29th International Conference on Computational Linguistics , pages 3226–3232, Gyeongju...
work page 1994
Show all 20 references
-
[1995]
[Dro19] M. Droste. Bauhaus. 1919-1933. K¨ oln: Taschen,
1919
-
[1998]
bibitem[Bra16]Brandon2016 C. Brandon. L’Entre [Corps/Machine]: La Princesse et Son Mac. Theses, Paris 1 - Panth´ eon-Sorbonne, 2016.https://hal.archives-ouvertes.fr/ tel-02111485. [Buc01] C. Buci-Glucksmann, L’esth´ etique du temps au Japon, Du Zen au Virtuel, (Galil´ ee, 2001...
2016
-
[2001]
Schmidhuber, Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010), in IEEE Transactions on Autonomous Mental Development, vol
[Sch10] J. Schmidhuber, Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010), in IEEE Transactions on Autonomous Mental Development, vol. 2, no. 3, pp. 230-247, Sept. 2010 [Sim58] G. Simondon, L’individuation. `A la lumi` ere des notions de forme et d’informa...
1990
-
[2003]
[Ar51] Arendt, H. (2017). The origins of totalitarianism (1951). Penguin Classics. [A WY99] K. Akimoto, S. Watanabe, and M. Yano. An insect robot controlled by the emergence of gait patterns. Artif. Life Robotics, 3:102–105,
2017
-
[2011]
2013 Life as we know it
[Fri13] Friston K. 2013 Life as we know it. J R Soc Interface 10: 20130475. [GOL19] R. M. Golinkoff, E. Hoff, M. L. Rowe, C. S. Tamis-LeMonda, and K. Hirsh-Pasek. Lan- guage matters: Denying the existence of the 30-million-word gap has serious consequences. Child Dev. , 90(3):985–992,
2013
-
[2012]
P´ erilli´ e
[Per04] J-L. P´ erilli´ e. ”Summetria” des Nombres de la ”R´ epublique”. Revue Philosophique de Lou- vain 102, no. 1 (2004): 35–58. https://doi.org/10.2143/RPL.102.1.503706. p.36 [PIT09] A Pitti, H Mori, S Kouzuma, Y Kuniyoshi. Contingency perception and agency measure in visu...
2004 doi
-
[2019]
Universal distributed sensing via random projections
[Dua06] Duarte, Marco F and Wakin, Michael B and Baron, Dror and Baraniuk, Richard G. Universal distributed sensing via random projections. Proceedings of the 5th international conference on Information processing in sensor networks, 177–185, 2006 [ENG17] D. F. English, S. McK...
2006
-
[2020]
A journey in ESN and LSTM visualisations on a language task
[VH20] Alexandre Variengien and Xavier Hinaut. A journey in ESN and LSTM visualisations on a language task. arXiv preprint arXiv:2012.01748 ,
2012 arXiv
-
[2021]
Cross-situational learning with reservoir computing for language acquisition modelling
[JH20] Alexis Juven and Xavier Hinaut. Cross-situational learning with reservoir computing for language acquisition modelling. In 2020 International Joint Conference on Neural Networks (IJCNN) , pages 1–8. IEEE,
2020
-
[2022]
Serial Order Codes for Dimensionality Reduction in the Learning of Higher-Order Rules and Compositionality in Planning
16 [LPMK24] Krzysztof Lebioda, Alexandre Pitti, Fabrice Morin, and Alois Knoll. Serial Order Codes for Dimensionality Reduction in the Learning of Higher-Order Rules and Compositionality in Planning. In Michael Wand, Krist ´ ına Malinovsk´ a, J¨ urgen Schmidhuber, and Igor V. ...
2024
-
[2023]
[FF12] W
arXiv:2311.10300 [cs]. [FF12] W. T. Fitch and A. D. Friederici. Artificial grammar learning meets formal language the- ory: An overview. Philosophical Transactions of the Royal Society B: Biological Sciences , 367:1933–1955,
1933 arXiv
-
[2024]
[MA85] J
arXiv:2410.05229 [cs]. [MA85] J. Milnor. On the concept of attractor. Commun. Math. Phys. , 99:177-195,
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.