Pith. sign in

REVIEW 4 major objections 5 minor 16 references

A Compositional Paradigm for Foundation Models: Towards Smarter Robotic Agents

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that composing small, specialized models—rather than scaling one monolithic foundation model—lets agents match or beat much larger systems in both vision and robotic manipulation while using a fraction of the training budg

desk verdict The direction is sensible, but the evidence is almost entirely self-cited prior work, and the robot win is uninterpretable as reported. read the letter →

arxiv 2510.18608 v1 pith:6HZ67X3N submitted 2025-10-21 cs.RO cs.LG

Luigi Quarantiello , Elia Piccoli , Jack Bell , Malio Li , Giacomo Carfì , Eric Nuertey Coleman , Gerlando Gramaglia , Lanpei Li
show 3 more authors
Mauro Madeddu Irene Testa Vincenzo Lomonaco
This is my paper
classification cs.ROcs.LG
keywords foundationmodelscontinuallearningcompositionalityLoRAadaptershierarchicaladaptermergingroboticmanipulationvision-language-actionparameter-efficientfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that foundation models do not have to be retrained or scaled to keep learning: composing multiple small, specialized modules can yield stronger performance with far less compute. In image classification on CUB200 with 50 tasks, a hierarchical adapter merging method reaches 55.17% accuracy, beating SD-LoRA (47.56%) and InfLoRA (36.02%) while training faster. In robotic manipulation, a small composed agent achieves a 0.91 success rate after 14 hours of training, compared with 0.0 for OpenVLA and InstructRL when those baselines are given the same computational budget. The authors read these results as evidence that compositionality and continual learning are a viable alternative to the scaling-only paradigm for developing adaptive, efficient AI.

What carries the argument

Two mechanisms carry the argument. The first is hierarchical adapter merging (HAM): for each task, a separate LoRA adapter—a small set of low-rank trainable matrices—is trained, then merged with previously learned adapters first within similarity groups and then across groups into a single module; this ordering is what the paper says reduces merging interference. The second is WSA, a lightweight robotic architecture that combines small off-the-shelf pre-trained models with small adapters and an attention mechanism to dynamically scale pre-trained components to the current context. Both are instances of the same thesis: compose rather than scale.

What would settle it

Train OpenVLA or InstructRL on the same, named manipulation task with their intended full compute budgets (not capped at 14 hours) and measure success rate; if either exceeds 0.91, the composed agent's win is a resource-starvation artifact. Alternatively, run WSA on a standard multi-task manipulation benchmark and compare against published baseline success rates.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that compositionality rescues foundation models from task rigidity and prohibitive retraining cost. The authors demonstrate this in two settings: a vision system that trains a separate LoRA adapter for each task and merges them hierarchically, first within similarity groups and then into one module, reducing interference and improving accuracy; and a robotic manipulation agent built from small pre-trained components, small adapters, and an attention mechanism that learns a manipulation skill in 14 hours with a 0.91 success rate, while OpenVLA and InstructRL under identical compute constraints score 0.0. The unifying claim is that a modular, continua

Load-bearing premise

The robotic result assumes that giving OpenVLA and InstructRL the same tiny training budget as the small agent is a fair comparison, and that the unnamed robot task is a meaningful test; if either is not true, the 0.91 vs 0.0 success gap does not prove the composed agent is better.

Editorial extensions

If this is right

  • If HAM holds up, adapting a foundation model to a new task becomes as cheap as training one small adapter and merging it, enabling continual learning without architectural changes or full retraining.
  • If WSA's robotic result is accurate, robot skill acquisition could drop from days of GPU time to hours, making learning-based manipulation practical for smaller research groups and edge robots.
  • The hierarchical merging order specifically reduces interference, implying that merge order matters and can be chosen to preserve performance across unrelated tasks.
  • The same compositional design should extend to any modality with a frozen backbone and small trainable adapters, since it does not rely on task-specific architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves the manipulation environment unnamed and does not specify the action or observation space; until those are disclosed, the 0.91 vs 0.0 success gap cannot be independently reproduced, so the obvious next step is to run WSA on a public multi-task manipulation benchmark with adequately provisioned baselines.
  • The motivation that scaling has hit diminishing returns is borrowed from a single contested preprint; even if that premise is wrong, the compositional mechanism could still be useful as a complementary strategy, a possibility the paper does not discuss.
  • If adapter merging truly resists interference, a natural testable extension is to apply HAM to large language models on sequential instruction-tuning tasks and measure forgetting against existing continual-learning baselines.
  • The paper implicitly assumes a single composed model must solve every seen task; an alternative design that routes each input to the relevant adapter might trade a little memory for even less interference, but that route is not explored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a compositional paradigm for foundation models, combining continual learning and compositionality to adapt to dynamic tasks. It presents two sets of experiments: (1) CUB200 image classification with hierarchical merging of LoRA adapters (HAM), reporting 55.17% accuracy vs 47.56% for SD-LoRA; (2) robotic manipulation with a 'WSA' architecture that reportedly achieves 0.91 success rate versus 0.0 for OpenVLA and InstructRL under apparently lower training time. The conclusion claims that composing multiple models enables strong performance with limited resources.

Significance. If the results held, the paper would support a promising and cost-efficient alternative to monolithic scaling for adapting foundation models, with implications for robotics. The compositional continual-learning direction is timely. However, the experimental support is currently not independently verifiable: the robot comparison is underspecified and likely confounded by unequal resources; the image experiments are described too briefly; and both sets of results appear to be drawn from the authors' own prior papers without appropriate disclosure or artifact release. The paper would be valuable as a research proposal or manifesto, but as a technical paper it needs substantial revision.

major comments (4)
  1. [Section IV, Table II] The robotic manipulation comparison is not interpretable. The task, environment, observation and action spaces, dataset size, number of demos, and evaluation protocol are not named. 'Reward/Step' is not defined, and no error bars or seeds are given. Critically, the text claims the baselines fail 'given the same computational constraints,' but Table II reports 14 h vs 40 h and 92 h training times, with hardware unspecified; this is consistent with resource under-provisioning of OpenVLA/InstructRL. The central success claim (0.91 vs 0.0) therefore rests on an unverifiable and possibly confounded comparison. Please specify exact compute, training steps, and provide per-seed results and learning curves.
  2. [Section IV, Table I] The image-classification claim is underspecified: '50 tasks on CUB200' does not state how the 200 classes were split (e.g., 2-class incremental tasks), the task order, or whether this is class-incremental or task-incremental evaluation. Only mean accuracy is reported; no per-task accuracy, forgetting, or task-order ablation is given. Without these details, the 55.17 vs 47.56 advantage cannot be assessed as a continual-learning result. Provide the full protocol and, ideally, release code.
  3. [Section IV (attribution)] The experimental evidence is not original to this submission but is taken from the authors' prior works [5] and [12], with the framing from [3]; none of this is disclosed in Section IV. This creates a circularity/verifiability problem: a reader cannot separate the claimed results from self-citation. Please state explicitly which results are new, and either include full descriptions from the prior papers or provide independent re-runs with artifacts.
  4. [Section III] The premise that scaling has hit diminishing returns is asserted from a single preprint [10] and presented as fact without engaging counter-evidence. Since the paper's motivation is that compositionality outperforms scaling, this load-bearing assumption needs stronger support or a more nuanced statement.
minor comments (5)
  1. [Abstract/Introduction] Reference [1] does not seem to be about foundation models generally (it is 'Learning interactive real-world simulators'); please check relevance.
  2. [Section I] Typo: 'we propose a alternative vision' should be 'we propose an alternative vision'.
  3. [Section IV] Typo: 'where its clear' should be 'where it is clear'.
  4. [General] There is no related-work section positioning the paper with respect to other compositional/continual learning methods; consider adding one.
  5. [Table II] Define 'Reward/Step' and specify how success rate is measured and over how many episodes.

Circularity Check

2 steps flagged · score 6.0 of 10

Both experimental tables are imported from the authors' own prior papers ([5], [12]) and presented as 'Ours'; the central claim therefore rests on a self-citation chain rather than on evidence derived in this manuscript.

  1. self citation load bearing [Section IV, Table I and surrounding text ("We employ a hierarchical merging process ... [5]")]
    "We employ a hierarchical merging process: a first combination is performed within LoRAs grouped by similarity, while group adapters are later merged into a single module, so to have a unified model capable of solving every seen task [5]. ... Table I shows the results obtained on CUB200 dataset with 50 tasks: our methods achieves an higher accuracy, while being faster in the training process."

    Table I's 'HAM (Ours)' row is the headline result of [5] (Coleman et al., 'Ham: Hierarchical adapter merging for scalable continual learning'), a paper co-authored by the present authors. This manuscript contributes no new experiment, data, code, or analysis for that table; the 'our method' result is imported wholesale from the authors' own prior work. Thus the image-classification evidence for the central compositional claim reduces to a self-citation rather than an independent demonstration in this paper.

  2. self citation load bearing [Section IV, Table II and surrounding text ("In the context of robotics applications, we propose a simple architecture ... [12]")]
    "In the context of robotics applications, we propose a simple architecture that using small adapters and attention mechanism can dynamically scale agents' pre-trained components while adapting to different contexts [12]. The results show that agents using small off-the-shelf pre-trained models and a small computational budget are able to learn to effectively solve robotics manipulation task while more complex counterparts as OpenVLA fail (given the same computational constraints)."

    Table II's 'WSA (Ours)' row (Reward/Step 0.60, Success Rate 0.91, 14h) is the result of [12] (Piccoli et al., 'Combining pre-trained models for enhanced feature representation in reinforcement learning', CoLLAs 2025), also co-authored by the present authors. The claim that the method outperforms OpenVLA and InstructRL is asserted here only by citing that prior work; no environment details, seeds, independent reruns, or error bars are supplied in this manuscript. The robot-manipulation support for the main conclusion 'composing multiple models enables strong performance with limited resources' is therefore the authors' own prior result, making the self-citation load-bearing.

full rationale

The manuscript contains no derivation or first-principles argument; it is a short position/experimental summary. The two quantitative claims—image classification accuracy on CUB200 and robot manipulation success—are not computed or evaluated in this paper. Table I's 'HAM (Ours)' row is the result of [5] (Coleman et al.), and Table II's 'WSA (Ours)' row is the result of [12] (Piccoli et al.); both references share authors with the present paper, and no code, data, checkpoints, or independent reruns are provided. The conclusion 'we show that composing multiple models enables strong performance with limited resources' is thus supported by a self-citation chain rather than by independent evidence in this manuscript. This is load-bearing because the entire empirical case rests on those two tables. I am not claiming the underlying prior results are fabricated; the circularity is that the present paper's demonstration reduces to its authors' earlier, not externally verified here, results. There is also a non-circular validity concern: the 'same computational constraints' statement in Section IV appears to conflict with Table II's training times (14h vs 40h/92h), and the robot environment is unnamed; however, that is an evaluation-quality issue rather than a definitional circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on: (1) the fairness premise of the equal-compute robot comparison, which the manuscript does not justify; (2) the diminishing-returns premise asserted in §III from a single contested preprint; (3) the assumption that adapter composition preserves task knowledge (interference is the known failure mode, only asserted to be reduced); and (4) unspecified evaluation choices (50-task CUB200 split; unnamed robot environment). The paper ships no code, data, or derivation, so every one of these premises is taken on faith. No new entities are invented: HAM and WSA are methods from the authors' prior papers, recycled here.

free parameters (3)
  • Adapter merging weights / grouping similarity threshold (HAM)
    The hierarchical merge in Table I depends on a similarity grouping criterion and merge coefficients from [5]; neither is given, and the result is sensitive to them.
  • WSA architecture hyperparameters
    Adapter sizes, attention configuration, and number of composed modules for the robot agent are from [12] and not specified here; the reported success rate presumably depends on them.
  • Task curriculum / number of training episodes for robot manipulation
    The paper reports reward/step and success rate but no environment, episode count, or task split; these would determine both numbers.
assumptions (4)
  • domain assumption Scaling of models/datasets is at diminishing returns
    Section III asserts this as the motivation for a paradigm shift, citing only a single arXiv preprint [10]; the claim is contested and not derived in this paper.
  • domain assumption Composition of independently trained adapters preserves and combines task knowledge without destructive interference
    The whole HAM result assumes merging adapted weights yields a single model that solves every seen task; interference is the known failure mode and is only asserted to be reduced, not measured (§IV).
  • domain assumption Equal compute budget is the correct comparison protocol for evaluating method quality
    Table II contrasts WSA (14h) with OpenVLA (92h) and InstructRL (40h) but notes baselines 'fail (given the same computational constraints)'; the fairness of this protocol is assumed, not argued.
  • domain assumption CUB200 split into 50 tasks is a valid continual-learning evaluation
    Section IV reports 'CUB200 dataset with 50 tasks' without defining class-to-task assignment or order; the difficulty of the benchmark depends on this unspecified choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Compositional Paradigm for Foundation Models: Towards Smarter Robotic Agents." pith.science (2026). https://pith.science/paper/6HZ67X3N

@misc{pith2026251018608,
  author       = {Pith},
  title        = {Pith review of: A Compositional Paradigm for Foundation Models: Towards Smarter Robotic Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6HZ67X3N}},
  note         = {Machine review of arXiv:2510.18608}
}
read the original abstract

The birth of Foundation Models brought unprecedented results in a wide range of tasks, from language to vision, to robotic control. These models are able to process huge quantities of data, and can extract and develop rich representations, which can be employed across different domains and modalities. However, they still have issues in adapting to dynamic, real-world scenarios without retraining the entire model from scratch. In this work, we propose the application of Continual Learning and Compositionality principles to foster the development of more flexible, efficient and smart AI solutions.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 6 linked inside Pith

  1. [3]

    The future of continual learning in the era of foundation models: Three key directions

    Jack Bell et al. The future of continual learning in the era of foundation models: Three key directions. arXiv preprint arXiv:2506.03320, 2025

    arXiv 2025

  2. [5]

    Ham: Hierarchical adapter merging for scalable continual learning

    Eric Nuertey Coleman et al. Ham: Hierarchical adapter merging for scalable continual learning. arXiv preprint arXiv:2509.13211, 2025

    arXiv 2025

  3. [12]

    Combining pre-trained models for enhanced feature representation in reinforcement learning

    Elia Piccoli et al. Combining pre-trained models for enhanced feature representation in reinforcement learning. CoLLAs, 2025

    2025

  4. [10]

    Has llm reached the scaling ceiling yet? unified insights into llm regularities and constraints.arXiv preprint arXiv:2412.16443, 2024

    Charles Luo. Has llm reached the scaling ceiling yet? unified insights into llm regularities and constraints.arXiv preprint arXiv:2412.16443, 2024

    arXiv 2024

  5. [1]

    Learning interactive real-world simulators[J]

    Yang M et al. Learning interactive real-world simulators[J]. arXiv preprint arXiv:2310.06114, 2023

    arXiv 2023

  6. [2]

    ”Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.” arXiv preprint arXiv:2412.05271, 2024

    Chen Zhe et al. ”Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.” arXiv preprint arXiv:2412.05271, 2024

    arXiv 2024

  7. [4]

    RT-1: robotics transformer for real-world control at scale

    Anthony Brohan et al. RT-1: robotics transformer for real-world control at scale. Robotics: Science and Systems XIX, 2023

    2023

  8. [6]

    Instruction-driven history-aware policies for robotic manipulations

    Pierre-Louis Guhur et al. Instruction-driven history-aware policies for robotic manipulations. Conference on Robot Learning, 2022

    2022

Show all 16 references
  1. [7]

    Lora: Low-rank adaptation of large language models

    Edward J Hu et al. Lora: Low-rank adaptation of large language models. ICLR, 2022

    2022

  2. [8]

    abs/2504.16054, 2025

    Physical Intelligence et al.π 0.5: a vision-language-action model with open-world generalization. abs/2504.16054, 2025

    2025 arXiv

  3. [9]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim et al. Openvla: An open-source vision-language-action model. Conference on Robot Learning, 2024

    2024

  4. [11]

    Open x-embodiment: Robotic learning datasets and RT-X models : Open x-embodiment collaboration

    Abby O’Neill et al. Open x-embodiment: Robotic learning datasets and RT-X models : Open x-embodiment collaboration. ICRA, 2024

    2024

  5. [13]

    Cliport: What and where pathways for robotic manipulation

    Mohit Shridhar et al. Cliport: What and where pathways for robotic manipulation. Conference on Robot Learning, 2021

    2021

  6. [14]

    Maniskill3: GPU parallelized robotics simulation and rendering for generalizable embodied AI

    Stone Tao et al. Maniskill3: GPU parallelized robotics simulation and rendering for generalizable embodied AI. abs/2410.00425, 2024

    2024 arXiv

  7. [15]

    Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning

    Tianhe Yu et al. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. Conference on Robot Learning, 2019

    2019

  8. [16]

    RT-2: vision-language-action models transfer web knowledge to robotic control

    Brianna Zitkovich et al. RT-2: vision-language-action models transfer web knowledge to robotic control. Conference on Robot Learning, 2023

    2023

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.