Pith. sign in

REVIEW 3 major objections 5 minor 129 references

Towards Conscious Service Robots

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that robots will reach the next level of capability only by adding two human-like conscious functions—global availability of information and metacognitive self-monitoring—to their learning architectures.

desk verdict A credible research agenda for consciousness-inspired service robots, but the causal claim about the working-memory bottleneck is asserted, not shown. read the letter →

arxiv 2501.15198 v1 pith:2TA4TABU submitted 2025-01-25 cs.RO cs.AI

Sven Behnke This is my paper · ORCID
classification cs.ROcs.AI
keywords consciousservicerobotssystematicgeneralizationglobalworkspacetheorymetacognitioncognitivearchitecturemobilemanipulationout-of-distributiondeepreinforcementlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that today's deep-learning robots, however good at routine perception and control, will keep failing when the world changes unless they adopt two functions from human conscious cognition: global availability of information and metacognition. The author's hypothesis is that adding C1 global availability and C2 metacognition to robot cognitive architectures will unlock systematic generalization—reusing knowledge of objects and tasks in novel combinations—and self-monitoring that catches risky or mistaken behavior. If this is right, the next generation of service robots could operate in open-ended homes and workplaces rather than in controlled settings, and the same architecture could inform how consciousness works in humans.

What carries the argument

The machinery is a two-part architecture layered on top of fast, unconscious C0 deep networks. C1 global availability is implemented as a learned working memory that selects a small set of elements from the highest-level perceptual representations and broadcasts them for serial, action-conditioned prediction and planning; the paper connects this to the consciousness prior, relational inductive biases, object-centered frames, and planning in the now. C2 metacognition is implemented by learning distributions over latent variables, sampling multiple plausible futures, and using the variance and worst-case return to drive information-gathering and risk-avoiding policies, with skill execution monitored against expected outcomes. The load-bearing mechanism is that the limited capacity of the workspace itself produces abstraction and compositionality.

What would settle it

Train otherwise identical robot-learning systems with and without a small working-memory bottleneck and metacognitive confidence estimation on the same compositionally split manipulation tasks. If the unbounded system generalizes to novel object-task combinations as well as or better than the bottlenecked system, or if humans with experimentally expanded working memory show no loss of systematic generalization, the causal story would be undercut.

Watch

Extended reading notes

Core claim

The central claim is stated in Section 5: using insights from human cognition by incorporating C1 global availability and C2 metacognition into robot cognitive architectures will enable the next level of robot capabilities. C1 is the conscious workspace bottleneck—a small set of selected items is broadcast across specialized modules, forcing composition of frequently co-occurring items and ignoring task-irrelevant ones, which the paper argues leads to systematic generalization. C2 is metacognition: estimating confidence in perceptions and predictions, detecting errors during execution, and knowing one's own limits. Equipped with both, robots would behave as if conscious: they would know that they are seeing something, report confidence, and handle novel situations by planning over a few bound variables in working memory.

Load-bearing premise

The load-bearing premise is that the limited-capacity conscious workspace is what causes systematic generalization in humans, and that this same mechanism, when implemented in an artificial neural architecture, will produce comparable generalization in robots.

Editorial extensions

If this is right

  • Service robots would generalize pick-and-place, tool use, and other skills to novel object arrangements rather than degrading on out-of-distribution scenes.
  • Robots would estimate confidence in what they perceive and predict, and choose actions that gather more information when uncertain.
  • Robots would monitor their own skill execution against expected outcomes, detecting errors early and mitigating them before damage occurs.
  • Task knowledge encoded by binding a few selected objects to rule variables could be reused across infinitely many novel situations.
  • If successful, such architectures would blur the line between machines that merely process information and machines that can report and reflect on their own cognitive states.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence the paper leaves implicit: the capacity limit should be treated as an engineered parameter, so a robot with a deliberately smaller working memory should generalize more systematically than one with an unbounded workspace on the same compositional tasks.
  • The same framework could be evaluated incrementally in simulation before real robots, by comparing otherwise identical deep reinforcement learning agents with and without a C1-style bottleneck and C2-style confidence estimates on compositionally split generalization benchmarks.
  • The paper's account suggests a behavioral definition of machine consciousness—global availability plus metacognitive reporting—that sidesteps the harder question of subjective experience, which is useful for building testable systems.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper is a position paper arguing that future autonomous service robots should incorporate two functional elements of human consciousness as characterized by Dehaene et al. [22]—C1 global availability and C2 metacognition—into their cognitive architectures. After a broad review of deep learning, large language models, 3D scene models, world models, and deep reinforcement learning in robotics, the paper identifies core challenges such as non-stationarity, high dimensionality, and out-of-distribution data. It then proposes that the limited-capacity global workspace in humans creates pressure toward compositionality and selective attention, which in turn 'leads to systematic generalization' (Section 4). The central hypothesis in Section 5 is that equipping C0-style deep RL systems with C1 and C2 will enable systematic generalization and self-monitoring, and Section 6 lays out objectives for conscious prediction/planning and self-monitoring. Section 7 proposes to realize this through a battery of inductive biases and gradual task scaling. The paper explicitly labels its main claim as a hypothesis and provides no experiments, simulations, or formal derivations.

Significance. The paper is a clearly written, synthetic position piece that maps a specific framework from cognitive science (Dehaene's C1/C2) onto a concrete robotics research agenda. It is transparent about the speculative status of its central claim, and the objectives and evaluation dimensions in Sections 6 and 7 are specific enough to guide future research. If the causal link between the global workspace bottleneck and systematic generalization were established, the proposed architecture would be a notable contribution to robot cognitive architectures. The paper's main weakness is that this causal link is asserted rather than demonstrated, and the manuscript does not isolate the contribution of C1/C2 from the many other inductive biases it lists. The paper ships no machine-checked proofs or reproducible code, which is expected for a position paper; its strength lies in synthesis rather than verification.

major comments (3)
  1. [Section 4, global workspace bottleneck paragraph] The claim that the limited-capacity System 2/global workspace 'leads to systematic generalization' is the load-bearing premise for the paper's central hypothesis in Section 5, but it is not supported by empirical or formal evidence. The two proposed mechanisms—chunking frequently co-occurring items and ignoring irrelevant items—are not sufficient to imply compositional generalization, because attention mechanisms in C0-style architectures already perform selective ignoring, and a hard bottleneck could plausibly restrict the number of variable bindings needed for compositional tasks. The paper should either provide a concrete argument or empirical support for the causal role of the bottleneck, or explicitly weaken the claim to a motivating conjecture with a stated falsification test.
  2. [Section 2, LLM paragraph vs. Section 4] The paper cites Lake and Baroni (2023) to note that autoregressively trained LLMs can acquire human-like systematic generalization through meta-learning, without the C1/C2 machinery argued for in Section 4. This does not refute a sufficiency claim, but it directly undercuts the narrative that the global workspace bottleneck is what produces systematic generalization. The paper should address this tension explicitly and clarify whether C1/C2 are claimed as necessary, sufficient, or merely one route among several; it should also state what experimental evidence would distinguish these readings.
  3. [Section 7, inductive biases list] Section 7 enumerates many inductive biases—relational bias, object-centric representations, canonical frames, hierarchical categorization, planning-in-the-now, sparse factor graphs, self-attention, and variable binding—that are already recognized as supporting systematic generalization in deep learning models. Without an ablation, a formal decomposition, or at least a clear argument for why C1/C2 are distinct from and necessary on top of these biases, the paper's central claim that C1 and C2 will enable the next level of robot capabilities is not separable from the broader structured-representation techniques it lists. A concrete test would compare a baseline augmented with all non-conscious biases against one augmented additionally with the WM/selection and metacognitive components proposed in Section 6.
minor comments (5)
  1. [Section 7, first sentence] There is a missing space in 'suitableinductive biases'; the sentence should read 'suitable inductive biases'.
  2. [References, [6]] Reference [6] (Battaglia et al., relational inductive biases) lists arXiv:2406.09246, which appears to be an incorrect identifier; the standard arXiv identifier for that paper is 1806.01261. Please verify and correct.
  3. [Section 2, scene perception paragraph] The paragraph beginning 'Scene perception.' appears to merge with the preceding paragraph after the RoboCup@Home sentence; a paragraph break should be inserted for readability.
  4. [Throughout] The paper switches between first-person singular ('I', 'My hypothesis', 'My team') and first-person plural ('we can act'), which is acceptable in a position paper but should be used consistently throughout.
  5. [Section 5, 'behave as if they were conscious'] The phrase 'they would behave as if they were conscious' could be clarified to distinguish functional C1/C2 processing from phenomenal consciousness, since the paper's argument only concerns the former.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a position paper presenting a hypothesis and research agenda, with no derived quantitative prediction that could reduce to its inputs by construction.

full rationale

The paper does not claim to derive a result from fitted parameters or to make a quantitative prediction. Section 4 offers a causal hypothesis that the limited capacity of System 2 creates pressure to compose items and to ignore irrelevant ones, and states that 'This leads to systematic generalization.' This is an unsupported empirical claim, not a circular reduction: systematic generalization was already defined independently as deriving meaning for novel compositions from the meaning of the composed concepts. No equation in the paper defines one claimed output in terms of the same data used to obtain it. The self-citations are limited to evidence of the author's prior robotic systems, competitions, and specific perception methods, such as PermutoSDF and object-centric video prediction; they are not invoked as authority for the C1/C2 hypothesis. The central claim is explicitly presented as a hypothesis in Section 5: 'My hypothesis is that using insights from human cognition for the cognitive architectures of robots by incorporating C1 global availability and C2 metacognition will enable the next level of robot capabilities.' Concern about whether the causal story is adequately supported is a correctness or evidence risk, not circularity. The paper is self-contained as a research proposal, and no self-imported uniqueness or ansatz is used to forbid alternatives.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim rests on cognitive-science assumptions about the causes of human generalization and on an engineering assumption that the proposed learning pipeline will scale. None of these are demonstrated in the paper.

assumptions (3)
  • domain assumption Human cognitive architecture, specifically System 1/2 and C1/C2, is a suitable blueprint for robot cognitive architectures.
    Invoked throughout Sections 4-6; the central hypothesis presupposes that insights from human cognition transfer to artificial systems without requiring human-like embodiment or phenomenology.
  • domain assumption The global workspace communication bottleneck is causally responsible for systematic generalization in humans.
    Section 4, paragraph 'While this severe communication bottleneck... This leads to systematic generalization.' This causal link is taken from cognitive-science literature and is load-bearing for the proposal.
  • domain assumption Deep reinforcement learning with suitable inductive biases, foundation-model distillation, and simulation can scale to open-ended real-world tasks.
    Section 7, 'I will not approach... in a single step' plus the use of simulation and foundation models; the roadmap assumes these tools will be sufficient.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Conscious Service Robots." pith.science (2026). https://pith.science/paper/2TA4TABU

@misc{pith2026250115198,
  author       = {Pith},
  title        = {Pith review of: Towards Conscious Service Robots},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2TA4TABU}},
  note         = {Machine review of arXiv:2501.15198}
}
read the original abstract

Deep learning's success in perception, natural language processing, etc. inspires hopes for advancements in autonomous robotics. However, real-world robotics face challenges like variability, high-dimensional state spaces, non-linear dependencies, and partial observability. A key issue is non-stationarity of robots, environments, and tasks, leading to performance drops with out-of-distribution data. Unlike current machine learning models, humans adapt quickly to changes and new tasks due to a cognitive architecture that enables systematic generalization and meta-cognition. Human brain's System 1 handles routine tasks unconsciously, while System 2 manages complex tasks consciously, facilitating flexible problem-solving and self-monitoring. For robots to achieve human-like learning and reasoning, they need to integrate causal models, working memory, planning, and metacognitive processing. By incorporating human cognition insights, the next generation of service robots will handle novel situations and monitor themselves to avoid risks and mitigate errors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

129 extracted references · 64 canonical work pages

  1. [22]

    Dehaene, S., Lau, H., Kouider, S.: What is consciousness, and could machines have it? Science 358(6362), 486–492 (2017) Towards Conscious Service Robots 17

    2017

  2. [1]

    CoRR (2023)

    Anil, R., Dai, A.M., Firat, O., et al.: PaLM 2 technical report. CoRR (2023). ArXiv:2305.10403

    arXiv 2023

  3. [2]

    IEEE Robotics Autom

    Asfour, T., Paus, F., et al.: ARMAR-6: A high-performance humanoid for human-robot collaboration in real-world scenarios. IEEE Robotics Autom. Mag. 26(4), 108–121 (2019)

    2019

  4. [3]

    In: IEEE/CVF Conf

    Assran, M., Duval, Q., Misra, I., et al.: Self-supervised learning from images with a joint- embedding predictive architecture. In: IEEE/CVF Conf. on CVPR (2023)

    2023

  5. [4]

    Cambridge University Press (1993)

    Baars, B.J.: A cognitive theory of consciousness. Cambridge University Press (1993)

    1993

  6. [5]

    Bahdanau, D., Murty, S., Noukhovitch, M., et al.: Systematic generalization: What is required and can it be learned? In: 7th Int. Conf. on Learning Representations (ICLR) (2019)

    2019

  7. [7]

    CoRR (2023)

    Beetz, M., Kazhoyan, G., Vernon, D.: The CRAM cognitive architecture for robot manipula- tion in everyday activities. CoRR (2023). ArXiv:2304.14119

    arXiv 2023

  8. [8]

    IEEE Robotics & Automation Magazine (RAM) 30(4), 98–104 (2023)

    Behnke, S., Adams, J.A., Locke, D.: The $10 million ANA Avatar XPRIZE competition: How it advanced immersive telepresence systems. IEEE Robotics & Automation Magazine (RAM) 30(4), 98–104 (2023)

    2023

Show all 129 references
  1. [9]

    CoRR (2017)

    Bengio, Y.: The consciousness prior. CoRR (2017). ArXiv:1709.08568

    2017 arXiv

  2. [10]

    IEEE Trans

    Bengio, Y., Courville, A.C., Vincent, P.: Representation learning: A review and new perspec- tives. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 35(8), 1798–1828 (2013)

    2013

  3. [11]

    Psychological Review 108(3), 624 (2001)

    Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S., Cohen, J.D.: Conflict monitoring and cognitive control. Psychological Review 108(3), 624 (2001)

    2001

  4. [12]

    In: Robotics: Science and Systems XIX (RSS) (2023)

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., et al.: RT-1: Robotics transformer for real-world control at scale. In: Robotics: Science and Systems XIX (RSS) (2023)

    2023

  5. [13]

    CoRR (2023)

    Bubeck, S., Chandrasekaran, V., Eldan, R., et al.: Sparks of artificial general intelligence: Early experiments with GPT-4. CoRR (2023). ArXiv:2303.12712

    2023 arXiv

  6. [14]

    In: IEEE Conf

    Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the Kinetics dataset. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (2017)

    2017

  7. [15]

    In: 37th Int

    Chen, T., Kornblith, S., Norouzi, M., Hinton, G.E.: A simple framework for contrastive learning of visual representations. In: 37th Int. Conf. on Machine Learning (ICML) (2020)

    2020

  8. [16]

    In: 11th Int

    Chen, Y., Yun, Z., Ma, Y., et al.: Minimalistic unsupervised representation learning with the sparse manifold transform. In: 11th Int. Conf. on Learning Representations (ICLR) (2023)

    2023

  9. [17]

    In: Advances in Neural Information Processing Systems 36 (NeurIPS) (2023)

    Chen, Z., Li, B., Wu, S., Jiang, K., Ding, S., Zhang, W.: Content-based unrestricted adversarial attack. In: Advances in Neural Information Processing Systems 36 (NeurIPS) (2023)

    2023

  10. [18]

    In: IEEE/RSJ Int

    Cornelio, C., Diab, M.: Recover: A neuro-symbolic framework for failure detection and recovery. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) (2024)

    2024

  11. [19]

    Cowan, N.: The magical mystery four: How is working memory capacity limited, and why? Current Directions in Psychological Science 19(1), 51–57 (2010)

    2010

  12. [20]

    Science Robotics 9(86) (2024)

    Dafarra, S., Pattacini, U., Romualdi, G., et al.: iCub3 avatar system: Enabling remote fully immersive embodiment of humanoid robots. Science Robotics 9(86) (2024)

    2024

  13. [21]

    In: Conference on Robot Learning (CoRL), pp

    Dasari, S., Ebert, F., Tian, S., Nair, S., Bucher, B., et al.: RoboNet: Large-scale multi-robot learning. In: Conference on Robot Learning (CoRL), pp. 885–897 (2019)

    2019

  14. [23]

    Dehghani, M., Djolonga, J., Mustafa, B., et al.: Scaling vision transformers to 22 billion parameters. In: Int. Conf. on Machine Learning (ICML), pp. 7480–7512 (2023)

    2023

  15. [24]

    In: 16th European Conference on Computer Vision (ECCV), pp

    Deng, B., Lewis, J.P., Jeruzalski, T., et al.: NASA: Neural articulated shape approximation. In: 16th European Conference on Computer Vision (ECCV), pp. 612–628 (2020)

    2020

  16. [25]

    In: IEEE Conf

    Deng, J., Dong, W., Socher, R., et al.: ImageNet: A large-scale hierarchical image database. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 248–255 (2009)

    2009

  17. [26]

    Driess, D., Xia, F., Sajjadi, M.S.M., et al.: PaLM-E: An embodied multimodal language model. In: Int. Conf. on Machine Learning (ICML), pp. 8469–8488 (2023)

    2023

  18. [27]

    Edelman, B.L., Goel, S., Kakade, S.M., Zhang, C.: Inductive biases and variable creation in self-attention mechanisms. In: Int. Conf. on Machine Learning (ICML) (2022)

    2022

  19. [28]

    IEEE Robotics Autom

    Florence, P.R., Manuelli, L., Tedrake, R.: Self-supervised correspondence in visuomotor policy learning. IEEE Robotics Autom. Lett. (RA-L) 5(2), 492–499 (2020)

    2020

  20. [29]

    Royal Institute of Philosophy Supplements 48, 227–242 (2001)

    Fodor, J.: Language, thought and compositionality. Royal Institute of Philosophy Supplements 48, 227–242 (2001)

    2001

  21. [30]

    Friston, K.: Does predictive coding have a future? Nature Neuroscience 21(8), 1019–1021 (2018)

    2018

  22. [31]

    In: IEEE/CVF Conf

    Gao, G., Liu, W., et al.: GraphDreamer: Compositional 3D scene synthesis from scene graphs. In: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) (2024)

    2024

  23. [32]

    Artificial Intelligence Review 56(11), 12387–12406 (2023)

    d’ Avila Garcez, A., Lamb, L.C.: Neurosymbolic AI: the 3rd wave. Artificial Intelligence Review 56(11), 12387–12406 (2023)

    2023

  24. [33]

    In: IEEE/CVF Conf

    Girdhar, R., El-Nouby, A., et al.: ImageBind one embedding space to bind them all. In: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) (2023)

    2023

  25. [34]

    Glanois, C., Jiang, Z., Feng, X., et al.: Neuro-symbolic hierarchical rule induction. In: Int. Conf. on Machine Learning (ICML), pp. 7583–7615 (2022)

    2022

  26. [35]

    Proceed- ings of the Royal Society A 478(2266), 20210068 (2022)

    Goyal, A., Bengio, Y.: Inductive biases for deep learning of higher-level cognition. Proceed- ings of the Royal Society A 478(2266), 20210068 (2022)

    2022

  27. [36]

    CoRR (2020)

    Greff, K., van Steenkiste, S., Schmidhuber, J.: On the binding problem in artificial neural networks. CoRR (2020). ArXiv:2012.05208

    2020 arXiv

  28. [37]

    IEEE Trans

    Gui, J., Chen, T., et al.: A survey on self-supervised learning: Algorithms, applications, and future trends. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 46(12), 9052–9071 (2024)

    2024

  29. [38]

    Science Robotics 9(89) (2024)

    Haarnoja, T., Moran, B., Lever, G., et al.: Learning agile soccer skills for a bipedal robot with deep reinforcement learning. Science Robotics 9(89) (2024)

    2024

  30. [39]

    In: IEEE/CVF Conf

    Han, J., Gong, K., Zhang, Y., et al.: OneLLM: One framework to align all modalities with language. In: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) (2024)

    2024

  31. [40]

    Hauser, K., Watson, E., Bae, J., Bankston, J., Behnke, S., et al.: Analysis and perspectives on the ANA Avatar XPRIZE competition. Int. Journal of Social Robotics (SORO) (2024)

    2024

  32. [41]

    In: IEEE/CVF Conf

    He, K., Chen, X., Xie, S., et al.: Masked autoencoders are scalable vision learners. In: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) (2022)

    2022

  33. [42]

    In: Robotics: Science and Systems XIX (RSS) (2023)

    Herzog, A., Rao, K., Hausman, K., et al.: Deep RL at scale: Sorting waste in office buildings with a fleet of mobile manipulators. In: Robotics: Science and Systems XIX (RSS) (2023)

    2023

  34. [43]

    CoRR (2019)

    Hessel, M., van Hasselt, H., Modayil, J., Silver, D.: On inductive biases in deep reinforcement learning. CoRR (2019). ArXiv:1907.02908

    2019 arXiv

  35. [44]

    IOS Press (2023)

    Hitzler, P., Sarker, M.K., Eberhart, A.: Compendium of Neurosymbolic Artificial Intelligence. IOS Press (2023)

    2023

  36. [45]

    In: 7th Int

    Hjelm, R.D., Fedorov, A., et al.: Learning deep representations by mutual information esti- mation and maximization. In: 7th Int. Conf. on Learning Representations (ICLR) (2019)

    2019

  37. [46]

    Neural Computation 9(8), 1735– 1780 (1997)

    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735– 1780 (1997)

    1997

  38. [47]

    In: Robotics: Science and Systems (RSS) (2022)

    Hughes, N., Chang, Y., Carlone, L.: Hydra: A real-time spatial perception system for 3D scene graph construction and optimization. In: Robotics: Science and Systems (RSS) (2022)

    2022

  39. [48]

    In: IEEE International Conference on Robotics and Automation (ICRA), pp

    Kaelbling, L.P., Lozano-P ´erez, T.: Hierarchical task and motion planning in the now. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1470–1477 (2011)

    2011

  40. [49]

    Macmillan (2011)

    Kahneman, D.: Thinking, fast and slow. Macmillan (2011)

    2011

  41. [50]

    In: Conference on Robot Learning (CoRL), pp

    Kalashnikov, D., Irpan, A., Pastor, P., et al.: Scalable deep reinforcement learning for vision- based robotic manipulation. In: Conference on Robot Learning (CoRL), pp. 651–673 (2018) 18 Sven Behnke

    2018

  42. [51]

    In: Conference on Robot Learning (CoRL), pp

    Kalashnikov, D., Varley, J., Chebotar, Y., Swanson, B., et al.: Scaling up multi-task robotic reinforcement learning. In: Conference on Robot Learning (CoRL), pp. 557–575 (2021)

    2021

  43. [52]

    In: IEEE Int

    Kazhoyan, G., Stelter, S., Kenfack, F.K., et al.: The robot household marathon experiment. In: IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 9382–9388 (2021)

    2021

  44. [53]

    IEEE Robotics Automation Magazine (RAM) 26(4), 30–45 (2019)

    Kheddar, A., Roa, M.A., Wieber, P., et al.: Humanoid robots in aircraft manufacturing: The Airbus use cases. IEEE Robotics Automation Magazine (RAM) 26(4), 30–45 (2019)

    2019

  45. [54]

    CoRR (2024)

    Kim, M.J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., et al.: OpenVLA: An open-source vision-language-action model. CoRR (2024). ArXiv:2406.09246

    2024 arXiv

  46. [55]

    In: Mensch und Computer (MuC) (2015)

    Kittmann, R., Fr ¨ohlich, T., Sch¨afer, J., Reiser, U., Weißhardt, F., Haug, A.: Let me introduce myself: I am Care-O-bot 4, a gentleman robot. In: Mensch und Computer (MuC) (2015)

    2015

  47. [56]

    IEEE Robotics & Automation Magazine (RAM) 26(4), 59–72 (2019)

    Klamt, T., Kamedula, M., Karaoguz, H., Kashiri, N., Laurenzi, A., Lenz, C., et al.: Flexible disaster response of tomorrow: Final presentation and evaluation of the CENTAURO system. IEEE Robotics & Automation Magazine (RAM) 26(4), 59–72 (2019)

    2019

  48. [57]

    Journal of Field Robotics (JFR) 37(5), 889–919 (2020)

    Klamt, T., Schwarz, M., Lenz, C., Baccelliere, L., Buongiorno, D., Cichon, T., others, Behnke, S.: Remote mobile manipulation with the Centauro robot: Full-body telepresence and au- tonomous operator assistance. Journal of Field Robotics (JFR) 37(5), 889–919 (2020)

    2020

  49. [58]

    In: 16th European Conference on Computer Vision (ECCV), pp

    Kolesnikov, A., Beyer, L., Zhai, X., et al.: Big transfer (bit): General visual representation learning. In: 16th European Conference on Computer Vision (ECCV), pp. 491–507 (2020)

    2020

  50. [59]

    IEEE Robotics Autom

    Kumagai, I., et al.: Toward industrialization of humanoid robots: Autonomous plasterboard installation to improve safety and efficiency. IEEE Robotics Autom. Mag.26(4), 20–29 (2019)

    2019

  51. [60]

    Lake, B.M., Baroni, M.: Generalization without Systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In: Int. C. on Machine Learning (ICML) (2018)

    2018

  52. [61]

    Nature 623(7985), 115–121 (2023)

    Lake, B.M., Baroni, M.: Human-like systematic generalization through a meta-learning neural network. Nature 623(7985), 115–121 (2023)

    2023

  53. [62]

    Behavioral and Brain Sciences 40, e253 (2017)

    Lake, B.M., Ullman, T.D., Tenenbaum, J.B., Gershman, S.J.: Building machines that learn and think like people. Behavioral and Brain Sciences 40, e253 (2017)

    2017

  54. [63]

    Lamb, L.C., d’ Avila Garcez, A.S., et al.: Graph neural networks meet neural-symbolic com- puting: A survey and perspective. In: Int. J. Conf. on Artificial Intelligence (IJCAI) (2020)

    2020

  55. [64]

    Nature 521(7553), 436–444 (2015)

    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    2015

  56. [65]

    IEEE Trans

    Lee, M.A., Zhu, Y., Zachares, P., Tan, M., Srinivasan, K., Savarese, S., Fei-Fei, L., Garg, A., Bohg, J.: Making sense of vision and touch: Learning multimodal representations for contact-rich tasks. IEEE Trans. Robotics 36(3), 582–596 (2020)

    2020

  57. [66]

    International Journal of Social Robotics (2023)

    Lenz, C., Schwarz, M., Rochow, A., P¨atzold, B., Memmesheimer, R., Schreiber, M., Behnke, S.: NimbRo wins ANA Avatar XPRIZE immersive telepresence competition: Human-centric evaluation and lessons learned. International Journal of Social Robotics (2023)

    2023

  58. [67]

    In: Conf

    Liang, J., Jiang, L., et al.: The garden of forking paths: Towards multi-future trajectory prediction. In: Conf. on Computer Vision and Pattern Recognition (CVPR) (2020)

    2020

  59. [68]

    CoRR (2023)

    Liu, B., Jiang, Y., Zhang, X., et al.: LLM+P: empowering large language models with optimal planning proficiency. CoRR (2023). ArXiv:2304.11477

    2023 arXiv

  60. [69]

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: Adv. in NeurIPS 36 (2023)

    2023

  61. [70]

    Science Robotics 7(69) (2022)

    Liu, S., Lever, G., Wang, Z., et al.: From motor control to team play in simulated humanoid football. Science Robotics 7(69) (2022)

    2022

  62. [71]

    In: IEEE Conf

    Mancini, M., Naeem, M.F., Xian, Y., Akata, Z.: Open world compositional zero-shot learning. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 5222–5230 (2021)

    2021

  63. [72]

    Artificial Intelligence 298, 103504 (2021)

    Manhaeve, R., Dumancic, S., Kimmig, A., Demeester, T., Raedt, L.D.: Neural probabilistic logic programming in DeepProbLog. Artificial Intelligence 298, 103504 (2021)

    2021

  64. [73]

    Artificial Intelligence 328, 104062 (2024)

    Marra, G., Dumancic, S., Manhaeve, R., Raedt, L.D.: From statistical relational to neurosym- bolic artificial intelligence: A survey. Artificial Intelligence 328, 104062 (2024)

    2024

  65. [74]

    In: IEEE Int

    Meeussen, W., Wise, M., Glaser, S., et al.: Autonomous door opening and plugging in with a personal robot. In: IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 729–736 (2010)

    2010

  66. [75]

    In: RoboCup 2024: Robot World Cup XXVII

    Memmesheimer, R., Nogga, J., P ¨atzold, B., Kruzhkov, E., Bultmann, S., Schreiber, M., Bode, J., Karacora, B., Park, J., Savinykh, A., Behnke, S.: RoboCup@Home 2024 OPL winner NimbRo: Anthropomorphic service robots using foundation models for perception and planning. In: RoboC...

    2025

  67. [76]

    In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Menapace, W., Lathuili `ere, S., Tulyakov, S., Siarohin, A., Ricci, E.: Playable video generation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    2021

  68. [77]

    Mildenhall, B., Srinivasan, P.P., Tancik, M., et al.: NeRF: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 65(1), 99–106 (2022)

    2022

  69. [78]

    Mitchell, T.M.: The need for biases in learning generalizations. Tech. rep., Department of Computer Science, Rutgers University, New Brunswick, NJ, USA (1980)

    1980

  70. [79]

    IEEE Robotics Autom

    Mittal, M., Yu, C., Yu, Q., et al.: Orbit: A unified simulation framework for interactive robot learning environments. IEEE Robotics Autom. Lett. (RA-L) 8(6), 3740–3747 (2023)

    2023

  71. [80]

    Pattern Recognit

    Moreno-Torres, J.G., Raeder, T., Ala ´ız-Rodr´ıguez, R., Chawla, N.V., Herrera, F.: A unifying view on dataset shift in classification. Pattern Recognit. 45(1), 521–530 (2012)

    2012

  72. [81]

    CoRR (2024)

    Mosbach, M., Ewertz, J.N., Villar-Corrales, A., Behnke, S.: SOLD: Reinforcement learning with slot object-centric latent dynamics. CoRR (2024). ArXiv:2410.08822

    2024 arXiv

  73. [82]

    Journal of Artificial Intelligence Research 20, 379–404 (2003)

    Nau, D.S., Au, T., Ilghami, O., Kuter, U., Murdock, J.W., Wu, D., Yaman, F.: SHOP2: an HTN planning system. Journal of Artificial Intelligence Research 20, 379–404 (2003)

    2003

  74. [83]

    In: Advances in NeurIPS 34 (2021)

    Nye, M.I., Tessler, M.H., et al.: Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning. In: Advances in NeurIPS 34 (2021)

    2021

  75. [84]

    In: IEEE Int

    O’Neill, A., Rehman, A., et al.: Open X-Embodiment: Robotic learning datasets and RT-X models. In: IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 6892–6903 (2024)

    2024

  76. [85]

    CoRR (2023)

    OpenAI: Gpt-4 technical report. CoRR (2023). ArXiv:2303.08774

    2023 arXiv

  77. [86]

    In: 12th Int

    Pernias, P., Rampas, D., et al.: W¨ urstchen: An efficient architecture for large-scale text-to- image diffusion models. In: 12th Int. Conf. on Learning Representations (ICLR) (2024)

    2024

  78. [87]

    IEEE Trans

    Placed, J.A., Strader, J., Carrillo, H., et al.: A survey on active simultaneous localization and mapping: State of the art and new frontiers. IEEE Trans. Robotics 39(3), 1686–1705 (2023)

    2023

  79. [88]

    Pratap, V., Tjandra, A., Shi, B., et al.: Scaling speech technology to 1,000+ languages. J. Mach. Learn. Res. (JMLR) 25, 97:1–97:52 (2024)

    2024

  80. [89]

    In: 12th Int

    Qin, Y., Liang, S., Ye, Y., et al.: ToolLLM: Facilitating large language models to master 16000+ real-world APIs. In: 12th Int. Conf. on Learning Representations (ICLR) (2024)

    2024

  81. [90]

    CoRR (2024)

    Qin, Y., Shi, Z., Yu, J., et al.: WorldSimBench: Towards video generation models as world simulators. CoRR (2024). ArXiv:2410.18072

    2024 arXiv

  82. [91]

    Radford, A., Kim, J.W., Xu, T., et al.: Robust speech recognition via large-scale weak super- vision. In: Int. Conf. on Machine Learning (ICML), pp. 28492–28518 (2023)

    2023

  83. [92]

    CoRR (2024)

    Ravi, N., Gabeur, V., Hu, Y., et al.: SAM 2: Segment anything in images and videos. CoRR (2024). ArXiv:2408.00714

    2024 arXiv

  84. [93]

    In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Rosu, R.A., Behnke, S.: PermutoSDF: Fast multi-view reconstruction with implicit surfaces using permutohedral lattices. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8466–8475 (2023)

    2023

  85. [94]

    In: Advances in Neural Information Processing Systems 35 (NeurIPS) (2022)

    Sajjadi, M.S.M., Duckworth, D., Mahendran, A., et al.: Object scene representation trans- former. In: Advances in Neural Information Processing Systems 35 (NeurIPS) (2022)

    2022

  86. [95]

    In: Robotics: Science and Systems (RSS) (2024)

    Schmid, L., Abate, M., et al.: Khronos: A unified approach for spatio-temporal metric- semantic SLAM in dynamic environments. In: Robotics: Science and Systems (RSS) (2024)

    2024

  87. [96]

    Proceedings of the IEEE 109(5), 612–634 (2021)

    Sch ¨olkopf, B., Locatello, F., Bauer, S., Ke, N.R., Kalchbrenner, N., Goyal, A., Bengio, Y.: Toward causal representation learning. Proceedings of the IEEE 109(5), 612–634 (2021)

    2021

  88. [97]

    Nature 588(7839), 604–609 (2020)

    Schrittwieser, J., Antonoglou, I., Hubert, T., et al.: Mastering Atari, Go, chess and shogi by planning with a learned model. Nature 588(7839), 604–609 (2020)

    2020

  89. [98]

    K¨ unstliche Intelligenz (Artificial Intelligence) 26(4), 357–363 (2012)

    Schulz, H., Behnke, S.: Deep learning – Layer-wise learning of feature hierarchies. K¨ unstliche Intelligenz (Artificial Intelligence) 26(4), 357–363 (2012)

    2012

  90. [99]

    In: 22nd IEEE-RAS Int

    Schwarz, M., Lenz, C., Memmesheimer, R., P ¨atzold, B., Rochow, A., Schreiber, M., Behnke, S.: Robust immersive telepresence and mobile telemanipulation: NimbRo wins ANA Avatar XPRIZE finals. In: 22nd IEEE-RAS Int. Conf. on Humanoid Robots (Humanoids) (2023)

    2023

  91. [100]

    Machine Learning 112(5), 1465–1497 (2023)

    Shindo, H., Pfanschilling, V., Dhami, D.S., Kersting, K.: 𝛼ILP: thinking visual scenes as differentiable logic programs. Machine Learning 112(5), 1465–1497 (2023)

    2023

  92. [101]

    Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6, 60 (2019)

    2019

  93. [102]

    In: 39th Conference on Artificial Intelligence (AAAI) (2025) 20 Sven Behnke

    Smet, L.D., Venturato, G., Raedt, L.D., Marra, G.: Relational neurosymbolic Markov models. In: 39th Conference on Artificial Intelligence (AAAI) (2025) 20 Sven Behnke

    2025

  94. [103]

    AI Magazine 43(3), 308–322 (2022)

    Smolensky, P., McCoy, R.T., et al.: Neurocompositional computing: From the central paradox of cognition to a new generation of AI systems. AI Magazine 43(3), 308–322 (2022)

    2022

  95. [104]

    Soulos, P., Hu, E.J., McCurdy, K., et al.: Differentiable tree operations promote compositional generalization. In: Int. Conf. on Machine Learning (ICML), pp. 32499–32520 (2023)

    2023

  96. [105]

    Frontiers Robotics AI 3, 58 (2016)

    St¨ uckler, J., Schwarz, M., Behnke, S.: Mobile manipulation, tool use, and intuitive interaction for cognitive service robot Cosero. Frontiers Robotics AI 3, 58 (2016)

    2016

  97. [106]

    IEEE Robotics & Automation Magazine 19(2), 34–42 (2012)

    St¨ uckler, J., Holz, D., Behnke, S.: RoboCup@Home: Demonstrating everyday manipulation skills in RoboCup@Home. IEEE Robotics & Automation Magazine 19(2), 34–42 (2012)

    2012

  98. [107]

    CoRR (2024)

    Sun, Q., Wang, J., Yu, Q., Cui, Y., Zhang, F., Zhang, X., Wang, X.: EV A-CLIP-18B: Scaling CLIP to 18 billion parameters. CoRR (2024). ArXiv:2402.04252

    2024 arXiv

  99. [108]

    MIT Press (2018)

    Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT Press (2018)

    2018

  100. [109]

    Science 331(6022), 1279–1285 (2011)

    Tenenbaum, J.B., Kemp, C., Griffiths, T.L., Goodman, N.D.: How to grow a mind: Statistics, structure, and abstraction. Science 331(6022), 1279–1285 (2011)

    2011

  101. [110]

    In: 32nd Conf

    Toyer, S., Trevizan, F.W., Thi´ebaux, S., Xie, L.: Action schema networks: Generalised policies with deep learning. In: 32nd Conf. on Artificial Intelligence (AAAI), pp. 6294–6301 (2018)

    2018

  102. [111]

    Nature 625(7995), 476–482 (2024)

    Trinh, T.H., Wu, Y., Le, Q.V., He, H., Luong, T.: Solving olympiad geometry without human demonstrations. Nature 625(7995), 476–482 (2024)

    2024

  103. [112]

    CoRR (2024)

    Valevski, D., Leviathan, Y., Arar, M., Fruchter, S.: Diffusion models are real-time game engines. CoRR (2024). ArXiv:2408.14837

    2024 arXiv

  104. [113]

    Varadarajan, B., et al.: MultiPath++: Efficient information fusion and trajectory aggregation for behavior prediction. In: Int. Conf. on Robotics and Automation (ICRA) (2022)

    2022

  105. [114]

    In: Advances in Neural Information Processing Systems 30 (NeurIPS), pp

    Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems 30 (NeurIPS), pp. 5998–6008 (2017)

    2017

  106. [115]

    In: Conference on Robot Learning (CoRL) (2019)

    Veerapaneni, R., Co-Reyes, J.D., Chang, M., Janner, M., et al.: Entity abstraction in visual model-based reinforcement learning. In: Conference on Robot Learning (CoRL) (2019)

    2019

  107. [116]

    In: IEEE International Conference on Image Processing (ICIP), pp

    Villar-Corrales, A., Wahdan, I., Behnke, S.: Object-centric video prediction via decoupling of object dynamics and interactions. In: IEEE International Conference on Image Processing (ICIP), pp. 570–574 (2023)

    2023

  108. [117]

    Nature 575(7782), 350–354 (2019)

    Vinyals, O., Babuschkin, I., Czarnecki, W.M., et al.: Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575(7782), 350–354 (2019)

    2019

  109. [118]

    In: IEEE Conf

    Wang, H., et al.: Normalized object coordinate space for category-level 6D object pose and size estimation. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (2019)

    2019

  110. [119]

    In: IEEE/CVF Conf

    Wang, X., Misra, I., Zeng, Z., et al.: VideoCutLER: Surprisingly simple unsupervised video instance segmentation. In: IEEE/CVF Conf. on CVPR (2024)

    2024

  111. [120]

    In: Conference on Robot Learning (CoRL), pp

    Wu, P., Escontrela, A., Hafner, D., Abbeel, P., Goldberg, K.: DayDreamer: World models for physical robot learning. In: Conference on Robot Learning (CoRL), pp. 2226–2240 (2022)

    2022

  112. [121]

    In: 11th Int

    Wu, Z., Dvornik, N., Greff, K., et al.: SlotFormer: Unsupervised visual dynamics simulation with object-centric models. In: 11th Int. Conf. on Learning Representations (ICLR) (2023)

    2023

  113. [122]

    IEEE Trans

    Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Networks Learn. Syst. 32(1), 4–24 (2021)

    2021

  114. [123]

    Nature 602(7896), 223–228 (2022)

    Wurman, P.R., Barrett, S., Kawamoto, K., et al.: Outracing champion Gran Turismo drivers with deep reinforcement learning. Nature 602(7896), 223–228 (2022)

    2022

  115. [124]

    In: IEEE/CVF Conf

    Xie, Q., Luong, M., et al.: Self-training with noisy student improves ImageNet classification. In: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) (2020)

    2020

  116. [125]

    In: RoboCup 2019: Robot World Cup XXIII

    Yamamoto, T., Takagi, Y., et al.: Human support robot as research platform of domestic mobile manipulator. In: RoboCup 2019: Robot World Cup XXIII. Springer (2019)

    2019

  117. [126]

    In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Yang, J., Gao, S., Qiu, Y., et al.: Generalized predictive model for autonomous driving. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    2024

  118. [127]

    IEEE Robo

    Yoshiike, T., et al.: The experimental humanoid robot E2-DR: A design for inspection and disaster response in industrial environments. IEEE Robo. Autom. Mag. 26(4), 46–58 (2019)

    2019

  119. [128]

    In: Robotics: Science and Systems XIX (RSS) (2023)

    Yu, T., Xiao, T., Tompson, J., et al.: Scaling robot learning with semantically imagined experience. In: Robotics: Science and Systems XIX (RSS) (2023)

    2023

  120. [129]

    In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Zhai, X., Kolesnikov, A., Houlsby, N., Beyer, L.: Scaling vision transformers. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1204–1213 (2022)

    2022

  121. [130]

    Cognition 244, 105711 (2024)

    Zhou, Y., Feinman, R., Lake, B.M.: Compositional diversity in visual concept learning. Cognition 244, 105711 (2024)

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.