Pith. sign in

REVIEW 4 major objections 6 minor 46 references

Learning Affordances from Interactive Exploration using an Object-level Map

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper's central claim is that an object-level map, which tracks each object instance across viewpoints, makes interactive exploration more efficient and the learned affordance predictions more accurate than labeling each frame…

desk verdict A real new idea in interactive affordance learning, but the headline claim about the map is confounded with the confidence labeler and needs an extra ablation to stand. read the letter →

arxiv 2501.06047 v1 pith:6ZUECQQY submitted 2025-01-10 cs.RO

Paula Wulkop , Halil Umut Özdemir , Antonia Hüfner , Jen Jen Chung , Roland Siegwart , Lionel Ott This is my paper · ORCID
classification cs.RO
keywords affordancelearninginteractiveexplorationobject-levelmappingreinforcementself-supervisedlabelingrobotmanipulationsemanticinstancesegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Robots that need to interact with objects must know what they can push or pick up, but those affordances depend on the robot's own body and gripper, not just on the object. This paper tries to let a robot discover those robot-specific affordances by itself: a reinforcement-learning explorer moves through a new scene, attempts interactions, and records which succeed. The proposed addition is an object-level map that remembers each object instance, so a single interaction can label that object in every later frame and the agent can see what it has already tried. The paper argues this map-based memory produces denser and cleaner training data, which in turn makes exploration more efficient and yields more accurate affordance predictions than the comparison method, especially for small pickupable objects.

What carries the argument

The central mechanism is the object-level map: a volumetric scene representation in which each object instance is stored as its own signed-distance layer, so the robot can re-identify the same object under different viewpoints and after it has been moved. Interaction outcomes, whether a pick-up or push succeeded or failed, are attached to the instance in the map rather than to a single image location. Two labeling strategies then convert the map into training data: annotation by interaction, which labels all frames where the object appears, and annotation by confidence, which pseudo-labels additional objects when the affordance network is very confident above 0.9 or very unconfident below 0.1. The map also supplies extra states to the reinforcement-learning policy, such as occupancy, locations of interacted and non-interacted objects, and action history, which guide the agent toward objects it has not tried yet.

What would settle it

Run the identical exploration loop with a learned segmentation model on real RGB-D data instead of simulator-provided masks and compare pick-up F1 on the same test scenes; if the gap over the map-free baseline disappears, the reported improvements come from perfect masks rather than from the object-level map.

Watch

Extended reading notes

Core claim

The paper's central claim is that using an object-level map in the interactive affordance-learning loop improves both the exploration policy and the final affordance model. Instead of marking a fixed sphere around each interaction point in each frame, the pipeline annotates the object instance in the map and propagates that annotation to every frame where the instance is visible. With this mechanism, the full method reaches an F1 of 0.22 for the pick-up affordance versus 0.04 for the map-free baseline on unseen test frames, and a pick-up interaction-success F1 of 0.60 versus 0.21. For push, the map-based method reaches 0.99 precision, meaning it almost never labels non-pushable surfaces as pushable, at the cost of lower recall. The authors conclude that retaining object identity across viewpoints is the reason for the improvement.

Load-bearing premise

The pipeline assumes that accurate per-pixel instance segmentation masks are available during exploration, because every label stored in the map is attached through those masks.

Editorial extensions

If this is right

  • A robot can build a training dataset for its own body and gripper without human labels, so affordance predictions reflect the robot's actual interaction capabilities.
  • One interaction with an object can train the model on many frames of that object, which makes data collection cheaper and denser.
  • The learned pick-up model improves interaction success on unseen scenes, with pick-up F1 rising from 0.21 in the baseline to 0.60.
  • The high precision of the push model means the robot mostly avoids wasting time pushing objects that cannot move.
  • The same integration of mapping, exploration, and affordance learning could be extended to other rigid-body interactions such as lifting and placing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the segmentation masks are the weakest link: if a real-world instance segmenter confuses objects or misses an object after a push, the map will store outcomes against the wrong instance, and the reported gains should diminish.
  • Beyond the paper, the rigid-object assumption in the map representation sets a boundary: actions like open, slice, or deform would need a different object model, not just more training data.
  • Beyond the paper, the confidence-threshold pseudo-labeling at 0.9 and 0.1 could be made self-calibrating; measuring the calibration of the affordance network would show whether the pseudo-labels are reliable.
  • Beyond the paper, the object-level map with interaction history is a reusable scene representation that could support downstream tasks such as object goal navigation, change detection, or semantic editing of the environment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents an interactive exploration pipeline for learning robot-specific affordances (pick up and push) in 3D indoor scenes. A PPO-trained agent explores iTHOR living rooms while a TSDF++ object-level map tracks object instances and stores interaction outcomes. The map is used both as part of the RL state and to propagate sparse interaction labels across viewpoints. An additional 'annotation by confidence' mechanism pseudo-labels objects using the affordance network's own confident predictions. The authors compare against IntExp and two ablations (No Map + Seg, No Map + No Seg) and report that their method achieves higher pick-up affordance F1 and interaction success rates, while push performance is mixed.

Significance. If the central claim holds, the paper makes a useful contribution: it demonstrates that an explicit object-level map can provide object permanence during interactive exploration, yielding denser and more accurate affordance annotations than per-frame backprojection. The pipeline is modular, and the comparison includes several metrics (pixel-level IoU, object-level accuracy, interaction success, annotation rates). The authors also correctly acknowledge limitations of TSDF++ (rigid objects only) and the overconfidence of neural networks in the confidence-labeling step. However, the main comparison is confounded by the simultaneous introduction of the map and of self-training via annotation by confidence, and the push results do not support the broad claim that the map improves affordance prediction quality. The evaluation is entirely in simulation with oracle segmentation masks, so real-world transfer is not demonstrated.

major comments (4)
  1. [§3.3, Eq. (5) and §4.1, Ablations] The comparison between 'Ours' and 'No Map + Seg' is confounded. Section 4.1 defines 'No Map + Seg' only as removing the map from the RL state and using a GRU, but it does not state whether annotation by confidence is disabled. Since the percentile computation in Eq. (5) requires associating the same object instance across frames, a no-map pipeline cannot perform this step, so the ablation must implicitly disable it. Consequently, the pick-up F1 gain in Table 1 (0.22 vs. 0.13) could be caused by the self-training pseudo-labels rather than by the object-level map's contribution. The authors should add an ablation that keeps the map but disables annotation by confidence, or otherwise disentangle the two mechanisms, to support the paper's central claim that the map itself improves affordance prediction.
  2. [§4.3, Table 2] The push-affordance results in Table 2 contradict the broad claim that the object-level map improves affordance prediction quality. For push, 'Ours' has lower Accuracy (0.61 vs. 0.76), lower Recall (0.43 vs. 0.91), and lower F1 (0.58 vs. 0.83) than 'No Map + No Seg' at the object-interaction level. The paper's explanation (precision/recall tradeoff, spherical annotations spilling over) does not resolve the fact that the full method is worse on an aggregate metric. The central claim should be restricted to pick-up, or the authors should provide an additional task-level or cost-sensitive analysis that justifies the precision-focused behavior for push.
  3. [§4.1, Baseline] The re-implementation of the IntExp baseline changes several factors at once: only two affordances are used instead of the original set, the environment is changed from kitchen to living room, the image size is increased from 80x80 to 128x128, and the number of parallel RL processes is halved from 16 to 8. These changes are not controlled, so the comparison does not isolate the effect of the object-level map. The authors should either justify that these changes affect IntExp and their method symmetrically, or provide a version of IntExp trained under the original settings and then evaluated on the new test scenes.
  4. [§3.3, Data generation and labeling] The pipeline relies on ground-truth instance segmentation masks during exploration, as stated in Section 3.3 ('the segmentation mask comes directly from the simulator'). The authors note that a separate segmentation network could be used, but they do not evaluate the effect of imperfect masks or TSDF++ tracking errors on label quality. Since all experimental claims are established under oracle perception, the extent to which the approach transfers to real-world settings remains unsubstantiated; this should be acknowledged more prominently and ideally tested with noisy masks.
minor comments (6)
  1. [§2.4 and §4.1] The baseline is referred to as both 'IntExp(PT)' and 'IntExp'; for consistency, the paper should use one name throughout and define it explicitly at first use.
  2. [Fig. 5 caption] The caption lists 'No Map' as one of the curves, but the legend in the figure area appears to include 'No Map + No Seg' and 'No Map + Seg'. Please clarify whether 'No Map' is a typo for 'No Map + No Seg' or a separate method.
  3. [Eq. (5)] The notation '{Fi}P95' is not defined; please specify that it denotes the 95th percentile of the affordance network's predicted scores for the pixels of the object in frame Fi, and similarly for the 5th percentile.
  4. [Table 1 caption] The caption says 'averaged over 611 frames of the test scenes', but the table reports medians and [10th, 90th] percentiles; please clarify whether the values are medians over frames or means, and adjust the wording.
  5. [§4.1, Ablations] The sentence 'the usage of both the segmentation masks and the map have a positive impact on all metrics' is not supported by the push results in Table 2, where 'No Map + No Seg' outperforms 'Ours' in accuracy and F1. Please qualify this statement.
  6. [§3.2, State space] The phrase '2D convolutional encoders for the image-like inputs with a dimension of (128, 128)' would be clearer as 'input resolution of 128x128'.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-training loop in annotation by confidence; central map claim remains externally evaluated.

  1. self definitional [Section 3.3, Data generation and labeling and Training procedure, Eq. (5)]
    "After the affordance network starts to converge, the affordance prediction is queried for each frame Fi and is used to further annotate objects with which the robot has not interacted. ... label = 1 if max {{Fi}P 95 ∀i} > ξT ... The training of the affordance network is then continued with this global dataset."

    The labels used to retrain the affordance network are generated by thresholding the same network's own confidence percentiles (Eq. 5). Thus part of the training signal is defined in terms of the model's current predictions, and the model is then trained to reproduce those predictions. This is a self-training loop that can reinforce the model's own biases and overconfidence, a concern the paper itself acknowledges when it notes that neural networks are known to be over-confident. The loop is not fully closed because final evaluation is performed on unseen test scenes with ground-truth labels, so the circularity is partial and does not by construction force the reported test results.

full rationale

The central derivation chain is otherwise self-contained and externally grounded: interaction outcomes from the simulator are stored in a TSDF++ object-level map, propagated to frames via segmentation masks, used to train a U-Net, and evaluated on 611 manually-steered frames from five unseen scenes with ground-truth interaction outcomes. The map-based propagation is a direct application of previously published methods (SEAL, TSDF++) and is not presented as a uniqueness theorem or a fitted result. The only partially circular element is annotation by confidence, where Eq. (5) defines new training labels from the affordance network's own confident predictions; retraining on these labels is a self-training loop. The paper discloses this mechanism and explicitly notes that calibration could be beneficial, and the final metrics in Tables 1 and 2 use ground-truth labels rather than the pseudo-labels, so the loop is not closed. The No Map + Seg ablation is confounded because removing the map also removes the cross-view instance tracking that annotation by confidence relies on, but this is an experimental-design weakness rather than a circular reduction of the central claim. No load-bearing self-citations or imported uniqueness arguments are present.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the availability of perfect segmentation masks and a reliable object-level tracking framework, both of which are taken from prior work or given by the simulator. The annotation thresholds and activation step are manually chosen free parameters.

free parameters (4)
  • confidence threshold xi_T = 0.9
    Used in Eq. (5) to label objects as interactable when the 95th percentile of affordance predictions exceeds 0.9.
  • confidence threshold xi_F = 0.1
    Used in Eq. (5) to label objects as non-interactable when the 5th percentile falls below 0.1.
  • annotation by confidence activation step = 400000
    Described as 'empirically determined' in Section 4.2; the pseudo-labeling is enabled only after this number of training steps.
  • reward weights alpha_1, alpha_2, alpha_3 = not reported
    The reward in Eq. (1) is a weighted sum of navigation, interaction, and failure rewards; the paper does not state the values.
assumptions (3)
  • domain assumption The iTHOR simulator provides realistic physics and interaction feedback for pick-up and push actions.
    All interaction outcomes (success/failure) are taken from the simulator; real-world physics may differ.
  • domain assumption TSDF++ can track rigid object instances across viewpoints and after robot interactions.
    The paper relies on TSDF++ to re-identify objects and propagate labels, citing [8] without new validation in this setting.
  • ad hoc to paper Ground-truth instance segmentation masks are available during exploration.
    Section 3.3 states the segmentation mask comes directly from the simulator; this removes the perception challenge.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Affordances from Interactive Exploration using an Object-level Map." pith.science (2026). https://pith.science/paper/6ZUECQQY

@misc{pith2026250106047,
  author       = {Pith},
  title        = {Pith review of: Learning Affordances from Interactive Exploration using an Object-level Map},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ZUECQQY}},
  note         = {Machine review of arXiv:2501.06047}
}
read the original abstract

Many robotic tasks in real-world environments require physical interactions with an object such as pick up or push. For successful interactions, the robot needs to know the object's affordances, which are defined as the potential actions the robot can perform with the object. In order to learn a robot-specific affordance predictor, we propose an interactive exploration pipeline which allows the robot to collect interaction experiences while exploring an unknown environment. We integrate an object-level map in the exploration pipeline such that the robot can identify different object instances and track objects across diverse viewpoints. This results in denser and more accurate affordance annotations compared to state-of-the-art methods, which do not incorporate a map. We show that our affordance exploration approach makes exploration more efficient and results in more accurate affordance prediction models compared to baseline methods.

Figures

Figures reproduced from arXiv: 2501.06047 by the authors.

Figure 1
Figure 1. An object-level map of a living room scene generated by TSDF++ [8] with the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of our method during training. At each time step, an action is executed and the simulator (blue) outputs if the action was successful, as well as the RGB-D image, ground truth instance segmentation mask, and robot pose. The mapping module (red) updates the map with this data, while the affordance module (yellow) predicts the affordances. The RL exploration policy module (green) estimates the next optimal ac… view at source ↗
Figure 3
Figure 3. Visual representation of the state space. The elements in the top row do not require a map, while the states in the bottom are obtained through the map. their obtained success and reward as well as the inventory indicating if the robot currently holds a picked up object. In addition to these states derived from the current RGB-D input, we add states based on the object-level map to improve exploration efficiency. An… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: An example image of the annotation approach used by [10] and the No Map + [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The training curves for the pick up affordance show that our approach leads to a higher interaction success rate and a better affordance estimation performance. steps, seems to boost the performance of the Interaction Success Rate since more informed interactions can b…
Figure 6
Figure 6. Figure 6: Pickupable annotations for an example frame. The cyan circle shows a small pickupable object for which the No Map + No Seg baseline annotates a fixed big area since it does not use the segmentation mask. The magenta circle shows an object that the baseline methods fail…
Figure 7
Figure 7. Figure 7: Affordance predictions for an example test frame. For the [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 45 canonical work pages

  1. [1]

    J. J. Gibson, The theory of affordances . Hilldale, USA, 1977

    work page 1977

  2. [2]

    Affordance detection of tool parts from geometric features,

    A. Myers, C. L. Teo, C. Ferm¨ uller, and Y. Aloimonos, “Affordance detection of tool parts from geometric features,” in IEEE Int. Conf. on Robotics & Automa- tion, 2015

    work page 2015

  3. [3]

    Affordancenet: An end-to-end deep learning approach for object affordance detection,

    T.-T. Do, A. Nguyen, and I. Reid, “Affordancenet: An end-to-end deep learning approach for object affordance detection,” in IEEE Int. Conf. on Robotics & Automation, 2018

    work page 2018

  4. [4]

    3d affordancenet: A benchmark for visual object affordance understanding,

    S. Deng, X. Xu, C. Wu, K. Chen, and K. Jia, “3d affordancenet: A benchmark for visual object affordance understanding,” in Advances in Neural Information Processing Systems, 2021

    work page 2021

  5. [5]

    Detecting ob- ject affordances with convolutional neural networks,

    A. Nguyen, D. Kanoulas, D. G. Caldwell, and N. G. Tsagarakis, “Detecting ob- ject affordances with convolutional neural networks,” inIEEE/RSJ Int. Conf. on Intelligent Robots and Systems , 2016

    work page 2016

  6. [6]

    Towards visual affordance learning: A benchmark for affordance segmentation and recognition,

    Z. Osama Khalifa and S. A. A. Shah, “Towards visual affordance learning: A benchmark for affordance segmentation and recognition,” arXiv e-prints , 2022

    work page 2022

  7. [7]

    It’s just semantics: How to get robots to understand the world the way we do,

    J. J. Chung, J. F¨ orster, P. Wulkop, N. Lawrance, L. Ott, and R. Siegwart, “It’s just semantics: How to get robots to understand the world the way we do,” in Int. Symposium on Robotics Research , 2022

    work page 2022

  8. [8]

    TSDF++: A multi-object formulation for dynamic object tracking and reconstruction,

    M. Grinvald, F. Tombari, R. Siegwart, and J. Nieto, “TSDF++: A multi-object formulation for dynamic object tracking and reconstruction,” inIEEE Int. Conf. on Robotics & Automation, 2021. Learning Affordances 15

    work page 2021

Show all 46 references
  1. [9]

    AI2-THOR: An Interactive 3D Environment for Visual AI,

    E. Kolve et al. , “AI2-THOR: An Interactive 3D Environment for Visual AI,” arXiv, 2017

    2017

  2. [10]

    Learning affordance landscapes for interaction exploration in 3d environments,

    T. Nagarajan and K. Grauman, “Learning affordance landscapes for interaction exploration in 3d environments,” Advances in Neural Information Processing Systems, 2020

    2020

  3. [11]

    Panoptic multi-tsdfs: A flexible representation for online multi- resolution volumetric mapping and long-term dynamic scene consistency,

    L. Schmid et al., “Panoptic multi-tsdfs: A flexible representation for online multi- resolution volumetric mapping and long-term dynamic scene consistency,” in IEEE Int. Conf. on Robotics & Automation , 2022

    2022

  4. [12]

    Affordances in psychology, neuroscience, and robotics: A sur- vey,

    L. Jamone et al., “Affordances in psychology, neuroscience, and robotics: A sur- vey,” IEEE Transactions on Cognitive and Developmental Systems , 2016

    2016

  5. [13]

    Affordance detection for task-specific grasping using deep learning,

    M. Kokic, J. A. Stork, J. A. Haustein, and D. Kragic, “Affordance detection for task-specific grasping using deep learning,” in IEEE-RAS Int. Conf. on Hu- manoid Robotics (Humanoids) , 2017

    2017

  6. [14]

    One-shot open affordance learn- ing with foundation models,

    G. Li, D. Sun, L. Sevilla-Lara, and V. Jampani, “One-shot open affordance learn- ing with foundation models,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2024

    2024

  7. [15]

    Learning human activities and object affordances from rgb-d videos,

    H. S. Koppula, R. Gupta, and A. Saxena, “Learning human activities and object affordances from rgb-d videos,” The Int. Journal of Robotics Research , 2013

    2013

  8. [16]

    Object-based affordances detection with convolutional neural networks and dense conditional random fields,

    A. Nguyen, D. Kanoulas, D. G. Caldwell, and N. G. Tsagarakis, “Object-based affordances detection with convolutional neural networks and dense conditional random fields,” in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems , 2017

    2017

  9. [17]

    Inferring forces and learning human utilities from videos,

    Y. Zhu, C. Jiang, Y. Zhao, D. Terzopoulos, and S.-C. Zhu, “Inferring forces and learning human utilities from videos,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2016

    2016

  10. [18]

    Grounded human-object interaction hotspots from video,

    T. Nagarajan, C. Feichtenhofer, and K. Grauman, “Grounded human-object interaction hotspots from video,” in IEEE Int. Conf. on Computer Vision , 2019

    2019

  11. [19]

    Demo2vec: Reasoning object affordances from online videos,

    K. Fang, T.-L. Wu, D. Yang, S. Savarese, and J. J. Lim, “Demo2vec: Reasoning object affordances from online videos,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2018

    2018

  12. [20]

    Joint discovery of object states and manipulation actions,

    J.-B. Alayrac, I. Laptev, J. Sivic, and S. Lacoste-Julien, “Joint discovery of object states and manipulation actions,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2017

    2017

  13. [21]

    Ego-topo: Environ- ment affordances from egocentric video,

    T. Nagarajan, Y. Li, C. Feichtenhofer, and K. Grauman, “Ego-topo: Environ- ment affordances from egocentric video,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2020

    2020

  14. [22]

    SceneFun3D: Fine-Grained Functionality and Affordance Understanding in 3D Scenes,

    A. Delitzas, A. Takmaz, F. Tombari, R. Sumner, M. Pollefeys, and F. Engelmann, “SceneFun3D: Fine-Grained Functionality and Affordance Understanding in 3D Scenes,” in IEEE Conf. on Computer Vision and Pattern Recognition , 2024

    2024

  15. [23]

    Reinforcement learning based nav- igation with semantic knowledge of indoor environments,

    T.-L. Nguyen, D.-V. Nguyen, and T.-H. Le, “Reinforcement learning based nav- igation with semantic knowledge of indoor environments,” in Int. Conf. on Knowledge and Systems Engineering (KSE) , 2019

    2019

  16. [24]

    Learning to navigate unseen environments: Back translation with environmental dropout,

    H. Tan, L. Yu, and M. Bansal, “Learning to navigate unseen environments: Back translation with environmental dropout,” in Conference of the North American Chapter of the Association for Computational Linguistics , 2019

    2019

  17. [25]

    Learning exploration policies for navigation,

    T. Chen, S. Gupta, and A. Gupta, “Learning exploration policies for navigation,” in Int. Conf. on Learning Representations , 2019

    2019

  18. [26]

    An exploration of em- bodied visual exploration,

    S. K. Ramakrishnan, D. Jayaraman, and K. Grauman, “An exploration of em- bodied visual exploration,” Int. Journal of Computer Vision , 2021

    2021

  19. [27]

    Learning to move with affordance maps,

    W. Qi, R. T. Mullapudi, S. Gupta, and D. Ramanan, “Learning to move with affordance maps,” in Int. Conf. on Learning Representations , 2020. 16 Paula Wulkop et al

    2020

  20. [28]

    Robot navigation in unseen spaces using an abstract map,

    B. Talbot, F. Dayoub, P. Corke, and G. Wyeth, “Robot navigation in unseen spaces using an abstract map,” IEEE Transactions on Cognitive and Develop- mental Systems , 2020

    2020

  21. [29]

    Learning to explore using active neural slam,

    D. S. Chaplot, D. Gandhi, S. Gupta, A. Gupta, and R. Salakhutdinov, “Learning to explore using active neural slam,” in Int. Conf. on Learning Representations , 2020

    2020

  22. [30]

    Alp: Action-aware embodied learning for perception,

    X. Liang, A. Han, W. Yan, A. Raghunathan, and P. Abbeel, “Alp: Action-aware embodied learning for perception,” arXiv preprint arXiv:2306.10190 , 2023

    2023 arXiv

  23. [31]

    Virtualhome: Simulating household activities via programs,

    X. Puig et al. , “Virtualhome: Simulating household activities via programs,” in IEEE Conf. on Computer Vision and Pattern Recognition , 2018

    2018

  24. [32]

    Vrkitchen: An interactive 3d virtual environment for task-oriented learning,

    X. Gao, R. Gong, T. Shu, X. Xie, S. Wang, and S.-C. Zhu, “Vrkitchen: An interactive 3d virtual environment for task-oriented learning,” arXiv, 2019

    2019

  25. [33]

    Alfred: A benchmark for interpreting grounded instructions for everyday tasks,

    M. Shridhar et al., “Alfred: A benchmark for interpreting grounded instructions for everyday tasks,” inIEEE Conf. on Computer Vision and Pattern Recognition, 2020

    2020

  26. [34]

    Embodied ques- tion answering,

    A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, and D. Batra, “Embodied ques- tion answering,” in IEEE Conf. on Computer Vision and Pattern Recognition , 2018

    2018

  27. [35]

    Iqa: Visual question answering in interactive environments,

    D. Gordon, A. Kembhavi, M. Rastegari, J. Redmon, D. Fox, and A. Farhadi, “Iqa: Visual question answering in interactive environments,” in IEEE Conf. on Computer Vision and Pattern Recognition , 2018

    2018

  28. [36]

    Navigating to objects in the real world,

    T. Gervet, S. Chintala, D. Batra, J. Malik, and D. S. Chaplot, “Navigating to objects in the real world,” Science Robotics, vol. 8, no. 79, 2023

    2023

  29. [37]

    3d-aware object goal navigation via simultaneous exploration and identification,

    J. Zhang et al. , “3d-aware object goal navigation via simultaneous exploration and identification,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2023

    2023

  30. [38]

    Seal: Self-supervised embodied active learning using exploration and 3d consistency,

    D. S. Chaplot, M. Dalal, S. Gupta, J. Malik, and R. R. Salakhutdinov, “Seal: Self-supervised embodied active learning using exploration and 3d consistency,” Advances in neural information processing systems , vol. 34, pp. 13 086–13 098, 2021

    2021

  31. [39]

    Building an affor- dances map with interactive perception,

    L. K. Le Goff, O. Yaakoubi, A. Coninx, and S. Doncieux, “Building an affor- dances map with interactive perception,” Frontiers in Neurorobotics , vol. 16, 2022

    2022

  32. [40]

    Affordance learning from play for sample-efficient policy learning,

    J. Borja-Diaz, O. Mees, G. Kalweit, L. Hermann, J. Boedecker, and W. Burgard, “Affordance learning from play for sample-efficient policy learning,” in IEEE Int. Conf. on Robotics & Automation , 2022

    2022

  33. [41]

    Interactive affordance map building for a robotic task,

    D. Kim and G. Sukhatme, “Interactive affordance map building for a robotic task,” in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems , 2015

    2015

  34. [42]

    An interactive navigation method with effect-oriented affordance,

    X. Wang, Y. Liu, X. Song, Y. Liu, S. Zhang, and S. Jiang, “An interactive navigation method with effect-oriented affordance,” in IEEE Conf. on Computer Vision and Pattern Recognition , 2024

    2024

  35. [43]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017

    2017

  36. [44]

    Segment anything,

    A. Kirillov et al., “Segment anything,” in IEEE Int. Conf. on Computer Vision , 2023

    2023

  37. [45]

    On calibration of modern neural networks,

    C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in Int. Conf. on Machine Learning , 2017

    2017

  38. [46]

    U-net: Convolutional networks for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer- Assisted Intervention, 2015

    2015

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.