iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement

Huan Hu; Kai Wang; Kohou Wang; Kun Fan; Lin Bai; Shiguo Lian; Xiang Liu; Zhaoxiang Liu

REVIEW 5 major objections 5 minor 18 references

iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A robot can improve its visual recognition after deployment by learning from ordinary conversations with non-expert users, using clarification questions, retrieval of past corrections, and periodic fine-tuning.

desk verdict A reasonable integration of known components whose reported gains are not credible because user-supplied labels serve as both training signal and evaluation ground truth on the same ten bottles. read the letter →

arxiv 2507.22896 v1 pith:IRQNVWX6 submitted 2025-06-25 cs.HC cs.AIcs.CVcs.RO

Kohou Wang , ZhaoXiang Liu , Lin Bai , Kun Fan , Xiang Liu , Huan Hu , Kai Wang , Shiguo Lian This is my paper

classification cs.HCcs.AIcs.CVcs.RO

keywords interactivelearningmultimodallargelanguagemodelchainofquestiondual-modalityretrievalLoRAfine-tuninghuman-robotdialoguevisualperceptioncontinuousimprovement

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

iLearnRobot is a framework for letting a deployed robot get better at seeing and recognizing objects after it has left the lab. The paper's central claim is that a multimodal large language model can learn from natural dialogues with non-expert users: the robot asks clarifying questions, retrieves similar past interaction events to avoid repeating corrected mistakes, and periodically fine-tunes on the accumulated dialogues. In a study with ten similar medicine bottles, average answer accuracy rose from 0.288 in the initial round to 0.616 with retrieval and 0.716 after fine-tuning, with user satisfaction scores moving in the same direction. The point of the work is that robot perception can keep improving in the field without a dedicated team of expert annotators.

What carries the argument

Three mechanisms carry the argument. The chain-of-question module prompts the MLLM to keep asking clarifying questions until it can state the user's intent, distilling the dialogue into a single clear question and a bounding box. The dual-modality retrieval module encodes the cropped subject region with a CLIP image encoder and the distilled question with a CLIP text encoder, then retrieves a past event when both cosine similarities clear fixed thresholds, feeding the stored answer into the response prompt. The model update module fine-tunes LLaVA-NeXT with LoRA (low-rank adaptation) on the accumulated events; the ablation that unfreezes the visual encoder is the part that lets the model distinguish objects with only tiny pixel differences.

What would settle it

Have a user deliberately supply a wrong label during a round-1 dialogue; if the round-2 retrieval and round-3 fine-tuned model confidently repeat that wrong label, the loop is encoding user error as ground truth.

Watch

Extended reading notes

Core claim

The paper claims that everyday human-robot dialogue can be turned into a self-improvement loop for visual perception. When a user asks about an object, the chain-of-question module first resolves the true intent; the dual-modality retrieval module then checks the history event database for a past interaction matching both the cropped object region and the distilled question, and if found, uses the stored correct answer as a reference. Every interaction is distilled into a question with clear intent, a subject bounding box, and the user-supplied correct answer, stored as embeddings, image, question, and answer. Once enough events accumulate, the robot fine-tunes LLaVA-NeXT with LoRA, and the authors' ablation shows that also updating the visual encoder, rather than freezing it, gives the best accuracy and lowest variance. The measured result is that accuracy on ten novel medicine bottles goes from 0.288 to 0.616 with retrieval, then to 0.716 after fine-tuning.

Load-bearing premise

The whole loop treats whatever answer a user gives during conversation as the correct answer, with no check on whether the user is right.

Editorial extensions

If this is right

Retrieval alone, without any weight update, can lift a robot's answer accuracy substantially between model updates, so users get corrected behavior immediately after a mistake is corrected once.
Periodic fine-tuning on distilled dialogues raises the no-retrieval baseline, meaning the robot's intrinsic perception improves and it no longer depends on the database for every answer.
Unfreezing the visual encoder during fine-tuning is what allows the model to separate visually similar novel objects, so future fine-tuning pipelines for novel perception should not assume frozen vision towers.
The same interactive loop can be applied to any object attribute a user can name, such as color, usage, breed, or shape, not just the name of a medicine bottle.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because user corrections are stored without verification, an implication the paper leaves implicit is that the system's long-run accuracy is capped by the reliability of non-expert labels; a wrong label will be retrieved and then reinforced by fine-tuning.
A natural next experiment would be to test the same pipeline on a broader set of visually similar objects and measure whether the round-3 gain transfers to objects never discussed in any dialogue.
The paper's ablation suggests a testable split: fine-tune with only the visual encoder updated versus only the LLM updated, to attribute the round-3 improvement to perception versus language.
The retrieval threshold is a free parameter; a useful extension would map accuracy and variance against threshold values to find where retrieval starts and stops helping.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Tim,

Quick read: this is a coherent system paper, not a crackpot one. The integration of chain-of-question clarification, CLIP-based dual-modality retrieval of past corrections, and LoRA fine-tuning of LLaVA-NeXT is genuinely assembled rather than just proposed, and the ablation comparing frozen versus updatable visual encoders is a sensible piece of engineering. The writing is clear, the motivation (learning from natural dialogue with non-experts) is real, and the hardware demonstration with medicine bottles is a nice concrete setting. Credit where due.

The problem is the evaluation. The stress-test note lands. Section 3.2 says the system distills whatever the user says into a "Correct Answer" with no verification, conflict resolution, or confidence score. Section 4.1 then asks the same 25 participants to judge whether the robot's answer is correct. So both the training labels and the metric are the participants' own opinions. If a participant misidentifies Thiamine as Riboflavin, that error is stored, retrieved in round 2, and reinforced by fine-tuning in round 3. The reported jump from 0.288 to 0.716 could reflect the model learning to parrot user beliefs, not learning true bottle attributes.

Even setting that aside, the evaluation is too weak to support the "continuous improvement" claim. Round 3 is tested on the same ten bottles used to build the training data, so the fine-tuned model is being measured on its training set. There are no error bars, no significance tests, and no independent ground-truth label set. The retrieval cosine-similarity thresholds are never reported. The variance numbers are interesting (round 2's variance jumps to 8.4, which they attribute to retrieval inconsistency), but without statistical analysis that's just descriptive. This is a small dataset and a single scenario; the generalization claim is unearned.

Who gets value from this? Someone building a similar dialogue-driven robot pipeline might pick up useful design ideas, especially the retrieval-augmented response generation and the visual-encoder-updating ablation. But they should not cite the accuracy numbers as evidence of effectiveness.

Recommendation: I would not accept this as-is, but I would send it to peer review rather than desk-reject. The flaws are addressable—an independent ground-truth set for bottle attributes, held-out bottles for fine-tuning evaluation, error bars, and reported thresholds would make the claims testable. A referee who understands evaluation issues could give the authors a clear path to a respectable revision.

Best, [Your name]

Referee Report

5 major / 5 minor

Summary. The paper proposes iLearnRobot, a framework that integrates interactive learning into an MLLM-based robot. It combines a chain-of-question module to clarify user intent, a dual-modality retrieval module that recalls past interaction events (image-region embeddings and question embeddings) before answering, and periodic LoRA fine-tuning of LLaVA-NeXT on distilled dialogue events. The authors report three rounds of experiments with 25 participants and 10 medicine bottles: accuracy rises from 0.288 in the baseline round to 0.616 with retrieval and to 0.716 after fine-tuning, with corresponding dialogue scores of 3.34, 5.38, and 7.1. An ablation compares freezing versus updating the visual encoder during fine-tuning. The central claim is that the robot can continuously improve its visual perception and recognition by learning from natural dialogues with non-expert users rather than relying on expert-annotated datasets.

Significance. If properly validated, the framework would address a genuine and under-studied problem: adapting a deployed robot's visual perception and recognition to novel domains by exploiting everyday human corrections, while maintaining a good interaction experience through clarification and retrieval. The architecture is simple, the qualitative examples are illustrative, and comparing visual-encoder freezing versus updating is a useful practical insight. However, the current empirical evidence does not support the central claim because the evaluation protocol is circular: the fine-tuned model is assessed on the same medicine bottles used to construct its training data, and the ground truth for both training and evaluation is supplied by the same non-expert participants without independent verification. The paper also omits key implementation details such as retrieval thresholds and lacks statistical validation. These are load-bearing issues that require a re-designed evaluation rather than cosmetic fixes.

major comments (5)

[§4.1, Fig. 12 and §3.3] The round-3 evaluation is performed on the same 10 medicine bottles whose interaction events were used to fine-tune LLaVA-NeXT, so the reported accuracy of 0.716 is at least partly a measure of training-data memorization rather than generalization to novel scenarios, which is the paper's stated motivation. Please add a held-out evaluation, for example leave-one-bottle-out or a new set of bottles and new participants, and report per-bottle and per-question accuracy for that split.
[§3.2 Data Construction and §4.1 evaluation protocol] The stored 'Correct Answer' is extracted from user interaction without any verification, and the evaluation score is also recorded by the same non-expert participants who hold the bottles. This makes the training signal and the evaluation ground truth the same subjective judgment, so the measured improvement may reflect the model learning to reproduce participant beliefs rather than true bottle attributes. I request independent ground-truth labels, for example labels verified against manufacturer information, and a separate evaluation of whether wrong user labels propagate through retrieval and fine-tuning.
[§3.1 Dual-Modality Retrieval] The retrieval success condition is described as 'If the cosine similarities each meet a certain specific threshold,' but the threshold values are never reported or analyzed. Without these values and a sensitivity analysis, the round-2 result of 0.616 cannot be reproduced or assessed; please report the thresholds, how they were chosen, and how retrieval precision and recall vary with them.
[§4.1, Table 1] The accuracy and dialogue-score results are reported as point estimates without error bars, confidence intervals, or significance tests. The round-to-round differences are based on the same 25 participants answering questions about the same 10 bottles, so the dependencies should be accounted for with paired tests or bootstrap resampling. Please include per-question counts and appropriate statistical tests.
[§3.2 Data Construction] The claim that the system avoids repeating mistakes is not tested against the failure mode that matters most for the proposed interactive-learning loop: a non-expert user supplies a wrong label. The paper does not describe any confidence scoring, conflict resolution, or evaluation of label quality, and Fig. 12 suggests confusions such as Thiamine versus Riboflavin. Please add a quantitative analysis of user-label accuracy and an explicit treatment of conflicting or incorrect corrections.

minor comments (5)

[§3.1] The paper states d=576 for the CLIP ViT-B/16 embedding dimension, but the standard CLIP ViT-B/16 image and text embedding dimension is 512; please verify the checkpoint and dimension.
[Table 1a] The entry '5.387.1' appears to be a typographical join of 5.38 and 7.1; please separate the values clearly.
[General] The paper does not state whether code, prompts beyond those shown, or collected data are available, which makes the reported experiments difficult to reproduce.
[§1 and Fig. 1] Several grammatical slips need correction, such as 'similar event' in the Fig. 1 caption and 'both quantitively and qualitatively' in the abstract.
[§4] The description of the medicine set says 'Vitamin B1 (Thiamine), Vitamin B6 (Riboflavin), etc.'; the example labels are confusing because Thiamine and Riboflavin are easy to confuse, and the source of the ground-truth bottle attributes is never defined.

Circularity Check

3 steps flagged · score 6.0 of 10

Reported accuracy gains are measured against the same user-supplied labels that form the retrieval and fine-tuning data, so the central improvement claims reduce partly to stored-answer lookup and in-distribution memorization.

self definitional [Section 3.2 (Data Construction) and Section 4 (Experiments)]
"Correct Answer: The system identifies the correct answer to the user’s query about the subject... This answer ensures that the robot can correctly identify and respond to the user’s query in later dialogues... Participants will record two scores based on each dialogue experience: 1).score of whether the robot generated the correct answer, which is simply right or wrong, which will be calculated for the overall accuracy"

The stored 'Correct Answer' Ai is extracted from the same natural dialogue whose outcome the participant later scores as right or wrong; no independent label set, verification step, or external ground truth is introduced. The accuracy metric is therefore defined as agreement with the user's own supplied answer, which is exactly the signal stored in the database and later used for retrieval and fine-tuning. Reported gains are by construction consistency with user beliefs rather than correctness against true bottle attributes.
other [Section 3.1 Dual-Modality Retrieval; Section 4 Round 2]
"If the cosine similarities each meet a certain specific threshold, it is considered a successful retrieval of a past event from the database. The correct answers included in this past event will then serve as a reference for the current question. ... In the second round, as there was already a database of past interaction events from the first round available for retrieval, the robot, even without training on this data, was able to significantly improve its performance by leveraging dual-modality retrieval."

Round 2 accuracy is produced by retrieving the stored 'correct answer' from a first-round event matched by cosine similarity and using it as the reference for the current answer. Since the stored answer was itself judged correct by the same user who evaluates the second-round response, a successful retrieval makes the judged outcome correct by construction. The 0.288-to-0.616 gain is a lookup success rate over previously stored user answers, not a prediction of unseen facts.

1 more flagged steps

fitted input called prediction [Section 3.3 Model Update; Section 4 Round 3]
"The training data, in the format of (Ii, Qi, Ai), is then fed into the LLaVA-NeXT model, which is fine-tuned one epoch from LLaVA checkpoints with LoRA [4]... After the first two rounds of dialogue, the data, stored in the history event database, will be used for fine-tuning the robot... The participants, again with the 10 bottles, will ask questions, record the answers, and score the dialogue process and the response."

Round 3 tests the same 10 medicine bottles and the same participant-provided labels that were used to build the fine-tuning set, because every participant interacted with all 10 bottles in rounds 1 and 2 and those interactions were stored as (Ii, Qi, Ai). The reported 0.716 accuracy on the measured set can therefore be achieved by memorizing the training labels; it is not a held-out test of generalization to new bottles or new user corrections.

full rationale

The system's components — chain-of-question prompting, CLIP-based dual-modality retrieval, and LoRA fine-tuning — are ordinary engineering choices and are not circular per se. The circularity enters in the empirical demonstration. The 'Correct Answer' field is extracted from the same non-expert dialogue that later supplies the right/wrong judgment (Section 3.2 versus Section 4), so there is no independent ground truth. Round 2 is a retrieval of that stored answer, making the improvement a lookup success rate rather than a model prediction. Round 3 fine-tunes and evaluates on the same 10 bottles and same participant labels, so the accuracy gain may be memorization rather than generalization. These are the load-bearing supports for the claims of 'avoid repeating same mistakes' and 'continuous improvement'. If the authors had used an independent label set or held-out bottles, the claims would be testable; as reported, the central quantitative evidence reduces to consistency with the user-supplied answers that constitute the training signal. No load-bearing self-citation was found.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced beyond software modules built from existing components. The central claim rests on several unverified domain assumptions, most notably the reliability of user corrections and the validity of the small in-sample evaluation protocol.

free parameters (3)

Retrieval cosine-similarity thresholds = not reported
The dual-modality retrieval module accepts an event when image and text cosine similarities each meet 'a certain specific threshold' (Section 3.1); the thresholds are never specified, so the retrieval success and all round-2 results depend on unstated hand-chosen values.
Model-update trigger threshold = not reported
Model update begins when accumulated events reach 'a predefined threshold' (Section 3.3); the threshold is not given and affects how quickly the system incorporates new corrections.
LoRA fine-tuning hyperparameters = not reported
The fine-tuning practice is described as following LLaVA's script with minor modification, but rank, alpha, learning rate, and number of steps are not reported; the round-3 results depend on these choices.

assumptions (4)

domain assumption User-provided corrections are treated as ground-truth correct answers.
Section 3.2 'Data Construction' states verbal corrections are 'invaluable' and stores them as the correct answer; no mechanism verifies them, so the learning loop assumes non-expert users are reliable labelers.
domain assumption CLIP embedding space aligns image crops and question text such that cosine similarity retrieves semantically relevant past events.
Section 3.1 'Dual-Modality Retrieval' relies on CLIP embeddings and unstated thresholds; the paper provides no evaluation that the right events are retrieved.
domain assumption Coarse one-epoch LoRA fine-tuning on a small dataset improves LLaVA-NeXT's perception on novel objects.
The method assumes a single epoch of fine-tuning on a few dozen interaction events is sufficient to update visual recognition; only an ablation on frozen vs. updatable visual encoder is reported, and generalization is untested.
domain assumption Participant scores on a 1-10 scale are a valid measure of dialogue quality and model accuracy.
Section 4 treats average scores and variances as evidence of performance, but no inter-rater reliability or validation of the scale is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement." pith.science (2026). https://pith.science/paper/IRQNVWX6

@misc{pith2026250722896,
  author       = {Pith},
  title        = {Pith review of: iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IRQNVWX6}},
  note         = {Machine review of arXiv:2507.22896}
}

read the original abstract

It is crucial that robots' performance can be improved after deployment, as they are inherently likely to encounter novel scenarios never seen before. This paper presents an innovative solution: an interactive learning-based robot system powered by a Multi-modal Large Language Model(MLLM). A key feature of our system is its ability to learn from natural dialogues with non-expert users. We also propose chain of question to clarify the exact intent of the question before providing an answer and dual-modality retrieval modules to leverage these interaction events to avoid repeating same mistakes, ensuring a seamless user experience before model updates, which is in contrast to current mainstream MLLM-based robotic systems. Our system marks a novel approach in robotics by integrating interactive learning, paving the way for superior adaptability and performance in diverse environments. We demonstrate the effectiveness and improvement of our method through experiments, both quantitively and qualitatively.

Figures

Figures reproduced from arXiv: 2507.22896 by the authors.

**Figure 2.** Illustration of our chain of question module. The integration of MLLMs into robotics has been a focal point. These approaches strive to empower robots to process and understand various types of data, and primarily focus on the domains of navigation and manipulation. In the field of navigation, Chi et al [1] proposes an interactive learning framework to endow the agent with the ability to ask for users’ help, which… view at source ↗

**Figure 3.** Prompt used for chain of question. The subsequent dialogue for this que [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (10 more)

**Figure 4.** Figure 4: Prompt used for distilling data. These user natural dialogues are distilled [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Illustration of our dual-modality retrieval module. precise intent. This question is also encoded into an embedding form using the CLIP text encoder. Dual-Modality Retrieval. In real-world applications, robotic systems will inevitably make various mistakes. However, co…

**Figure 6.** Figure 6: Illustration of our response generation module. to extract the text embedding etext ∈ Rd of Qi . The symbol d refers to the feature dimension (e.g., d = 576 for CLIP ViT-B/16). These embeddings are then used to calculate the cosine similarity with 1) embeddings of the …

**Figure 8.** Figure 8: figure 8. This module processes complete user interactions, which include multi- [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

**Figure 7.** Figure 7: Left: prompt and the generated responses. Right: responses generated [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

**Figure 8.** Figure 8: Illustration of our data construction module. – Subject Bounding-Box: The system identifies the bounding box of the subject within the image associated with the user’s query. The bounding box allows the system to accurately crop the image to focus on the relevant area …

**Figure 9.** Figure 9: Prompt used for distilling natural dialogs into a concise data format that [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

**Figure 10.** Figure 10: A typical scenario of our experiment. An illustration is shown in figure 11. In the first round, the robot is restricted from retrieving any past interaction events, it just utilizes an ordinary public pre-trained MLLM equipped with our chain of question prompts; In t…

**Figure 11.** Figure 11: An illustration of our experiment flow. representing greater satisfaction. The participants will also record whether the robot successfully generated the correct answer to the question. Round 2: Retrieval Test. In this round, after the first round of experiment, a co…

**Figure 12.** Figure 12: Accuracy. VB for Vitamin B, CPM for Chlorpheniramine Maleate. The [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 9 canonical work pages

[1]

In: Proceedings of the AAAI conference on artificial intelligence

Chi, T.C., Shen, M., Eric, M., Kim, S., Hakkani-Tur, D.: Just ask: An interac- tive learning framework for vision and language navigation. In: Proceedings of the AAAI conference on artificial intelligence. vol. 34, pp. 2459–2466 (2020)

work page 2020
[2]

arXiv preprint arXiv:2003.04641 (2020)

Deng, Y., Guo, D., Guo, X., Zhang, N., Liu, H., Sun, F.: Mqa: Answering the question via robotic manipulation. arXiv preprint arXiv:2003.04641 (2020)

arXiv 2020
[3]

arXiv preprint arXiv:2303.03378 (2023)

Driess, D., Xia, F., Sajjadi, M.S., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al.: Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378 (2023)

arXiv 2023
[4]

arXiv preprint arXiv:2106.09685 (2021)

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

arXiv 2021
[5]

Journal of Management Analytics 7(2), 139–172 (2020)

Kang,Y.,Cai,Z.,Tan,C.W.,Huang,Q.,Liu,H.:Naturallanguageprocessing(nlp) in management research: A literature review. Journal of Management Analytics 7(2), 139–172 (2020)

work page 2020
[6]

In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Kenfack, F.K., Siddiky, F.A., Balint-Benczedi, F., Beetz, M.: Robotvqa—a scene- graph-and deep-learning-based visual question answering system for robot manip- ulation. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 9667–9674. IEEE (2020)

work page 2020
[7]

Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., Lee, Y.J.: Llava-next: Improved reasoning, ocr, and world knowledge (2024)

2024
[8]

Advances in neural information processing systems36, 2434–2447 (2024)

Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems36, 2434–2447 (2024)

work page 2024

Show all 18 references

[9]

In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction

Menezes, J.C.: Mumodar: Multi-modal framework for human-robot collaboration in cyber-physical systems. In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction. pp. 755–759 (2024) 5 The authors have no competing interests to declare that are relev...

2024
[10]

Sensors20(3), 722 (2020)

Müller, S., Wengefeld, T., Trinh, T.Q., Aganian, D., Eisenbach, M., Gross, H.M.: A multi-modal person perception framework for socially interactive mobile service robots. Sensors20(3), 722 (2020)

2020
[11]

arXiv preprint arXiv:2303.09014 (2023)

Paranjape, B., Lundberg, S., Singh, S., Hajishirzi, H., Zettlemoyer, L., Ribeiro, M.T.: Art: Automatic multi-step reasoning and tool-use for large language models. arXiv preprint arXiv:2303.09014 (2023)

2023 arXiv
[12]

In: International conference on machine learning

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)

2021
[13]

In: 2019 International Conference on Robotics and Automation (ICRA)

Thomason, J., Padmakumar, A., Sinapov, J., Walker, N., Jiang, Y., Yedidsion, H., Hart, J., Stone, P., Mooney, R.J.: Improving grounded natural language un- derstanding through human-robot dialog. In: 2019 International Conference on Robotics and Automation (ICRA). pp. 6934–694...

2019
[14]

In: 2023 IEEE International Conference on Big Data (BigData)

Wu, J., Gan, W., Chen, Z., Wan, S., Philip, S.Y.: Multimodal large language models: A survey. In: 2023 IEEE International Conference on Big Data (BigData). pp. 2247–2256. IEEE (2023)

2023
[15]

arXiv preprint arXiv:2303.11381 (2023)

Yang, Z., Li, L., Wang, J., Lin, K., Azarnasab, E., Ahmed, F., Liu, Z., Liu, C., Zeng, M., Wang, L.: Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303.11381 (2023)

2023 arXiv
[16]

100211 (2024)

Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., Zhang, Y.: A survey on large language model(llm)securityandprivacy:Thegood,thebad,andtheugly.High-Confidence Computing p. 100211 (2024)

2024
[17]

arXiv preprint arXiv:2311.10678 (2023)

Zha, L., Cui, Y., Lin, L.H., Kwon, M., Arenas, M.G., Zeng, A., Xia, F., Sadigh, D.: Distilling and retrieving generalizable knowledge for robot manipulation via language corrections. arXiv preprint arXiv:2311.10678 (2023)

2023 arXiv
[18]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Zhu, F., Zhu, Y., Chang, X., Liang, X.: Vision-language navigation with self- supervised auxiliary reasoning tasks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10012–10022 (2020)

2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

In: Proceedings of the AAAI conference on artificial intelligence

Chi, T.C., Shen, M., Eric, M., Kim, S., Hakkani-Tur, D.: Just ask: An interac- tive learning framework for vision and language navigation. In: Proceedings of the AAAI conference on artificial intelligence. vol. 34, pp. 2459–2466 (2020)

work page 2020

[2] [2]

arXiv preprint arXiv:2003.04641 (2020)

Deng, Y., Guo, D., Guo, X., Zhang, N., Liu, H., Sun, F.: Mqa: Answering the question via robotic manipulation. arXiv preprint arXiv:2003.04641 (2020)

arXiv 2020

[3] [3]

arXiv preprint arXiv:2303.03378 (2023)

Driess, D., Xia, F., Sajjadi, M.S., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al.: Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378 (2023)

arXiv 2023

[4] [4]

arXiv preprint arXiv:2106.09685 (2021)

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

arXiv 2021

[5] [5]

Journal of Management Analytics 7(2), 139–172 (2020)

Kang,Y.,Cai,Z.,Tan,C.W.,Huang,Q.,Liu,H.:Naturallanguageprocessing(nlp) in management research: A literature review. Journal of Management Analytics 7(2), 139–172 (2020)

work page 2020

[6] [6]

In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Kenfack, F.K., Siddiky, F.A., Balint-Benczedi, F., Beetz, M.: Robotvqa—a scene- graph-and deep-learning-based visual question answering system for robot manip- ulation. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 9667–9674. IEEE (2020)

work page 2020

[7] [7]

Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., Lee, Y.J.: Llava-next: Improved reasoning, ocr, and world knowledge (2024)

2024

[8] [8]

Advances in neural information processing systems36, 2434–2447 (2024)

Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems36, 2434–2447 (2024)

work page 2024