Pith. sign in

REVIEW 4 major objections 5 minor 59 references

In LLM-assisted VR scene editing, users systematically underestimate confabulations, and under high error load their awareness saturates so that perceived error counts become uncorrelated with actual ones—meaning human oversight cannot serv

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A 24-user VR study identifies nine types of LLM scene-editing confabulations and shows users systematically underestimate their frequency under high error load.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection Useful empirical taxonomy and a real construct, but the headline claim about saturated user vigilance is partly built on a mismatched questionnaire, and the 'actual' ground truth needs reliability evidence. the 4 major comments →

arxiv 2607.23213 v1 pith:6O3CTU2M submitted 2026-07-25 cs.HC

A Taxonomy of Confabulations and the Perception-Reality Gap in LLM-Assisted Immersive Scene Editing

Junlong Chen , Per Ola Kristensson This is my paper
classification cs.HC
keywords confabulationhallucinationLLMvirtual reality3D scene editingperception-reality gaphuman-AI collaborationerror detection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that when people edit 3D scenes in virtual reality with an LLM-based assistant, the system frequently produces confabulations—plausible but incorrect edits—and, crucially, that users cannot be relied on to notice them. In a study of 24 non-expert users across three VR scenes, 27 percent of interaction turns contained at least one confabulation, and users consistently under-reported them: in the highest-error scene, the median perceived count was 4.0 while the median actual count was 10.5, and individual perceived counts were not significantly correlated with actual counts. The paper concludes that confabulation awareness saturates under load, making human vigilance structurally unreliable as a detection mechanism and putting the burden of mitigation on the system. It also contributes a nine-type taxonomy of confabulations and a construct, the perception-reality gap, for measuring this divergence.

Core claim

The paper's central discovery is an empirical demonstration that user awareness of LLM errors in immersive editing is bounded: in low-error conditions people track confabulations well (Spearman rho=0.71), but in high-error conditions perceived counts stop tracking actual counts (office rho=0.15; city rho=0.30, n.s.), a pattern the authors call saturation. They arrive at this by building a JSON-in/JSON-out LLM scene-editing workflow, logging 1,612 interaction turns from 24 users, manually coding every turn for nine confabulation types and 24 subtypes (437 confabulation turns total), and comparing these actual counts with users' post-experience questionnaire reports. The discovered asymmetry i

What carries the argument

The load-bearing construct is the perception-reality gap: the discrepancy between the number and types of confabulations users believe they encountered (measured by post-experience questionnaire) and those that actually occurred (measured by manual log coding). The companion mechanism is saturation: as confabulation load rises, perceived counts plateau and lose correlation with actual counts, rendering human detection ineffective. The taxonomy of nine confabulation types (spatial placement errors, reference frame ambiguity, orientation errors, command misinterpretation without disclosure, false scene claims, edit scope errors, material/style errors, missing information without disclosure, an

Load-bearing premise

The reference counts of 'actual' confabulations come from a single researcher's manual coding of all 1,612 interaction turns, with no second coder, no inter-rater reliability, and no formal protocol beyond inductive open coding, so every saturation and perception-reality result depends on that labeling being correct.

What would settle it

Re-code the 1,612 interaction turns with two independent coders following a pre-registered protocol, or automatically detect confabulations by mechanical checks (e.g., geometric overlap detection, out-of-region placement, comparison of claimed vs. applied edits). If a second coding pass yields materially different counts—or if the automatic check argues with the manual labels—the saturation and uncorrelated-counts results could vanish. A simpler check: a direct replication with twice the participants plus instrumented ground-truth logging would show whether perceived counts again plateau near

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • System-side verification becomes the primary safeguard: scene state should be re-checked against the LLM's prior claims before each new interaction round, rather than trusting either the model's text or the user's memory.
  • Confident textual outputs from the LLM should not be treated as evidence that an edit succeeded; independent structural validation is needed for claimed deletions, undos, and placements.
  • Mitigation can exploit the perceptual asymmetry: spatial errors are self-revealing in VR, so effort should concentrate on semantic interpretation errors and silent failures, which users rarely notice.
  • Task framing and the presence of a reference state shape error tolerance: stylistic tasks without a clear reference induced higher trust and lower perceived disruption, suggesting task design can be a mitigation lever.
  • Out-of-vocabulary internal tokens (like the '_st' suffix) should be flagged as unknown rather than silently mapped to a plausible meaning, since such silent misreadings dominated the stylistic task.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The saturation finding likely generalizes beyond VR to any human-AI interface where error rates are high, implying that user vigilance can be modeled as a limited-capacity channel; a quantitative model of that capacity could predict when monitoring will fail.
  • The 'actual' counts could be obtained automatically in future work—using geometric collision checks and out-of-region tests to flag confabulations by rule—which would let the perception-reality gap be studied at scale without manual labeling and would test whether the manual coding explains the results.
  • The taxonomy, being task-derived rather than model-derived, may transfer to other structured-data LLM agents (scene graphs, CAD, code) where silent misinterpretation and false completion claims are likely to appear in similar proportions.
  • The 46 percent false-alarm rate for object omission in the city scene, driven by false 'already done' claims, hints that users infer failure from system denial; studying denial-based inference could turn misleading output into a diagnostic signal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents an exploratory user study (N=24) of LLM-assisted immersive 3D scene editing, using a JSON-in/JSON-out GPT-5.2 pipeline in three VR scenes (office, loft, city) with distinct task types. Based on manual review of 1,612 interaction turns, the authors construct a nine-type taxonomy of confabulations, report their prevalence and perceived disruptiveness, and introduce the construct of a perception–reality gap by comparing log-based actual confabulation counts with post-experience questionnaire reports. The central claim is that under high confabulation load, user awareness saturates and becomes uncorrelated with actual counts, so human oversight is structurally unreliable and mitigation must be system-side. The paper also derives four design implications (DI1–DI4) and acknowledges its own limitations, including a footnote admitting a structural blind spot in the questionnaire.

Significance. If the empirical foundation holds, the paper makes a timely contribution: it is one of the few studies to characterize LLM confabulations in immersive scene editing from real interaction data, and it offers a concrete taxonomy plus a measurable construct (perception–reality gap) that can be reused by other XR/LLM researchers. The design implications are actionable and grounded in observed failure modes. The authors are also transparent about the questionnaire mismatch and about the limited power for type-level analyses. However, the two load-bearing pillars—the manual coding of ground-truth confabulations and the comparability of perceived versus actual counts—are not yet sufficiently validated. The central conclusion about saturation therefore requires additional evidence before it can be accepted at face value.

major comments (4)
  1. [§4.1] The ground-truth actual confabulation counts rest entirely on manual coding by the authors: 'Each interaction turn was manually reviewed ... to identify whether confabulation was present,' and labels were assigned via inductive open coding. No inter-rater reliability, second coder, codebook, or shared annotation dataset is reported. Since every prevalence percentage, every actual-count median, and every perception–reality statistic is computed from this coding, a systematically lenient or strict coder would change the headline results. The authors should report at least a reliability check (e.g., Cohen's kappa on a random subset, dual coding of a sample) and make the coding protocol or annotated data available. This is a necessary validity check for the central claim, not a stylistic suggestion.
  2. [§4.3, footnote 3] The perceived-confabulation instrument is systematically incomplete relative to the taxonomy. Footnote 3 states that FSC, AEF, and MSE 'did not correspond to questionnaire items, which resulted in a structural blind spot.' Yet actual counts include those categories: FSC accounts for 8% of confabulation turns in the city scene, FSC+AEF account for 10% in the office scene, and MSE accounts for 25% in the loft scene. Because the perceived-count measure cannot register these types by construction, the observed underestimation and the non-significant correlations in office/city may be inflated by instrument asymmetry rather than by a genuine perceptual ceiling. The paper acknowledges this in a footnote but still uses the result to claim that 'user vigilance is structurally unreliable as a detection mechanism' (Section 5). The authors should provide a sensitivity analysis that excludes or sepa
  3. [§4.3] The statistical evidence for 'saturation' and 'unrelated' counts is weaker than the prose suggests. The office (ρ=0.15, p=.47) and city (ρ=0.30, p=.16) Spearman correlations are non-significant, but N=24 gives low power, and ρ=0.30 is a moderate effect; non-significance does not establish that perceived and actual counts are unrelated. Moreover, no formal test of saturation is presented—the claim rests on comparing medians and correlations across scenes, with no model of a plateau or a nonlinearity. The authors should either fit a saturation model, test for a difference in correlation magnitudes across high/low-load conditions, or soften the 'structurally unreliable' conclusion to 'not reliably correlated in this sample.'
  4. [§3.4 / §4.3] The operationalization of the perceived confabulation count is underspecified. The post-experience questionnaire asked about 'the type, frequency level of disruption, and a short description of the confabulation,' but it is not stated exactly how these answers were converted into a numeric perceived count, how multi-type reports were handled, or whether participants were forced to choose from predefined categories. Because the perception–reality gap is the paper's central construct, the exact instrument wording, the mapping rules from questionnaire categories to taxonomy types, and the aggregation rule should be reported in enough detail for replication.
minor comments (5)
  1. [§4.2] Typo: 'χ2 =32,8' should presumably be 'χ2 =32.8'.
  2. [§3.4 / §4.2] The disruption question uses the term 'hallucination' ('When a hallucination occurred, it disrupted my task'), while the paper deliberately defines and uses 'confabulation.' Please make the terminology consistent, or explain why the questionnaire used a different term.
  3. [Figure 5] The legend for hatched cells combines two different conditions ('category absent in this scene' and 'questionnaire type could not be mapped to a taxonomy type'). These should be separated visually or in the caption, because they carry different implications for interpretability.
  4. [§4.1] Table 1 notes that counts do not sum to totals because interaction turns can receive multiple labels. It would help to also report the number of turns with only one label versus multiple labels, since multi-label coding affects how the prevalence percentages should be read.
  5. [§5.2] The Limitations section mentions the small sample and single-system design, but does not mention the absence of inter-rater reliability or the questionnaire blind spot. Adding both to the limitations would help calibrate reader expectations.

Circularity Check

0 steps flagged

No significant circularity; the perception-reality gap is an empirical comparison of independent measurements, not a construction, and the only self-citations are background.

full rationale

The paper's derivation chain is empirical rather than formal. The taxonomy (Section 4.1) is built by inductive open coding of 1,612 logged interaction turns; perceived confabulations (Section 4.3) come from a separate post-experience questionnaire. The perception-reality gap is the difference between these two independently collected measurements, and no parameter is fitted from one to predict the other. The saturation claim (Section 5) is therefore not circular: it is an interpretation of observed counts and correlations, and it does not reduce to the definition of either measure. The acknowledged 'structural blind spot' (footnote 3: FSC, AEF, and MSE did not correspond to questionnaire items) is a genuine validity limitation that should temper the strength of the perceived-actual comparison, but it is disclosed and it does not make the central gap equivalent to an input by construction; the largest actual category (SPE) is present in the questionnaire, and the gap persists in the count comparison. Self-citations ([6], [7]) support background claims about LLM+XR editing and reference-frame ambiguity; they are not load-bearing for the taxonomy, the gap analysis, or the design implications. Accordingly, no circular step is identified.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 2 invented entities

The paper does not fit any numeric parameters to data; its contributions are qualitative/empirical categories and comparative statistics. The main burden is therefore not parametric but definitional: the 'actual confabulations' anchor is author-based manual coding, justified as inductive open coding without inter-rater reliability, and the perceived-report instrument was designed before the taxonomy existed, creating a known structural blind spot for three confabulation types.

axioms (4)
  • domain assumption The manual log-labeling of 1,612 interaction turns into confabulation presence/type is treated as an objective ground truth.
    All prevalence counts, detection rates, and perception-reality-gap analyses (Section 4.1, Table 1, Figure 5) depend on these labels, yet no inter-rater reliability or independent coding is reported.
  • domain assumption The specific GPT-5.2 model with the authors' prompt and Unity pipeline is a representative testbed for general LLM-assisted immersive scene editing.
    Generalization from a single system and model to other workflows is asserted in the introduction and discussion, but only one model/pipeline is tested (Section 3.1, 5.2).
  • domain assumption Post-hoc questionnaire categories can be mapped to the inductively derived taxonomy for comparing actual vs perceived confabulation types.
    Section 4.3 Footnote 3 states that several observed types (FSC, AEF, MSE) had no questionnaire counterpart, producing a 'structural blind spot' that biases type-level comparisons.
  • domain assumption The three scenes and three task types span the relevant design space of LLM-assisted immersive 3D scene editing.
    The paper claims the scenes are designed to reflect many use cases (Section 3.3), but the conclusions about task-type-specific confabulations rest on one indoor object-placement scene, one stylistic scene, and one outdoor functional scene.
invented entities (2)
  • Perception-reality gap independent evidence
    purpose: A construct quantifying the discrepancy between the number/types of confabulations users believe they encountered versus those that actually occurred.
    It is operationally defined through two independent measurements (questionnaire reports of perceived confabulations vs manual log annotation of actual confabulations) and the paper offers a falsifiable handle: it predicts underestimation that saturates under load. This is an empirical construct, not an unobserved entity.
  • Nine-type confabulation taxonomy independent evidence
    purpose: Classifies LLM system failures in immersive scene editing into clusters/types/subtypes (SPE, RFA, FSC, CMD, ESE, ORI, MSE, MID, AEF).
    The taxonomy is grounded in logged interaction data and could be applied by independent coders to new datasets, making it falsifiable, though the paper itself provides no reliability evidence.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of A Taxonomy of Confabulations and the Perception-Reality Gap in LLM-Assisted Immersive Scene Editing." pith.science (2026). https://pith.science/paper/6O3CTU2M

@misc{pith2026260723213,
  author       = {Pith},
  title        = {Pith review of: A Taxonomy of Confabulations and the Perception-Reality Gap in LLM-Assisted Immersive Scene Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6O3CTU2M}},
  note         = {Machine review of arXiv:2607.23213}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large language models (LLMs) are being increasingly integrated into immersive environments and design workflows, providing application prospects in areas such as rapid scene prototyping for non-expert users and scene understanding capabilities for accessibility design. While many workflows that incorporate LLMs in immersive spaces are proposed, such systems can exhibit errors, potentially resulting in frustration, loss of user trust, and compromised user safety. This paper studies the underexplored area of LLM confabulations in immersive 3D scene editing contexts. Through an exploratory study with 24 non-expert users, we construct a taxonomy of the different types of confabulation observed in LLM-assisted immersive 3D scene editing. We report their prevalence and disruptiveness, and define the construct perception-reality gap to help understand the gap between the actual and perceived occurrence of confabulations. We highlight the observed saturation of confabulation awareness under load and conclude by discussing design implications for confabulation mitigation in future LLM-assisted systems in immersive 3D scenes.

Figures

Figures reproduced from arXiv: 2607.23213 by Junlong Chen, Per Ola Kristensson.

Figure 1
Figure 1. Figure 1: Examples of different types of confabulations identified through an empirical user study of LLM-assisted immersive scene [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Workflow of the LLM-assisted immersive scene editing system adopted in the study. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: User interface of the LLM-assisted immersive scene editing system with the main panel containing selected objects and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Violin plots of the user perceived trust ratings on a scale of 1 to 7 (left), unweighted NASA-TLX overall load from a scale of 1 to [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Actual and perceived confabulation counts in both trials of all participants in the office, loft, and city scenes (left) with [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 8 linked inside Pith

  1. [1]

    Saleema Amershi, Dan Weld, Mihaela V orvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. 2019. Guidelines for human-AI interaction. InProceedings of the 2019 chi conference on human factors in computing systems. 1–13

    2019

  2. [2]

    Tita Alissa Bach, Amna Khan, Harry Hallock, Gabriela Beltrão, and Sonia Sousa. 2024. A systematic literature review of user trust in AI-enabled systems: An HCI perspective.International Journal of Human–Computer Interaction40, 5 (2024), 1251–1266

    2024

  3. [3]

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. 2024. Hallucination of multimodal large language models: A survey.arXiv preprint arXiv:2404.18930(2024). Manuscript submitted to ACM A Taxonomy of Confabulations and the Perception-Reality Gap in LLM-Assisted Immersive Scene Editing 17

    Pith/arXiv arXiv 2024

  4. [4]

    John Brooke et al. 1996. SUS-A quick and dirty usability scale.Usability evaluation in industry189, 194 (1996), 4–7

    1996

  5. [5]

    Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making.Proceedings of the ACM on Human-computer Interaction5, CSCW1 (2021), 1–21

    2021

  6. [6]

    Junlong Chen, Rosella P Galindo Esparza, Vanja Garaj, Per Ola Kristensson, and John Dudley. 2025. EnVisionVR: A scene interpretation tool for visual accessibility in virtual reality.IEEE Transactions on Visualization and Computer Graphics(2025)

    2025

  7. [7]

    Junlong Chen, Jens Grubert, and Per Ola Kristensson. 2025. Analyzing multimodal interaction strategies for llm-assisted manipulation of 3d scenes. In 2025 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 206–216

    2025

  8. [8]

    Jiangong Chen, Xiaoyi Wu, Tian Lan, and Bin Li. 2025. Llmer: Crafting interactive extended reality worlds with json data generated by large language models.IEEE Transactions on Visualization and Computer Graphics(2025)

    2025

  9. [9]

    Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang, Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, and Guosheng Lin. 2024. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 21476–21485

    2024

  10. [10]

    Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. 2024. Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis16, 1 (2024), 64–93

    2024

  11. [11]

    Fernanda De La Torre, Cathy Mengying Fang, Han Huang, Andrzej Banburski-Fahey, Judith Amores Fernandez, and Jaron Lanier. 2024. Llmr: Real-time prompting of interactive worlds using large language models. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–22

    2024

  12. [12]

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. 2023. LayoutGPT: Compositional visual planning and generation with large language models.Advances in Neural Information Processing Systems36 (2023), 18225–18250

    2023

  13. [13]

    Xueyang Feng, Zhi-Yuan Chen, Yujia Qin, Yankai Lin, Xu Chen, Zhiyuan Liu, and Ji-Rong Wen. 2024. Large language model-based human-agent collaboration for complex task solving. InFindings of the Association for Computational Linguistics: EMNLP 2024. 1336–1357

    2024

  14. [14]

    Andrew U Frank. 1998. Formal models for cognition—taxonomy of spatial location description and frames of reference. InSpatial cognition: An interdisciplinary approach to representing and processing spatial knowledge. Springer, 293–312

    1998

  15. [15]

    Daniele Giunchi, Nels Numan, Elia Gatti, and Anthony Steed. 2024. DreamCodeVR: Towards Democratizing Behavior Design in Virtual Reality with Speech-Driven Programming. In2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 579–589

    2024

  16. [16]

    Alexandra-Elena Gurita and Radu-Daniel Vatavu. 2025. Breaking Bad (Design): Challenging AI User Interface Accessibility Guardrails. InProceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–7

    2025

  17. [17]

    Hart and Lowell E

    Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, V ol. 52. North-Holland, 139–183. doi:10.1016/ S0166-4115(08)62386-9

    1988

  18. [18]

    Teresa Hirzle, Florian Müller, Fiona Draxler, Martin Schmitz, Pascal Knierim, and Kasper Hornbæk. 2023. When XR and AI Meet - A Scoping Review on Extended Reality and Artificial Intelligence. InProceedings of the 2023 CHI conference on human factors in computing systems. 1–45

    2023

  19. [19]

    Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A Ross, Cordelia Schmid, and Alireza Fathi. 2024. Scenecraft: An LLM agent for synthesizing 3D scenes as blender code. InForty-first International Conference on Machine Learning

    2024

  20. [20]

    Ian Huang, Guandao Yang, and Leonidas Guibas. 2024. BlenderAlchemy: Editing 3D Graphics with Vision-Language Models. InEuropean Conference on Computer Vision. Springer, 297–314

    2024

  21. [21]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al

  22. [22]

    Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, et al. 2023. Language is not all you need: Aligning perception with language models.Advances in Neural Information Processing Systems36 (2023), 72096–72109

    2023

  23. [23]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation.ACM computing surveys55, 12 (2023), 1–38

    2023

  24. [24]

    Ryutaro Kurai, Takefumi Hiraki, Yuichi Hiroi, Yutaro Hirao, Monica Perusquía-Hernández, Hideaki Uchiyama, and Kiyoshi Kiyokawa. 2025. MagicItem: Dynamic Behavior Design of Virtual Objects With Large Language Models in a Commercial Metaverse Platform.IEEE Access(2025)

    2025

  25. [25]

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Evaluating object hallucination in large vision-language models. InProceedings of the 2023 conference on empirical methods in natural language processing. 292–305

    2023

  26. [26]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning.Advances in neural information processing systems36 (2023), 34892–34916

    2023

  27. [27]

    Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. 2024. A survey on hallucination in large vision-language models.arXiv preprint arXiv:2402.00253(2024)

    Pith/arXiv arXiv 2024

  28. [28]

    Meta. 2025. Interaction SDK Overview | Meta Horizon OS Developers. https://developers.meta.com/horizon/documentation/unity/unity-isdk- interaction-sdk-overview/. Online, accessed 08-March-2026. Manuscript submitted to ACM 18 Chen et al

    2025

  29. [29]

    Caterina Moruzzi and Solange Margarido. 2024. A user-centered framework for human-ai co-creativity. InExtended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–9

    2024

  30. [30]

    Mahmud Omar, Vera Sorin, Jeremy D Collins, David Reich, Robert Freeman, Nicholas Gavin, Alexander Charney, Lisa Stump, Nicola Luigi Bragazzi, Girish N Nadkarni, et al. 2025. Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support.Communications Medicine5, 1 (2...

    2025

  31. [31]

    Ruiying Peng, Kaiyuan Li, Weichen Zhang, Chen Gao, Xinlei Chen, and Yong Li. 2025. Understanding and evaluating hallucinations in 3d visual language models.arXiv preprint arXiv:2502.15888(2025)

    Pith/arXiv arXiv 2025

  32. [32]

    Giuseppe Romeo and Daniela Conti. 2026. Exploring automation bias in human–AI collaboration: a review and implications for explainable AI.AI & SOCIETY41, 1 (2026), 259–278

    2026

  33. [33]

    Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. 2024. A comprehensive survey of hallucination in large language, image, video and audio foundation models.Findings of the Association for Computational Linguistics: EMNLP 2024(2024), 11709–11724

    2024

  34. [34]

    Johnny Saldaña. 2021. The coding manual for qualitative researchers. (2021)

    2021

  35. [35]

    Philipp Schmidt, Felix Biessmann, and Timm Teubner. 2020. Transparency and trust in artificial intelligence systems.Journal of Decision Systems29, 4 (2020), 260–278

    2020

  36. [36]

    Md Shamsujjoha, Qinghua Lu, Dehai Zhao, and Liming Zhu. 2025. Swiss cheese model for ai safety: A taxonomy and reference architecture for multi-layered guardrails of foundation model based agents. In2025 IEEE 22nd International Conference on Software Architecture (ICSA). IEEE, 37–48

    2025

  37. [37]

    Ben Shneiderman and Pattie Maes. 1997. Direct manipulation vs. interface agents.interactions4, 6 (1997), 42–61

    1997

  38. [38]

    Zhenyu Shu, Junlong Yu, Kai Chao, Shiqing Xin, and Ligang Liu. 2025. Gaussedit: Adaptive 3d scene editing with text and image prompts.IEEE Transactions on Visualization and Computer Graphics(2025)

    2025

  39. [39]

    Andrew L Smith, Felix Greaves, and Trishan Panch. 2023. Hallucination or confabulation? Neuroanatomy as metaphor in large language models. PLOS Digital Health2, 11 (2023), e0000388

    2023

  40. [40]

    Fan-Yun Sun, Weiyu Liu, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, and Jiajun Wu. 2025. LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models. InProceedings of the Computer Vision and Pattern Recognition Conference. 29469–29478

    2025

  41. [41]

    Yiliu Tang, Jason Situ, Andrea Yaoyun Cui, Mengke Wu, and Yun Huang. 2025. LLM Integration in Extended Reality: A Comprehensive Review of Current Trends, Challenges, and Future Perspectives. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–24

    2025

  42. [42]

    SMTI Tonmoy, SM Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. 2024. A comprehensive survey of hallucination mitigation techniques in large language models.arXiv preprint arXiv:2401.013136 (2024)

    Pith/arXiv arXiv 2024

  43. [43]

    Lukas Twist, Jie M Zhang, Mark Harman, and Helen Yannakoudakis. 2025. Library Hallucinations in LLMs: Risk Analysis Grounded in Developer Queries.arXiv preprint arXiv:2509.22202(2025)

    Pith/arXiv arXiv 2025

  44. [44]

    Hongbo Wang, Jie Cao, Jin Liu, Xiaoqiang Zhou, Huaibo Huang, and Ran He. 2024. Hallo3d: Multi-modal hallucination detection and mitigation for consistent 3d content generation.Advances in Neural Information Processing Systems37 (2024), 118883–118906

    2024

  45. [45]

    Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, and Philip S Yu. 2023. Multimodal large language models: A survey. In2023 IEEE International Conference on Big Data (BigData). IEEE, 2247–2256

    2023

  46. [46]

    Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2024. Hallucination is inevitable: An innate limitation of large language models.arXiv preprint arXiv:2401.11817(2024)

    Pith/arXiv arXiv 2024

  47. [47]

    Dongil Yang, Minjin Kim, Sunghwan Mac Kim, Beong-woo Kwak, Minjun Park, Jinseok Hong, Woontack Woo, and Jinyoung Yeo. 2025. Llm meets scene graph: Can large language models understand and generate scene graphs? a benchmark and empirical study. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)...

    2025

  48. [48]

    Jianing Yang, Xuweiyi Chen, Nikhil Madaan, Madhavan Iyengar, Shengyi Qian, David F Fouhey, and Joyce Chai. 2025. 3d-grand: A million-scale dataset for 3d-llms with better grounding and less hallucination. InProceedings of the Computer Vision and Pattern Recognition Conference. 29501–29512

    2025

  49. [49]

    Yixuan Yang, Junru Lu, Zixiang Zhao, Zhen Luo, Wanxi Dong, Victor Sanchez, and Feng Zheng. 2025. LLplace: Embodied 3D Indoor Layout Synthesis Framework with Large Language Model. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 20685–20691

    2025

  50. [50]

    John Zerilli, Umang Bhatt, and Adrian Weller. 2022. How transparency modulates trust in artificial intelligence.Patterns3, 4 (2022)

    2022

  51. [51]

    Lei Zhang, Jin Pan, Jacob Gettig, Steve Oney, and Anhong Guo. 2024. Vrcopilot: Authoring 3d layouts with generative ai models in vr. InProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. 1–13

    2024

  52. [52]

    Shuning Zhang, Hui Wang, and Xin Yi. 2025. Exploring collaboration patterns and strategies in human-ai co-creation through the lens of agency: A scoping review of the top-tier hci literature.Proceedings of the ACM on Human-Computer Interaction9, 7 (2025), 1–43

    2025

  53. [53]

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2025. Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models.Computational Linguistics51, 4 (2025), 1373–1418

    2025

  54. [54]

    Ziyao Zhang, Chong Wang, Yanlin Wang, Ensheng Shi, Yuchi Ma, Wanjun Zhong, Jiachi Chen, Mingzhi Mao, and Zibin Zheng. 2025. Llm hallucinations in practical code generation: Phenomena, mechanism, and mitigation.Proceedings of the ACM on Software Engineering2, ISSTA Manuscript submitted to ACM A Taxonomy of Confabulations and the Perception-Reality Gap in L...

    2025

  55. [55]

    Yang Zhao, Shizhao Sun, Meisheng Zhang, Yingdong Shi, Xubo Yang, and Jiang Bian. 2026. SceneReVis: A Self-Reflective Vision-Grounded Framework for 3D Indoor Scene Synthesis via Multi-turn RL.arXiv preprint arXiv:2602.09432(2026)

    arXiv 2026

  56. [56]

    Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, Kevin Lin, Jianfeng Wang, Lijuan Wang, and Xin Wang. 2025. EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing. InInternational Conference on Learning Representations, V ol. 2025. 86791–86808

    2025

  57. [57]

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2023. Analyzing and mitigating object hallucination in large vision-language models.arXiv preprint arXiv:2310.00754(2023)

    Pith/arXiv arXiv 2023

  58. [58]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592(2023). Manuscript submitted to ACM

    Pith/arXiv arXiv 2023

  59. [2025]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems43, 2 (2025), 1–55

    2025

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.