Recent Advances and Future Directions in Extended Reality (XR): Exploring AI-Powered Spatial Intelligence

Baichuan Zeng

REVIEW 4 major objections 4 minor 23 references

Recent Advances and Future Directions in Extended Reality (XR): Exploring AI-Powered Spatial Intelligence

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This review argues that XR is heading toward AI-powered spatial intelligence, becoming an active part of daily life rather than a passive display.

desk verdict A competent but undistinguished XR survey whose useful framework and product table are undermined by an unsupported forecast about spatial intelligence. read the letter →

arxiv 2504.15970 v1 pith:JUY6CPB7 submitted 2025-04-22 cs.HC cs.CVcs.MA

Baichuan Zeng This is my paper · ORCID

classification cs.HCcs.CVcs.MA

keywords ExtendedrealityspatialintelligenceaugmentedvirtualmixedmultimodalAIdigitaltwinshuman-computerinteraction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This review sets out to establish that Extended Reality will evolve from a passive display technology into an active, spatially intelligent interface. It argues that combining multimodal AI, IoT-driven digital twins, and adaptive systems will let XR devices understand the physical environment and the user's state, not just render images. The paper supports this forecast by analyzing XR's hardware, algorithm, and interface layers and by comparing state-of-the-art headsets such as Apple Vision Pro and Meta Quest 3. A sympathetic reading treats the central claim as a forward-looking prediction: spatial intelligence is the next frontier in human-computer interaction.

What carries the argument

The load-bearing frame is the paper's three-layer XR architecture (hardware, visual algorithms, and UI/UX) combined with the concept of spatial intelligence. Spatial intelligence is defined as machine perception, interaction, and learning within the 3D world, going beyond basic visual recognition. The architecture organizes the survey of existing technology, while the spatial-intelligence concept carries the forecast: it is the mechanism by which XR becomes an active, adaptive interface rather than a passive screen.

What would settle it

Measure the end-to-end latency, power draw, and cost of a state-of-the-art multimodal language model answering spatial queries about an indoor scene on a device like Apple Vision Pro, against a real-time interaction threshold of roughly 100 milliseconds. If no configuration meets that threshold, the mechanism behind the forecast fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that XR is on the cusp of transforming human-computer interaction by integrating AI-powered spatial intelligence. In the author's account, spatial intelligence means machines that not only perceive the 3D world but also interpret, adapt to, and interact within it, sensing both the environment and nuanced user needs. The paper forecasts that XR systems will evolve from passive display mechanisms to active participants in how we work, learn, and interact, driven by multimodal large language models, IoT-connected digital twins, and adaptive systems that respond to user behavior. It grounds this forecast in a three-layer technical framework of hardware, visual algorithms, and user interface, plus comparative performance data for state-of-the-art headsets.

Load-bearing premise

The forecast depends on AI models with genuine spatial reasoning being embeddable in XR devices so that they can run in real time without unacceptable latency, power, or cost; the review does not test this.

Editorial extensions

If this is right

If the forecast is correct, future XR devices will need to run multimodal AI models capable of spatial reasoning in real time, not just render graphics.
Digital twins will shift from static 3D models to dynamic, IoT-fed representations that remain persistent and shared across sessions and devices.
Interaction will move further from physical controllers toward gaze, gesture, voice, and possibly brain-computer interfaces.
High-end spatial tracking and user experience, as demonstrated by Apple Vision Pro, will need to be delivered while reducing weight, price, and battery-life penalties for mass adoption.
Safety-critical applications such as surgical navigation and workforce training will depend on accurate spatial alignment, making precision a core requirement rather than a luxury.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the forecast is right, an immediate testable consequence is that a multimodal LLM embedded in a headset should be able to answer spatial queries about a room in real time; building such a benchmark would separate the vision from the mechanism.
The paper leaves open whether spatial intelligence must run on-device; an alternative path is split or edge computing, which would relax the latency assumption but add dependence on connectivity.
The digital-twin and IoT direction implies a standards problem: persistent, shared spatial maps require interoperability across devices, a question the paper does not address.
The comparative method could be extended into a longitudinal study: if Apple Vision Pro-class devices improve tracking error metrics while weight and price fall, the transition from passive to active XR becomes measurable.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one thing to know: this is a pure review, not a research contribution. It has no new experiments, datasets, or derivations. What it does well is organize the field into a three-layer framework (hardware, algorithms, UI/UX) and assemble a readable product comparison table. That table, plus the AVP deep-dive, is genuinely useful for someone new to XR and could serve as a starting bibliography. The descriptive parts are broadly consistent with the cited sources, and the structure is logical.

Where it softens is exactly where the stress-test note lands. The paper's closing forecast — XR evolving from passive displays to active, spatially intelligent systems driven by multimodal LLMs — is asserted, not argued. The phrase "realize everything spontaneously, instantaneously and vividly" is hand-waving. There is no latency, power, thermal, or memory budget for on-device spatial reasoning, and the paper's own hardware section concedes that cloud processing has latency problems and edge computing is only a "suitable trade-off." The mechanism is missing. That is a real soft spot, not a manufactured one, because the conclusion leans on it.

There are also two smaller issues. The text says AVP has the "second highest price" and then parenthetically excludes HoloLens 2 as out of manufacturing, but Table 1 includes HoloLens 2 and shows Varjo XR-4 at nearly double the AVP price; the ranking is confusing. Second, "spatial intelligence" is cited to Gardner's human multiple-intelligences theory and then applied to AI systems like World Labs; the term's rhetorical weight in the conclusion exceeds any technical definition in the paper. These are minor in the sense that they do not sink the survey, but they should be fixed in revision.

I do not think the paper is incoherent or a sham. It is an honest, somewhat lightweight overview. But it is not a serious research contribution. The forward-looking section reads like a plausible industry keynote, not a reasoned argument. For a research venue, I would desk reject unless the venue explicitly publishes broad surveys and the authors are willing to add a feasibility analysis and sharper definitions. For a magazine or a tech blog, it is fine. If it came to me as a referee, I would ask for the spatial-intelligence claim to be either qualified substantially or supported with concrete evidence from the headset/LLM literature.

Recommendation: do not send this to serious peer review in its current form. It is a passable survey for a general audience, but it needs real work before it deserves referee time.

Referee Report

4 major / 4 minor

Summary. The paper is a review-style survey of Extended Reality (XR) technology. It organizes the XR ecosystem into three layers—hardware architecture, visual algorithms, and UI/UX—and then uses that framework to compare six current commercial XR headsets, with particular attention to the Apple Vision Pro versus the Meta Quest 3. The paper closes with a Discussion and Conclusion arguing that the future of XR lies in AI-powered spatial intelligence, driven by multimodal AI, IoT-driven digital twins, and adaptive systems, and that XR will evolve from a passive display technology into an active, integral part of everyday life. The descriptive portions of the survey are broadly consistent with the cited literature, but the forward-looking central claim is asserted rather than demonstrated.

Significance. If the descriptive survey is taken on its own terms, the paper provides a compact, accessible snapshot of current XR hardware and algorithms, and its product comparison table is a useful reference point for readers new to the field. The paper's original contribution, however, is its spatial-intelligence thesis, and that thesis currently rests on qualitative speculation rather than evidence or analysis. The paper also ships no machine-checked proofs, code, or quantitative models; its positive value is as a survey and a statement of research directions, not as a tested technical claim. The comparison data and the forward-looking forecast therefore carry the burden of the paper's significance, and both need strengthening.

major comments (4)

[Discussion, Spatial Intelligence] The load-bearing step of the paper's central claim is the sentence 'This is how XR towards spatial intelligence, by utilizing multi-modal LLM to realize everything spontaneously, instantaneously and vividly.' This is asserted without any feasibility argument. The paper's own Hardware Architecture subsection acknowledges that cloud-based processing 'must contend with latency challenges' and describes edge computing only as 'a suitable trade-off,' while Table 1 shows standalone headsets with 2-3 hour battery life and mobile-class processors. No latency, power, thermal, or memory budget is provided for on-device multimodal spatial reasoning, and no benchmark is cited demonstrating that an LLM-scale model can sustain the real-time loop of tracking, scene understanding, user-state estimation, and content generation within headset constraints. Please either supply concrete feasibility evidence or explicitly recast this as an open research question rather than a forecast.
[Discussion, Spatial Intelligence] The term 'spatial intelligence' is used equivocally. The paper cites Gardner's theory of human multiple intelligences (ref. 22) as a basis for spatial intelligence, but then applies the term to machine perception, claiming spatial intelligence 'encompasses not only machine perception of the three-dimensional world but also sophisticated interaction and learning within it.' These are conceptually different constructions: one is a human cognitive faculty, the other is an AI capability. Please disambiguate the two senses and avoid letting Gardner's theory lend implicit empirical support to the AI claim.
[Case Study and Applications, Table 1 and Figure 4] The comparison between the product table and the empirical comparison figure is inconsistent. Table 1 lists the Varjo XR-4, while the text and Figure 4 describe the comparison as involving the Varjo XR-3, a different model. In addition, the normalized metrics APE_r, RPE_r, CSQ-VR_r, and TLX_r are introduced without any definition of the regularization procedure, so the reader cannot verify the relative performance claims. These inconsistencies undermine the paper's central comparative analysis and must be fixed.
[Case Study and Applications, price discussion] The text states that AVP has the 'second highest price' among being-sold XR products, with the parenthetical that Microsoft HoloLens 2 is out of manufacturing. Table 1 still lists HoloLens 2 with a price of $3,500, which is essentially identical to AVP's $3,499, and Varjo XR-4 at $5,990. The claim can be made consistent only if the HoloLens 2 row is explicitly excluded, but the table does not say so. Please clarify the exclusion or revise the price-ranking sentence to match the table.

minor comments (4)

[Throughout] There are several typographical and grammatical slips, including 'tread-offs' for 'trade-offs', 'base on' for 'based on', the stray spacing in 'M R' in the Introduction, and 'being-sold products' for 'currently sold products'. These should be corrected in a careful proofreading pass.
[Figure 4] Figure 4 has no axis labels or legend explaining the normalization of APE_r, RPE_r, CSQ-VR_r, and TLX_r, making the figure hard to interpret on its own. Please add axis labels and a caption that defines each metric.
[Case Study and Applications, Figure 3] The teardown image in Figure 3 is attributed to Lumafield in the references but the figure caption itself does not carry a source credit. Please add proper attribution directly in the caption.
[Introduction] The Introduction defines four layers of the XR ecosystem but the Technical Framework section presents three layers (Hardware, Visual Algorithm, UI/UX). This structural mismatch should be reconciled so that the reader does not encounter two different decompositions of the same space.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a qualitative survey whose forward-looking conclusion is asserted through external citations, not derived from its own inputs.

full rationale

This manuscript is a narrative review of XR hardware, algorithms, UI/UX, and commercial products. It contains no equations, fitted parameters, or quantitative model, so there is no fitted input renamed as a prediction and no quantity that reduces by construction to an input. The concluding forecast that XR 'will evolve from a passive display technology to an integral, active component of our daily lives' is a directional statement supported by qualitative discussion and citations to external work, not by a derivation chain. The Discussion's assertion that multimodal LLMs will realize XR experiences 'spontaneously, instantaneously and vividly' is a feasibility claim rather than a derived result, so it raises an evidence or correctness concern, not a circularity concern. The term 'spatial intelligence' is imported from Gardner's theory of human multiple intelligences and applied to AI systems, which is a conceptual equivocation, but the paper does not claim to derive spatial intelligence from XR or vice versa. No load-bearing argument is justified by a self-citation; the single author does not cite their own prior work. Since the paper is self-contained as a review against external sources and makes no formal predictive claim with a mechanism that could be circular, the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper contributes no new physical or mathematical model, so no fitted parameters or invented entities are introduced. Its conclusions rely on accepted definitions of XR terms, on product specifications taken from vendor pages and VRcompare, and on two external comparison studies; these are listed as domain assumptions rather than independent evidence. The forward-looking claim that AI and digital twins will enable adaptive XR is an assumption about future technology development, not a derived result.

assumptions (4)

domain assumption Product specifications in Table 1, compiled from VRcompare and vendor pages, are accurate and current.
The paper performs no measurements and does not audit vendor claims.
domain assumption The performance comparisons for Apple Vision Pro, Meta Quest 3, and Varjo XR-4 from refs 17 and 18 are valid and representative.
The review's only quantitative evidence for product quality is drawn from these two external studies.
domain assumption The XR ecosystem can be adequately decomposed into hardware architecture, visual algorithms, and UI/UX, and these categories cover the relevant design space.
The framework is presented as given and is not derived or validated against alternative taxonomies.
domain assumption Definitions of AR, VR, MR, and AV from refs 1-3 are accepted.
Background definitions are uncontroversial and used to set terminology.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Recent Advances and Future Directions in Extended Reality (XR): Exploring AI-Powered Spatial Intelligence." pith.science (2026). https://pith.science/paper/JUY6CPB7

@misc{pith2026250415970,
  author       = {Pith},
  title        = {Pith review of: Recent Advances and Future Directions in Extended Reality (XR): Exploring AI-Powered Spatial Intelligence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JUY6CPB7}},
  note         = {Machine review of arXiv:2504.15970}
}

read the original abstract

Extended Reality (XR), encompassing Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR), is a transformative technology bridging the physical and virtual world and it has diverse potential which will be ubiquitous in the future. This review examines XR's evolution through foundational framework - hardware ranging from monitors to sensors and software ranging from visual tasks to user interface; highlights state of the art (SOTA) XR products with the comparison and analysis of performance based on their foundational framework; discusses how commercial XR devices can support the demand of high-quality performance focusing on spatial intelligence. For future directions, attention should be given to the integration of multi-modal AI and IoT-driven digital twins to enable adaptive XR systems. With the concept of spatial intelligence, future XR should establish a new digital space with realistic experience that benefits humanity. This review underscores the pivotal role of AI in unlocking XR as the next frontier in human-computer interaction.

Figures

Figures reproduced from arXiv: 2504.15970 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 22 canonical work pages

[1]

R. T. Azuma. A survey of augmented reality. In Presence: Teleoperators and Virtual Environments 6, 4, 355- 385 (August 1997)

work page 1997
[2]

P. A. Rauschnabel, R. Felix, C. Hinsch, H. Shahab, & F. Alt. What is XR? Towards a framework for augmented and virtual reality. Computers in human behavior, 133, 107289 (2022)

work page 2022
[3]

Milgram, H

P. Milgram, H. Takemura, A. Utsumi, & F. Kishino. Augmented reality: A class of displays on the reality-virtu- ality continuum. In Telemanipulator and telepresence technologies (Vol. 2351, pp. 282-292) (1995)

work page 1995
[4]

Kourtesis

P. Kourtesis. A Comprehensive Review of Multimodal XR Applications, Risks, and Ethical Challenges in the Metaverse. Multimodal Technologies and Interaction, 8(11), 98 (2024)

work page 2024
[5]

V. G. Kini, S. B. Ganeshrao, & P. C. Siddalingaswamy. XR review: A comprehensive analysis of visual func- tion testing and gamification in extended reality environments. IEEE Access (2024)

work page 2024
[6]

Garon, P

M. Garon, P. O. Boulet, J. P. Doiron, L. Beaulieu, & J. F. Lalonde. Real-time high resolution 3D data on the HoloLens. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) (pp. 189-191). IEEE (2016, September)

work page 2016
[7]

Tech Specs, Apple Vision Pro

Apple. Tech Specs, Apple Vision Pro. https://www.apple.com/apple-vision-pro/specs/. (2025)

work page 2025
[8]

X. Qiao, P. Ren, S. Dustdar, L. Liu, H. Ma, & J. Chen. Web AR: A promising future for mobile augmented re- ality—State of the art, challenges, and insights. Proceedings of the IEEE, 107(4), 651-666 (2019)

work page 2019

Show all 23 references

[9]

S. Zhai, N. Wang, X. Wang, D. Chen, W. Xie, H. Bao, & G. Zhang. XR-VIO: High-precision Visual Inertial Odometry with Fast Initialization for XR Applications. arXiv preprint arXiv:2502.01297 (2025)

2025 arXiv
[10]

Lindlbauer, & A

D. Lindlbauer, & A. D. Wilson. Remixed reality: Manipulating space and time in augmented reality. In Pro- ceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1-13) (2018, April)

2018
[11]

Lahtinen, A

T. Lahtinen, A. Costin, & G. Suarez-Tangil. Brain-Computer Interface Integration With Extended Reality (XR): Future, Privacy And Security Outlook. In Proceedings of the European Conference on Cyber Warfare and Se- curity (Vol. 23, No. 1). Academic Conferences International Ltd (2024)

2024
[12]

Tech specs

Meta. Tech specs. Meta Quest 3. https://www.meta.com/quest/quest-3/?srsltid=Afm- BOoo7UR2sdS8avX6H95B7oWRR6qdRuff8lcsxND5eJ2FYmN71POrf#specs (2025)

2025
[13]

Varjo XR-4

Varjo. Varjo XR-4. https://b2b-store.varjo.com/product/xr-4 (2025)

2025
[14]

Xreal. Air 2. https://www.xreal.com/air2/ (2025)

2025
[15]

ThinkReality A3 Smart Glasses

Lenovo. ThinkReality A3 Smart Glasses. https://www.lenovo.com/us/en/p/smart-devices/virtual-reality/thinkre- ality-a3-pc-edition/wmd00000500 (2025)

2025
[16]

https://vr-compare.com/ (2025)

VRcompare -The Internet's Largest VR & AR Headset Database. https://vr-compare.com/ (2025)

2025
[17]

T. Hu, F. Yang, T. Scargill, & M. Gorlatova. Apple vs Meta: A Comparative Study on Spatial Tracking in SOTA XR Headsets. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking (pp. 2120-2127) (2024, December)

2024
[18]

F. Vona, J. Schorlemmer, M. Stern, N. Ashrafi, M. Vergari, T. Kojic, & J. N. Voigt-Antons. Comparing Pass- Through Quality of Mixed Reality Devices: A User Experience Study During Real-World Tasks. arXiv preprint arXiv:2502.06382 (2025)

2025 arXiv
[19]

Apple Vision Pro and Meta Quest Non-Destructive Teardown

Lumafield. Apple Vision Pro and Meta Quest Non-Destructive Teardown. https://www.lumafield.com/arti- cle/apple-vision-pro-meta-quest-pro-3-non-destructive-teardown (2024)

2024
[20]

XR Technology and Consultancy Service

Hong Kong Productivity Council. XR Technology and Consultancy Service. https://www.hkpc.org/en/our-ser- vices/digital-transformation/xr-technology-and-consultancy-service (2025)

2025
[21]

Durante, Q

Z. Durante, Q. Huang, N. Wake, R. Gong, J. S. Park, B. Sarkar, ... & J. Gao. Agent ai: Surveying the horizons of multimodal interaction. arXiv preprint arXiv:2401.03568 (2024)

2024 arXiv
[22]

H. Gardner. The Theory of Multiple Intelligences. Annals of Dyslexia, 37, 19–35. http://www.jstor.org/sta- ble/23769277 (1987)

1987
[23]

https://www.worldlabs.ai/ (2025)

World Labs. https://www.worldlabs.ai/ (2025)

2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

R. T. Azuma. A survey of augmented reality. In Presence: Teleoperators and Virtual Environments 6, 4, 355- 385 (August 1997)

work page 1997

[2] [2]

P. A. Rauschnabel, R. Felix, C. Hinsch, H. Shahab, & F. Alt. What is XR? Towards a framework for augmented and virtual reality. Computers in human behavior, 133, 107289 (2022)

work page 2022

[3] [3]

Milgram, H

P. Milgram, H. Takemura, A. Utsumi, & F. Kishino. Augmented reality: A class of displays on the reality-virtu- ality continuum. In Telemanipulator and telepresence technologies (Vol. 2351, pp. 282-292) (1995)

work page 1995

[4] [4]

Kourtesis

P. Kourtesis. A Comprehensive Review of Multimodal XR Applications, Risks, and Ethical Challenges in the Metaverse. Multimodal Technologies and Interaction, 8(11), 98 (2024)

work page 2024

[5] [5]

V. G. Kini, S. B. Ganeshrao, & P. C. Siddalingaswamy. XR review: A comprehensive analysis of visual func- tion testing and gamification in extended reality environments. IEEE Access (2024)

work page 2024

[6] [6]

Garon, P

M. Garon, P. O. Boulet, J. P. Doiron, L. Beaulieu, & J. F. Lalonde. Real-time high resolution 3D data on the HoloLens. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) (pp. 189-191). IEEE (2016, September)

work page 2016

[7] [7]

Tech Specs, Apple Vision Pro

Apple. Tech Specs, Apple Vision Pro. https://www.apple.com/apple-vision-pro/specs/. (2025)

work page 2025

[8] [8]

X. Qiao, P. Ren, S. Dustdar, L. Liu, H. Ma, & J. Chen. Web AR: A promising future for mobile augmented re- ality—State of the art, challenges, and insights. Proceedings of the IEEE, 107(4), 651-666 (2019)

work page 2019