Pith. sign in

REVIEW 5 major objections 5 minor 87 references

A3FR: Agile 3D Gaussian Splatting with Incremental Gaze Tracked Foveated Rendering in Virtual Reality

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A3FR claims that running gaze tracking on the CPU in parallel with incremental 3D Gaussian Splatting rendering on the GPU reduces end-to-end foveated-rendering latency by up to 2x while preserving perceived visual quality.

desk verdict A3FR is a genuine systems contribution with credible latency gains, but its visual-quality claim rests on a static experiment that does not exercise the live gaze pipeline. read the letter →

arxiv 2507.04147 v1 pith:TNRV4F7L submitted 2025-07-05 cs.GR cs.CVcs.DC

Shuo Xin , Haiyu Wang , Sai Qian Zhang This is my paper
classification cs.GRcs.CVcs.DC
keywords gaze-trackedfoveatedrendering3DGaussianSplattingearly-exitneuralnetworksincrementallatencyoptimizationvirtualreality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Foveated rendering in virtual reality saves computation by rendering only a small high-resolution region where the user is looking, but the gaze tracker that decides where that region sits adds its own delay. This paper argues that the usual serial pipeline—sense, send, track, render, display—is unnecessarily slow because gaze tracking and rendering do not need to wait for each other. The proposed A3FR framework runs the gaze-tracking network on the CPU while 3D Gaussian Splatting renders on the GPU, and gives the tracker early-exit heads so it can emit a rough gaze position quickly and refine it later. Rendering starts with the rough position and is corrected incrementally as later predictions arrive, so the total frame latency becomes the slower of tracking or rendering rather than their sum. The paper reports up to a 2x end-to-end latency reduction at high resolutions with no user-visible quality loss, based on a user study on a commercial VR headset.

What carries the argument

The load-bearing object is A3FR-ViT, a six-block vision transformer with a linear gaze-prediction head after every block, trained with a multi-exit loss so early heads give a quick rough gaze direction and later heads refine it. The incremental foveal-region rule of Eq. (4) uses the training-data expectation of the distance between early and final predictions to cap each round's high-resolution radius, ensuring early-rendered pixels lie inside the final foveal region. Adaptive mesh refinement (AMR) renders only a subset of pixels within each $2\times2$ tile depending on eccentricity from the gaze center, giving four resolution levels, and speculative incremental rendering lets the GPU keep rendering around a stale gaze prediction up to radius $r_j^{\max}=r_{f,N}+E[dist(u_j,u_N)]$ before it has to wait.

What would settle it

Run A3FR on a real headset with live gaze tracking during natural saccades, measure the actual early-exit error distances, and compare end-to-end latency and a perceptual quality metric against full-resolution rendering; if the deployment errors exceed the offline expectation used in Eq. (4), the foveal regions will be mis-sized and the claimed 2x-latency-without-quality-loss tradeoff will fail.

Watch

Extended reading notes

Core claim

The paper's central claim is that gaze-tracked foveated rendering can be restructured from a serial chain $T_{tot}=T_s+T_c+T_d+T_r$ into a parallel composition $T_{tot}=T_s+T_c+\max(T_d,T_r)$ without wasting the early rendering work. The mechanism is an incremental rendering rule: when an early-exit gaze prediction $u_i$ is known to be within a statistical distance $E(dist(u_i,u_N))$ of the final prediction, the radius of the high-resolution region rendered at round $i$ is capped at $r_{f,i}=\max(0, r_{f,N}-E(dist(u_i,u_N)))$, so the region rendered on a rough prediction is already inside the final foveal region rather than being thrown away. With the gaze tracker on the CPU, the 3DGS renderer on the GPU, and speculative rendering to cover CPU slowdowns, the evaluation reports consistent latency reductions across four scenes and three resolutions, reaching up to 2x at high resolution.

Load-bearing premise

The foveal region sizes in Eq. (4) assume the gaze-error distribution at runtime matches the distribution in the offline eye-tracking training data; if real headsets have larger or differently shaped errors, regions will be too small (visible blur) or too large (speedup lost).

Editorial extensions

If this is right

  • Because total latency becomes $\max(T_d,T_r)$ rather than $T_d+T_r$, the gaze tracker's cost stops adding to the critical path whenever rendering is the slower stage, which is the case at high resolution.
  • Early predictions are not wasted work: enforcing the radius rule means every high-resolution region rendered from a rough gaze estimate lies inside the eventual foveal region, so rendering budget accumulates across rounds.
  • Speculative incremental rendering keeps the GPU busy when the CPU is slow, so the latency benefit survives uneven CPU load instead of collapsing under jitter.
  • At 1440p, AMR contributes roughly a 10 percent additional latency saving on top of the parallelization, and the two-interval forced-choice study found no reliable perceptual preference between A3FR and full-resolution rendering.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the gaze-error distribution on a deployed headset differs from the offline training data, the radius rule will mis-size foveal regions; a runtime calibration or error-feedback loop would be needed to keep both the quality and the speedup promises in real units.
  • The CPU/GPU overlap idea is renderer-agnostic in principle, though the buffered reuse of 3DGS preprocessing is specific to that algorithm; other tile-based renderers could adopt the same early-exit scheduling.
  • The 2x ceiling depends on rendering being slower than gaze tracking; at lower resolutions or with a faster tracker the overlap gain shrinks, as the paper's own 720p results show.
  • The user study used a fixed simulated gaze point, so a live-tracked study with natural saccades would be the test of whether incremental refinement keeps up when the gaze target moves.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper proposes A3FR, a framework for gaze-tracked foveated rendering with 3D Gaussian Splatting in VR. The main idea is to offload an early-exit ViT gaze tracker to the CPU and run it in parallel with GPU-side incremental 3DGS rendering; early predictions trigger low-resolution peripheral rendering and later predictions refine foveal regions. The authors also add an adaptive mesh refinement (AMR) scheme that varies per-tile pixel density with eccentricity. They report latency comparisons against full-resolution and sequential foveated baselines on simulated Jetson-like hardware, an ablation of speculative incremental rendering, and a 2IFC user study with eight participants. The headline claim is up to 2x end-to-end latency reduction without visible quality loss.

Significance. If the claims were fully validated, the paper would be a useful systems contribution: it identifies a real bottleneck (gaze-tracking latency serialized with rendering), gives a clean formulation of the total latency as Ts+Tc+max(Td,Tr), and provides modular components (early-exit gaze ViT, incremental rendering with Eq. 3-5, AMR) that are each ablated. The latency experiments are internally consistent, and the SIR and AMR ablations are informative. The credibility of the central claim, however, is currently limited by validation gaps: the user study does not exercise the live gaze-tracked pipeline, and the latency experiments do not run on a real HMD with an actual eye camera. The strength of the paper lies in its architecture and offline measurements; the missing live validation is the main barrier to acceptance.

major comments (5)
  1. [§4.7] The 2IFC user study validates static foveated images with a predefined fixed gaze point, not the A3FR system as described in Section 3.3. Because participants are instructed to fixate a marked point and the A3FR image is generated from that known point, the study bypasses both the early-exit gaze predictions and the incremental refinement rounds. Consequently, the abstract's claim that A3FR maintains visual quality is an extrapolation: the temporal and spatial errors caused by early-exit gaze mispredictions and by the moving foveal patch are never measured. Please either run a live gaze-tracked user study on an HMD or restrict the quality claim to the AMR rendering strategy.
  2. [§3.3.2, Eq. 4-5] The size of each incremental foveal region and the speculative rendering radius are set using E(dist(u_i,u_N)) measured on OpenEDS training data. This is a legitimate calibration step, but there is no validation that the gaze-error distribution transfers to the deployment setting (different eye camera, lighting, users, headset geometry). If the real distribution is heavier-tailed or biased, Eq. 4 will make the early foveal regions too small, causing visible blur, or too large, eroding the latency gain. A concrete test would be to measure dist(u_i,u_N) on the target HMD's eye camera during live operation and compare the resulting radii; until then, the 'no visible quality loss' and the speedup numbers are conditional on a distributional assumption.
  3. [§4.3, Fig. 11] The abstract's 'up to 2x' is not the appropriate comparison for the proposed parallelization. Section 4.3 reports average speedups of about 20% over SFR and 40% over FRR at 1080p/1440p, and Figure 11 shows that the 2x figure arises only when A3FR is compared with full-resolution rendering (FRR), which is not the sequential foveated baseline. The contribution of the paper is the parallel/incremental scheme, so the headline should be reported relative to SFR, and the FRR comparison should be labeled as a combined foveation-plus-parallelism gain.
  4. [§4.1, §4.7] There is no end-to-end measurement on a real HMD with a live eye camera. The latency experiments use a desktop GPU/CPU with clock and core counts adjusted to approximate Jetson Orin NX, while the user study uses a Quest Pro only as a display for static images. The claimed latency model T_tot = T_s + T_c + max(T_d, T_r) depends on actual CPU/GPU overlap and communication, which should be demonstrated with real sensing and display I/O, or at least with a hardware-in-the-loop setup. Without this, the central end-to-end latency claim is not fully validated.
  5. [§2.4 and related work] The closest cited prior art, MetaSapiens [40], is not compared empirically. Since MetaSapiens also targets real-time foveated neural rendering with efficiency-aware pruning, a direct latency/quality comparison is needed to substantiate the claimed improvements over the state of the art.
minor comments (5)
  1. [§4.2] The text says 'with 20% of tokens pruned, A3FR-ViT achieves an end-to-end latency of 26.28ms. Increasing the pruning ratio to 20% further reduces the latency to 21.64ms'; the first percentage should be 10%.
  2. [Algorithm 1] The line numbering is duplicated (lines 4 and 5 appear twice) and the shared variable S is written by Process 1 and read by Process 2 without any synchronization primitives; the pseudocode should be cleaned up.
  3. [§3.3.1] Equation 4 is referenced before it is introduced ('determined by Equation 4'); renumber or reorder the equations so that the reference points to a defined quantity.
  4. [Figure 12] The label 'EM' is used in the latency breakdown figures but is never defined in the text.
  5. [Table 2] The sentence 'The lower two rows for each model in Table 2 report the by-layer and cumulative latencies' is inaccurate because each model block has three rows (Error, Latency, Cumulative).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the 2x latency claim is an end-to-end measurement, and Eq. 4 is a calibrated sizing rule rather than a restatement of the result.

full rationale

The paper's derivation chain is not circular. The foveal radius sizing in Eq. 4, r_f,i = max(0, r_f,N - E(dist(u_i,u_N))), is an offline calibration of rendering regions to the expected gaze-error distance measured on OpenEDS; it is a design input, and the reported latency gains are then measured end-to-end against FRR and SFR in Sec. 4.3, not algebraically forced by Eq. 4. The containment inequality Eq. 3 is a geometric definition of 'no wasted rendering' (render only inside the final foveal disc), not a prediction. The self-citations in Table 1 ([42][43], FovealNet from the same group) motivate the CPU-offload design, but the SFR baseline is independently timed in Fig. 11, so the sequential-vs-parallel comparison does not reduce to the citation. The main empirical limitation is Sec. 4.7: the user study uses 'a pre-defined fixed point to simulate the gaze direction,' so it validates the AMR resolution falloff under a known gaze, not the live incremental early-exit gaze-tracked rendering; this is an evidence gap (and the 'up to 2x' headline is against full-resolution rendering, with only 20-40% gain over the sequential foveated baseline), but it is an overclaim/correctness risk, not a circular derivation. Extraneous extracted table fragments in the text do not affect the argument.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim relies on a small number of design constants calibrated on OpenEDS, and on domain assumptions about human vision and VR SoC resource sharing. No new physical entities are introduced; A3FR-ViT is a network architecture, not a new ontological object.

free parameters (5)
  • E(dist(u_i,u_N)) = computed from OpenEDS training set (not reported numerically)
    Used in Eq. 4 to set incremental foveal radii; if the real gaze-error distribution differs, the latency/quality tradeoff changes. This is a data-calibrated constant, not a fitted result.
  • lambda_n (early-exit loss weights) = not reported in text
    Eq. 2 weights each early-exit gaze loss; the choice affects early-exit accuracy and downstream rendering efficiency.
  • token pruning threshold sigma = not reported
    Tokens with attention score below sigma are discarded; Table 2 tests 10% and 20% pruning ratios, but the threshold value is not given.
  • M (selected exit points) = varies; example shows M=2
    Offline profiling selects a subset of N exits to synchronize CPU and GPU; the selection rule is not specified in detail.
  • foveal radius r_f,N = based on theta_i=18 degrees and Delta theta (Eq. 1)
    Defines the highest-resolution region; depends on assumed acuity angle and tracking error margin.
assumptions (5)
  • standard math Euclidean circle-containment geometry (Eq. 3)
    The claim that no rendering is wasted if r_f,i <= max(0, r_f,N - dist(u_i,u_N)) follows from geometry of disks.
  • domain assumption Human visual acuity falloff with eccentricity and 18 degree foveal angle
    Section 2.2 and Eq. 1 adopt theta_i=18 degrees from Lin et al. [40]; if this angle is wrong for the target HMD, the quality/latency tradeoff shifts.
  • domain assumption CPU and GPU resources on the HMD SoC can execute gaze DNN and 3DGS concurrently without interference
    Section 3.1 assumes offloading gaze tracking to CPU yields T_tot = T_s + T_c + max(T_d,T_r); contention is only partly handled by speculative rendering.
  • domain assumption OpenEDS gaze-error distribution is representative of online deployment
    Section 3.3.2 uses E(dist(u_i,u_N)) from OpenEDS for all rendering-radius decisions; no on-device recalibration is evaluated.
  • domain assumption Un-rendered pixels in peripheral tiles can be interpolated from rendered neighbors without visible artifacts
    Section 3.4 AMR strategy renders only a subset of pixels per tile; the paper relies on the user study to validate this, but only for static images.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A3FR: Agile 3D Gaussian Splatting with Incremental Gaze Tracked Foveated Rendering in Virtual Reality." pith.science (2026). https://pith.science/paper/TNRV4F7L

@misc{pith2026250704147,
  author       = {Pith},
  title        = {Pith review of: A3FR: Agile 3D Gaussian Splatting with Incremental Gaze Tracked Foveated Rendering in Virtual Reality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TNRV4F7L}},
  note         = {Machine review of arXiv:2507.04147}
}
abstract

Virtual reality (VR) significantly transforms immersive digital interfaces, greatly enhancing education, professional practices, and entertainment by increasing user engagement and opening up new possibilities in various industries. Among its numerous applications, image rendering is crucial. Nevertheless, rendering methodologies like 3D Gaussian Splatting impose high computational demands, driven predominantly by user expectations for superior visual quality. This results in notable processing delays for real-time image rendering, which greatly affects the user experience. Additionally, VR devices such as head-mounted displays (HMDs) are intricately linked to human visual behavior, leveraging knowledge from perception and cognition to improve user experience. These insights have spurred the development of foveated rendering, a technique that dynamically adjusts rendering resolution based on the user's gaze direction. The resultant solution, known as gaze-tracked foveated rendering, significantly reduces the computational burden of the rendering process. Although gaze-tracked foveated rendering can reduce rendering costs, the computational overhead of the gaze tracking process itself can sometimes outweigh the rendering savings, leading to increased processing latency. To address this issue, we propose an efficient rendering framework called~\textit{A3FR}, designed to minimize the latency of gaze-tracked foveated rendering via the parallelization of gaze tracking and foveated rendering processes. For the rendering algorithm, we utilize 3D Gaussian Splatting, a state-of-the-art neural rendering technique. Evaluation results demonstrate that A3FR can reduce end-to-end rendering latency by up to $2\times$ while maintaining visual quality.

Figures

Figures reproduced from arXiv: 2507.04147 by the authors.

Figure 1
Figure 1. The framework of gaze-tracked foveated rendering. (a) Sub-network 2 Sub-network 3 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) A3FR framework. (b) and (c) depict the workflow [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. (a) Trace of gaze location. (b) Human eye motion [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: (a) Relationship between tracking error and foveal region size. (b) Meta Quest Pro Head-Mounted Display (HMD). (c) [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: An example of 3DGS Process. (a) Given a camera [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: (a) Multi-resolution training framework. (b) An [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Illustration of the proposed parallel TFR procedure for a simple eye-tracking model only 1 early exit. In round 1, the [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: An illustration of the A3FR operation is provided: [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: The adaptive rendering strategy of A3FR involves [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Left: Joint and marginalized distribution of gaze [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: (a) Comparison of rendering latency for A3FR [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: (a) Latency breakdown of A3FR framework. For 3DGS, the gray bar represents the preprocessing latency, while the [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 13
Figure 13. Figure 13: (a) Ablation study of AMR strategies. A3FR ren [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 15
Figure 15. Figure 15: Participants are conducting the user study on the Quest Pro. User 1 User 2 User 3 User 4 User 5 User 6 User 7 User 8 Total 0% 20% 40% 60% 80% 100% TFR Full Res [PITH_FULL_IMAGE:figures/full_fig_p012_15.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

87 extracted references · 55 canonical work pages

  1. [40]

    Weikai Lin, Yu Feng, and Yuhao Zhu. 2024. MetaSapiens: Real-Time Neural Rendering with Efficiency-Aware Pruning and Accelerated Foveated Rendering. https://doi.org/10.1145/3669940.3707227 arXiv:2407.00435 [cs.GR]

    arXiv 2024

  2. [1]

    [n. d.]. NVIDIA Jetson Orin. https://www.nvidia.com/en-us/autonomous- machines/embedded-systems/jetson-orin/

  3. [2]

    [n. d.]. What is Mobile Industry Processor Interface (MIPI) Proto- col? https://www.synopsys.com/blogs/chip-design/what-is-mobile-industry- processor-interface-protocol.html

  4. [3]

    Abdullah M Al-Ansi, Mohammed Jaboob, Askar Garad, and Ahmed Al-Ansi. 2023. Analyzing augmented reality (AR) and virtual reality (VR) recent development in education. Social Sciences & Humanities Open 8, 1 (2023), 100532

    2023

  5. [4]

    Rachel Albert, Anjul Patney, David Luebke, and Joohwan Kim. 2017. Latency requirements for foveated rendering in virtual reality. ACM Transactions on Applied Perception (TAP) 14, 4 (2017), 1–13

    2017

  6. [5]

    Anastasios N Angelopoulos, Julien NP Martel, Amit PS Kohli, Jorg Conradt, and Gordon Wetzstein. 2020. Event based, near eye gaze tracking beyond 10,000 hz. arXiv preprint arXiv:2004.03577 (2020)

    arXiv 2020

  7. [6]

    Rishi Athavale, Lakshmi Sritan Motati, and Rohan Kalahasty. 2022. One Eye is All You Need: Lightweight Ensembles for Gaze Estimation with Single Encoders. arXiv:2211.11936 [cs.CV] https://arxiv.org/abs/2211.11936

    arXiv 2022

  8. [7]

    Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P

    Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. 2021. Mip-NeRF: A Multiscale Repre- sentation for Anti-Aliasing Neural Radiance Fields. In ICCV. 5835–5844

    2021

Show all 87 references
  1. [8]

    Barron, Ben Mildenhall, Dor Verbin, Pratul P

    Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. 2022. Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. In CVPR. 5460–5469

    2022

  2. [9]

    Barron, Ben Mildenhall, Dor Verbin, Pratul P

    Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. 2023. Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields. In ICCV. 19697–19705

    2023

  3. [10]

    Berger and Joseph Oliger

    Marsha J. Berger and Joseph Oliger. 1984. Adaptive Mesh Refinement for Hyperbolic Partial Differential Equations. J. Comput. Phys. 53 (1984), 484. https://doi.org/10.1016/0021-9991(84)90073-1

    1984 doi

  4. [11]

    Fergus W Campbell and Robert H Wurtz. 1978. Saccadic omission: why we do not see a grey-out during a saccadic eye movement. Vision research 18, 10 (1978), 1297–1303

    1978

  5. [12]

    Chaudhary, Rakshit Kothari, Manoj Acharya, Shusil Dangi, Nitinraj Nair, Reynold Bailey, Christopher Kanan, Gabriel Diaz, and Jeff B

    Aayush K. Chaudhary, Rakshit Kothari, Manoj Acharya, Shusil Dangi, Nitinraj Nair, Reynold Bailey, Christopher Kanan, Gabriel Diaz, and Jeff B. Pelz. 2019. RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking. In 2019 IEEE/CVF International Conference on Compute...

    2019

  6. [13]

    Rajeswari Chengoden, Nancy Victor, Thien Huynh-The, Gokul Yenduri, Rutvij H Jhaveri, Mamoun Alazab, Sweta Bhattacharya, Pawan Hegde, Praveen Ku- mar Reddy Maddikunta, and Thippa Reddy Gadekallu. 2023. Metaverse for healthcare: a survey on potential applications, challenges and...

    2023

  7. [14]

    Woranipit Chidsin, Yanlei Gu, and Igor Goncharenko. 2021. AR-based navigation using RGB-D camera and hybrid map. Sustainability 13, 10 (2021), 5585

    2021

  8. [15]

    Alexey Dosovitskiy. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

    2020 arXiv

  9. [16]

    2017.Eye tracking methodology: Theory and practice

    Andrew T Duchowski and Andrew T Duchowski. 2017.Eye tracking methodology: Theory and practice. Springer

    2017

  10. [17]

    Kara J Emery, Marina Zannoli, James Warren, Lei Xiao, and Sachin S Talathi. 2021. OpenNEEDS: A dataset of gaze, head, hand, and scene signals during exploration in open-ended VR environments. In ACM Symposium on Eye Tracking Research and Applications. 1–7

    2021

  11. [18]

    Jasper H Fabius, Alessio Fracasso, Tanja CW Nijboer, and Stefan Van der Stigchel

  12. [19]

    Yu Feng, Nathan Goulding-Hotta, Asif Khan, Hans Reyserhove, and Yuhao Zhu

  13. [20]

    Stephan J Garbin, Yiru Shen, Immo Schuetz, Robert Cavin, Gregory Hughes, and Sachin S Talathi. 2019. Openeds: Open eye dataset.arXiv preprint arXiv:1905.03702 (2019)

    2019 arXiv

  14. [21]

    Jaris Gerup, Camilla B Soerensen, and Peter Dieckmann. 2020. Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review. International journal of medical education 11 (2020), 1

    2020

  15. [22]

    Antonin Gilles, Pierre Le Gargasson, Grégory Hocquet, and Patrick Gioia. 2023. Holographic near-eye display with real-time embedded rendering. In SIGGRAPH Asia 2023 Conference Papers . 1–10

    2023

  16. [23]

    Dan Witzner Hansen and Qiang Ji. 2010. In the Eye of the Beholder: A Survey of Models for Eyes and Gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 3 (2010), 478–500. https://doi.org/10.1109/TPAMI.2009.30

    2010 doi

  17. [24]

    Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. 2024. OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning. arXiv preprint arXiv:2406.08858 (2024). A3FR: Agile 3D Gaussian ...

    2024 arXiv

  18. [25]

    Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Dret- takis, and Gabriel Brostow. 2018. Deep blending for free-viewpoint image- based rendering. ACM Trans. Graph. 37, 6, Article 257 (Dec. 2018), 15 pages. https://doi.org/10.1145/3272127.3275084

    2018

  19. [26]

    Meta Platform Inc. 2022. Gaze-tracked Foveated Rendering in Meta Quest Pro. https://developers.meta.com/horizon/blog/save-gpu-with-eye-tracked- foveated-rendering/

    2022

  20. [27]

    Ye-Joon Jo, Jun-Seok Choi, Jin Kim, Hyo-Joon Kim, and Seong-Yong Moon. 2021. Virtual reality (VR) simulation and augmented reality (AR) navigation in orthog- nathic surgery: a case report. Applied Sciences 11, 12 (2021), 5673

    2021

  21. [29]

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis

  22. [30]

    Joohwan Kim, Michael Stengel, Alexander Majercik, Shalini De Mello, David Dunn, Samuli Laine, Morgan McGuire, and David Luebke. 2019. NVGaze: An Anatomically-Informed Dataset for Low-Latency, Near-Eye Gaze Estimation. In Proceedings of the 2019 CHI Conference on Human Factors ...

    2019

  23. [31]

    Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. 2017. Tanks and temples: benchmarking large-scale scene reconstruction. ACM Trans. Graph. 36, 4, Article 78 (July 2017), 13 pages. https://doi.org/10.1145/3072959.3073599

    2017

  24. [32]

    Zhenglun Kong, Peiyan Dong, Xiaolong Ma, Xin Meng, Mengshu Sun, Wei Niu, Xuan Shen, Geng Yuan, Bin Ren, Minghai Qin, Hao Tang, and Yanzhi Wang

  25. [33]

    ACM TOG 42, 4 (7 2023)

    3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM TOG 42, 4 (7 2023)

    2023

  26. [34]

    Bernard C Kress. 2020. Optical architectures for augmented-, virtual-, and mixed- reality headsets. (No Title) (2020)

    2020

  27. [35]

    Yuna Kwak, Eric Penner, Xuan Wang, Mohammad R Saeedpour-Parizi, Olivier Mercier, Xiuyun Wu, Scott Murdison, and Phillip Guan. 2024. Saccade-Contingent Rendering. In ACM SIGGRAPH 2024 Conference Papers . 1–9

    2024

  28. [36]

    Philippe Lancheres and Mohamed Hafed. 2019. The MIPI C-PHY standard: A generalized multiconductor signaling scheme. IEEE Solid-State Circuits Magazine 11, 2 (2019), 69–77

    2019

  29. [37]

    arXiv:2112.13890 [cs.CV] https://arxiv.org/abs/2112.13890

    SPViT: Enabling Faster Vision Transformers via Soft Token Pruning. arXiv:2112.13890 [cs.CV] https://arxiv.org/abs/2112.13890

    arXiv

  30. [38]

    Eileen Kowler. 2011. Eye movements: The past 25 years. Vision research 51, 13 (2011), 1457–1483

    2011

  31. [39]

    Bin Li, Hong Fu, Desheng Wen, and WaiLun LO. 2018. Etracker: A Mobile Gaze- Tracking System with Near-Eye Display Based on a Combined Gaze-Tracking Algorithm. Sensors 18, 5 (2018). https://doi.org/10.3390/s18051626

    2018 doi

  32. [41]

    Chiao Liu, Lyle Bainbridge, Andrew Berkovich, Song Chen, Wei Gao, Tsung- Hsun Tsai, Kazuya Mori, Rimon Ikeno, Masayuki Uno, Toshiyuki Isozaki, et al

  33. [42]

    Steven M LaValle, Anna Yershova, Max Katsev, and Michael Antonov. 2014. Head tracking for the Oculus Rift. In 2014 IEEE international conference on robotics and automation (ICRA). IEEE, 187–194

    2014

  34. [43]

    Pil-Ho Lee and Young-Chan Jang. 2019. A 6.84 Gbps/lane MIPI C-PHY transceiver bridge chip with level-dependent equalization. IEEE Transactions on Circuits and Systems II: Express Briefs 67, 11 (2019), 2672–2676

    2019

  35. [44]

    Frank Löffler, Joshua Faber, Eloisa Bentivegna, Tanja Bode, Peter Diener, Roland Haas, Ian Hinder, Bruno C Mundim, Christian D Ott, Erik Schnetter, et al. 2012. The Einstein Toolkit: a community computational infrastructure for relativistic astrophysics. Classical and Quantum ...

    2012

  36. [45]

    Conny Lu, Praneeth Chakravarthula, Kaihao Liu, Xixiang Liu, Siyuan Li, and Henry Fuchs. 2022. Neural 3D Gaze: 3D Pupil Localization and Gaze Tracking based on Anatomical Eye Model and Neural Refraction Correction. In 2022 IEEE International Symposium on Mixed and Augmented Rea...

    2022

  37. [46]

    Feng Lu, Takahiro Okabe, Yusuke Sugano, and Yoichi Sato. 2011. A Head Pose- free Approach for Appearance-based Gaze Estimation. In British Machine Vision Conference. https://api.semanticscholar.org/CorpusID:7733236

    2011

  38. [47]

    Henna Mäkinen, Elina Haavisto, Sara Havola, and Jaana-Maija Koivisto. 2022. User experiences of virtual reality technologies for healthcare in learning: an integrative review. Behaviour & Information Technology 41, 1 (2022), 1–17

    2022

  39. [48]

    Wenxuan Liu, Budmonde Duinkharjav, Qi Sun, and Sai Qian Zhang. 2025. Foveal- net: Advancing ai-driven gaze tracking solutions for efficient foveated rendering in virtual reality. IEEE Transactions on Visualization and Computer Graphics (2025)

    2025

  40. [49]

    Wenxuan Liu, Budmonde Duinkharjav, Qi Sun, and Sai Qian Zhang. 2025. Foveal- Net: Advancing AI-Driven Gaze Tracking Solutions for Optimized Foveated Rendering System Performance in Virtual Reality. IEEE Transactions on Visual- ization and Computer Graphics (IEEE VR) (2025)

    2025

  41. [50]

    Pier Luigi Mazzeo, Dilan D’Amico, Paolo Spagnolo, and Cosimo Distante. 2021. Deep Learning based Eye gaze estimation and prediction. In2021 6th International Conference on Smart and Sustainable Technologies (SpliTech). 1–6. https://doi.org/ 10.23919/SpliTech52315.2021.9566413

    2021

  42. [51]

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. 2020. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV. 405–421

    2020

  43. [52]

    Komogortsev, and Sachin S

    Cristina Palmero, Abhishek Sharma, Karsten Behrendt, Kapil Krishnakumar, Oleg V. Komogortsev, and Sachin S. Talathi. 2020. OpenEDS2020: Open Eyes Dataset. arXiv:2005.03876 [cs.CV] https://arxiv.org/abs/2005.03876

    2020 arXiv

  44. [53]

    Junrui Pan and Timothy G Rogers. [n. d.]. CRISP: Concurrent Rendering and Compute Simulation Platform for GPUs. ([n. d.])

  45. [54]

    Rados Mantiuk, Bartosz Bazyluk, and Anna Tomaszewska. 2011. Gaze-Dependent depth-of-field effect rendering in virtual environments. In Proceedings of the Second International Conference on Serious Games Development and Applications . Springer-Verlag, Berlin, Heidelberg. https:...

    2011 doi

  46. [55]

    Ethel Matin. 1974. Saccadic suppression: a review and an analysis. Psychological bulletin 81, 12 (1974), 899

    1974

  47. [56]

    Meta Quest Pro. 2022. https://www.meta.com/quest/quest-pro/tech-specs/#tech- specs

    2022

  48. [57]

    DA Robinson. 1964. The mechanics of human saccadic eye movement. The Journal of physiology 174, 2 (1964), 245

    1964

  49. [58]

    Mohammadreza Saed, Yuan Hsi Chou, Lufei Liu, Tyler Nowicki, and Tor M. Aamodt. 2022. Vulkan-Sim: A GPU Architecture Simulator for Ray Tracing. In 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) . 263–281. https://doi.org/10.1109/MICRO56248.2022.00027

    2022

  50. [59]

    Rahul Singh, Muhammad Huzaifa, Jeffrey Liu, Anjul Patney, Hashim Sharif, Yifan Zhao, and Sarita Adve. 2023. Power, Performance, and Image Quality Tradeoffs in Foveated Rendering. In2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR). 205–214. https://doi.org/10.11...

    2023

  51. [60]

    Anjul Patney, Joohwan Kim, Marco Salvi, Anton Kaplanyan, Chris Wyman, Nir Benty, Aaron Lefohn, and David Luebke. 2016. Perceptually-based foveated virtual reality. In ACM SIGGRAPH 2016 Emerging Technologies (Anaheim, California) (SIGGRAPH ’16). Association for Computing Machin...

    2016

  52. [61]

    Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards foveated rendering for gaze-tracked virtual reality. ACM Trans. Graph. 35, 6 (2016). https://doi.org/10. 1145/2980179.2980246

    2016

  53. [62]

    Xiaoyu Sun, Xiaochen Peng, Sai Zhang, Jorge Gomez, Win-San Khwa, Syed Sarwar, Ziyun Li, Weidong Cao, Zhao Wang, Chiao Liu, et al. 2024. Estimating Power, Performance, and Area for On-Sensor Deployment of AR/VR Workloads Using an Analytical Framework. ACM Transactions on Design...

    2024

  54. [63]

    Lech Świrski and Neil A. Dodgson. 2013. A fully-automatic, temporal approach to single camera, glint-free 3D eye model fitting [Abstract]. In Proceedings of ECEM 2013 (Lund, Sweden). http://www.cl.cam.ac.uk/research/rainbow/projects/ eyemodelfit/

    2013

  55. [64]

    Khaled Takrouri, Edward Causton, and Benjamin Simpson. 2022. AR technologies in engineering education: Applications, potential, and limitations. Digital 2, 2 (2022), 171–190

    2022

  56. [65]

    Haiyu Wang, Wenxuan Liu, and Sai Qian Zhang. [n. d.]. Hardware and Algorithm Codesign for Efficient Gaze Tracking in Virtual Reality System. ([n. d.])

  57. [66]

    Rahul Singh, Muhammad Huzaifa, Jeffrey Liu, Anjul Patney, Hashim Sharif, Yifan Zhao, and Sarita Adve. 2023. Power, performance, and image quality tradeoffs in foveated rendering. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 205–214

    2023

  58. [67]

    Yusuke Sugano, Yasuyuki Matsushita, and Yoichi Sato. 2014. Learning-by- Synthesis for Appearance-Based 3D Gaze Estimation. In 2014 IEEE Conference on Computer Vision and Pattern Recognition . 1821–1828. https://doi.org/10.1109/ CVPR.2014.235

    2014

  59. [68]

    Erroll Wood, Tadas Baltrušaitis, Louis-Philippe Morency, Peter Robinson, and Andreas Bulling. 2016. A 3D Morphable Eye Region Model for Gaze Estimation. In Computer Vision – ECCV 2016 , Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publis...

    2016

  60. [69]

    Erroll Wood, Tadas Baltrušaitis, Louis-Philippe Morency, Peter Robinson, and Andreas Bulling. 2016. Learning an appearance-based gaze estimator from one million synthesised images. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications (ETRA...

    2016

  61. [70]

    Zhiwen Yan, Chen Li, and Gim Hee Lee. 2023. NeRF-DS: Neural Radiance Fields for Dynamic Specular Objects. In CVPR. 8285–8295

    2023

  62. [71]

    Jiannan Ye, Anqi Xie, Susmija Jabbireddy, Yunchuan Li, Xubo Yang, and Xiaoxu Meng. 2022. Rectangular Mapping-based Foveated Rendering. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) . 756–764. https://doi. org/10.1109/VR51125.2022.00097

    2022

  63. [72]

    Kang Wang and Qiang Ji. 2017. Real Time Eye Gaze Tracking with 3D Deformable Eye-Face Model. In 2017 IEEE International Conference on Computer Vision (ICCV) . 1003–1011. https://doi.org/10.1109/ICCV.2017.114

    2017 doi

  64. [73]

    Thomas Westin, José Neves, Peter Mozelius, Carla Sousa, and Lara Mantovan

  65. [74]

    In European Conference on Games Based Learning , Vol

    Inclusive AR-games for education of deaf children: Challenges and oppor- tunities. In European Conference on Games Based Learning , Vol. 16. 597–604

  66. [75]

    Baoheng Zhang, Yizhao Gao, Jingyuan Li, and Hayden Kwok-Hay So. 2024. Co- designing a Sub-millisecond Latency Event-based Eye Tracking System with Submanifold Sparse CNN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 5771–5779

    2024

  67. [76]

    Tongyu Zhang, Yiran Shen, Guangrong Zhao, Lin Wang, Xiaoming Chen, Lu Bai, and Yuanfeng Zhou. 2024. Swift-Eye: Towards Anti-blink Pupil Tracking for Precise and Robust High-Frequency Near-Eye Movement Analysis with Event Cameras. IEEE Transactions on Visualization and Computer...

    2024

  68. [77]

    Weiqun Zhang, Andrew Myers, Kevin Gott, Ann Almgren, and John Bell. 2021. AMReX: Block-structured adaptive mesh refinement for multiphysics applica- tions. The International Journal of High Performance Computing Applications 35, 6 (2021), 508–526

    2021

  69. [78]

    Xucong Zhang, Yusuke Sugano, and Andreas Bulling. 2019. Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems . ACM. https://doi.org/10.1145/3290605.3300646

    2019

  70. [79]

    Yaffa Yeshurun, Marisa Carrasco, and Laurence T Maloney. 2008. Bias and sensitivity in two-interval forced choice procedures: Tests of the difference model. Vision research 48, 17 (2008), 1837–1851

    2008

  71. [80]

    Flanagin, Peter zu Eulenburg, and Seyed-Ahmad Ahmadi

    Yuk-Hoi Yiu, Moustafa Aboulatta, Theresa Raiser, Leoni Ophey, Virginia L. Flanagin, Peter zu Eulenburg, and Seyed-Ahmad Ahmadi. 2019. DeepVOG: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning. Journal of Neuroscience Methods 324 (2019), 10...

    2019 doi

  72. [81]

    Haoran You, Yang Zhao, Cheng Wan, Zhongzhi Yu, Yonggan Fu, Jiayi Yuan, Shang Wu, Shunyao Zhang, Yongan Zhang, Chaojian Li, et al. 2023. EyeCoD: Eye Tracking System Acceleration via FlatCam-Based Algorithm and Hardware Co-Design. IEEE Micro 43, 4 (2023), 88–97

    2023

  73. [82]

    Yiwei Zhao, Ziyun Li, Win-San Khwa, Xiaoyu Sun, Sai Qian Zhang, Syed Shakib Sarwar, Kleber Hugo Stangherlin, Yi-Lun Lu, Jorge Tomas Gomez, Jae-Sun Seo, et al. 2024. Neural Architecture Search of Hybrid Models for NPU-CIM Hetero- geneous AR/VR Devices. arXiv preprint arXiv:2410...

    2024 arXiv

  74. [86]

    Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. 2015. Appearance-based gaze estimation in the wild. In 2015 IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR) . 4511–4520. https://doi.org/10.1109/ CVPR.2015.7299081

    2015

  75. [87]

    Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. 2017. MPI- IGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation. arXiv:1711.09017 [cs.CV] https://arxiv.org/abs/1711.09017

    2017 arXiv

  76. [88]

    Ziliang Zhang, Zexin Li, Hyoseung Kim, and Cong Liu. 2024. BOXR: Body and head motion Optimization framework for eXtended Reality. arXiv preprint arXiv:2410.13084 (2024)

    2024 arXiv

  77. [2019]

    Proceedings of the National Academy of Sciences 116, 6 (2019), 2027–2032

    Time course of spatiotopic updating across saccades. Proceedings of the National Academy of Sciences 116, 6 (2019), 2027–2032

    2019

  78. [2020]

    In 2020 IEEE International Electron Devices Meeting (IEDM)

    A 4.6 𝜇m, 512× 512, ultra-low power stacked digital pixel sensor with triple quantization and 127dB dynamic range. In 2020 IEEE International Electron Devices Meeting (IEDM). IEEE, 16–1

    2020

  79. [2023]

    ACM Trans

    3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph. 42, 4 (2023), 139–1

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.