Pith. sign in

REVIEW 4 major objections 6 minor 73 references

FovealNet: Advancing AI-Driven Gaze Tracking Solutions for Optimized Foveated Rendering System Performance in Virtual Reality

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read VR gaze-tracking long-tail error shrinks from 8.21 to 2.31 degrees.

desk verdict A genuinely new performance-aware gaze-tracking loss that credibly cuts long-tail error on OpenEDS2020; the headline 1.42x system speedup is not actually measured end-to-end and rests on the same latency model used for training. read the letter →

arxiv 2412.10456 v2 pith:4QCZVHYO submitted 2024-12-12 cs.CV cs.AI

Wenxuan Liu , Monde Duinkharjav , Qi Sun , Sai Qian Zhang This is my paper · ORCID
classification cs.CVcs.AI
keywords foveatedrenderinggazetrackingperformance-awaretrainingevent-basedcroppingtokenpruningvisiontransformervirtualrealitylatencyoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that gaze-tracking errors for foveated rendering should be optimized against the downstream task, not against average accuracy alone. It introduces FovealNet, a vision-transformer gaze tracker trained with a performance-aware loss that maps gaze error to expected rendering latency. On the OpenEDS2020 dataset, that loss cuts the 95th-percentile gaze error from 8.21° to 2.31° for the unpruned model, removing the long tail of large errors that previously forced foveated renderers to enlarge the high-resolution region. A reader should care because the long tail is what breaks foveated rendering: when the predicted gaze misses the true fovea, visual quality drops and rendering latency rises.

What carries the argument

The load-bearing mechanism is the performance-aware loss of Eq. (4): a soft-max approximation of the maximum per-batch gaze error, passed through a fitted piecewise-linear latency function $U(\cdot)$ that converts an angular tracking error into the rendering latency needed to keep the foveal region aligned. $U(\cdot)$ is constructed from rendering-latency measurements on a Meta Quest Pro (50 frames from one scene) and a mobile GPU (four scenes) at three resolutions. Because the same $U(\cdot)$ appears in the training loss and in the reported system-speedup numbers, the argument reduces to a single identity: if $U(\cdot)$ predicts real TFR latency from gaze error, then minimizing $U$ of the worst-case error directly lowers the component of total latency that gaze error controls. Around this core sit two efficiency mechanisms: an event-based cropping algorithm that locates the pupil as the largest connected dark component and crops a 450 × 200 region, removing over 64.8% of pixels, and a token-pruning pass that drops low-attention ViT tokens on the fly.

What would settle it

Measure the actual end-to-end latency of a TFR system with FovealNet running on a Meta Quest Pro or comparable headset, including real on-device gaze tracking plus foveated rendering, and compare the P90/P95-driven $T_{\text{tracking}} + T_{\text{fr}}$ against the paper's simulated figures; if real total latency is not at least 1.42x lower than the best baseline under the same scene and resolution conditions, the central speedup claim fails. A simpler check is whether the fitted $U(\cdot)$ curve, taken from the paper, predicts a measured rendering latency on a held-out scene within 10%.

Watch

Extended reading notes

Core claim

FovealNet's central claim is that a gaze-tracking DNN can be trained to directly minimize the total latency of a gaze-tracked foveated-rendering (TFR) system by replacing the usual mean gaze error with a loss that emphasizes the worst errors and weighs them by their rendering cost. The proposed loss, Eq. (4), is $U\left(\frac{1}{N}\ln\left(\sum_{d\in D_b} e^{N\|\theta_d - \theta_d^g\|_2}\right)\right)$, where $U(\cdot)$ is a piecewise-linear function fitted to measured rendering latency versus foveal eccentricity, and the soft-max inside approximates the maximum error in a batch. Training with this loss reduces the 95th-percentile error from 8.21° to 2.31° for the full FovealNet (0.0) and the maximum error from 24.2° to 5.22° on OpenEDS2020. The paper also claims that this training objective, combined with event-based pupil-centered cropping and token-wise pruning of the ViT, yields a per-frame latency $T_{\text{tracking}} + T_{\text{fr}}$ of about 10.2 ms in the P90 scenario, at least 1.42x faster than prior gaze trackers, and a foveated-rendering perceptual JND of 0.07, which the paper reads as imperceptible degradation.

Load-bearing premise

The paper assumes that the piecewise-linear function $U(\cdot)$, fitted to rendering-latency measurements on a Meta Quest Pro (50 frames from one scene) and a mobile GPU (four scenes), accurately predicts real end-to-end TFR latency, and it uses that same model both to define the training loss and to compute the reported speedups; tracking latency on the Quest Pro is simulated, not measured on the device.

Editorial extensions

If this is right

  • Training a gaze tracker with a performance-aware loss directly shrinks the worst-case errors that force foveated renderers to enlarge the high-resolution region, so the reported gains are latency gains, not just accuracy gains.
  • The unpruned FovealNet cuts 95th-percentile gaze error from 8.21° to 2.31° and maximum error from 24.2° to 5.22°, which the paper ties to a foveated-image JND of 0.07.
  • Event-based cropping removes over 64.8% of input pixels and token-wise pruning keeps FovealNet in the 2.08 to 2.80 billion FLOP range, making the ViT cheap enough for real-time headset use.
  • The multi-resolution framework yields one ViT with six subnetworks of depths 3 through 8; under 720P rendering the depth-3 subnetwork achieves $T_{\text{tracking}} + T_{\text{fr}}$ of 6.19 ms, while under 1440P rendering the depth-6 subnetwork achieves 16.4 ms, demonstrating runtime depth selection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because $U(\cdot)$ is just a mapping from error to downstream cost, the same training principle should transfer to other perception-to-action loops, such as autofocus, gaze-contingent depth of field, or adaptive bitrate streaming, whenever a measurable cost curve exists.
  • The paper's speedup numbers inherit $U(\cdot)$'s accuracy: if $U(\cdot)$ were refitted with real on-device tracking latency and more rendering scenes, the relative standing of FovealNet versus baselines like Seg or NVGaze could change, since those baselines were not retrained under the performance-aware objective.
  • A testable extension is to train FovealNet with separate cost functions for power instead of latency, since the paper says the framework can be extended to other system performance metrics.
  • The multi-resolution selection rule could be made fully automatic by predicting $T_{\text{fr}}$ from current scene complexity and resolution, rather than choosing subnet depth by render resolution alone.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes FovealNet, a ViT-based gaze tracker for foveated rendering in VR, with three main contributions: an event-based cropping method that removes background pixels, a token-pruning strategy for the ViT, and a "performance-aware" training loss (Eq. (4)) that applies a fitted piecewise-linear rendering-latency function U(.) to a soft-max approximation of the maximum gaze error. The authors evaluate tracking accuracy on OpenEDS2020, report reductions in the 95th-percentile gaze error (e.g., for FovealNet 0.0, from 8.21 to 2.31 degrees), a foveated perceptual-quality improvement in JND, and system-level latency figures on a Meta Quest Pro (simulated with GPGPU-sim) and a mobile GPU, claiming at least a 1.42x speedup over prior methods.

Significance. If the reported accuracy numbers hold, the work provides a potentially useful observation: training a gaze tracker with a soft-max maximum-error loss substantially reduces the long tail of gaze errors, which is directly relevant to foveated rendering quality. The cropping and token-pruning techniques are sensible and appear to reduce FLOPs, and the OpenEDS2020 evaluation is a standard benchmark. However, the central system-performance contribution is not established: the latency model U(.) used to define the training loss is also used to compute the reported speedups, and the loss is mathematically equivalent to a monotone transform of the soft-max error if U(.) is monotonic, which weakens the claim that the training is genuinely "system performance-aware." The paper also omits code/weights and at least one key definition (the cropping parameters), limiting reproducibility.

major comments (4)
  1. [Sec. 4.3, Eq. (4)] The performance-aware loss in Eq. (4) applies the piecewise-linear function U(.) to the soft-max approximation of the maximum gaze error. Since rendering latency U(.) is monotonically increasing in the eccentricity angle, as shown in Fig. 6(c), minimizing U(softmax_error) is equivalent to minimizing the soft-max error itself; the specific shape of U(.) cannot affect the training optimum. Thus the claim that the loss "directly optimizes TFR system latency" is not supported, and the improvements in Fig. 10 should be attributed to the maximum-error objective of Eq. (3) rather than to the fitted latency model. To substantiate the system-awareness claim, the authors should either use a non-monotonic U(.) or demonstrate that different fitted latency curves yield different optimal models.
  2. [Sec. 6.1] The reported T_fr values and the "at least 1.42x speedup" claim in the abstract are computed from the same piecewise-linear function U(.) that defines the training loss in Eq. (4), and the Quest Pro T_tracking numbers are obtained from GPGPU-sim rather than on-device measurements. Because FovealNet is explicitly trained to minimize U(.), the resulting T_fr reduction is at least partly an in-sample outcome of the loss, and the paper provides no independent validation of U(.) against end-to-end TFR latency across scenes, resolutions, or system load. The speedup claim should be backed by direct end-to-end measurement on the target devices, or explicitly re-labeled as an estimate under the fitted latency model with a sensitivity analysis.
  3. [Sec. 5.1] The cropping parameters beta1=0.2 and beta2=500 are introduced with a reference to "??", but the formula or section that defines them is missing. Since event-based cropping is one of the three main contributions, the omitted definition prevents readers from reproducing the reported results.
  4. [Sec. 6.2, Tab. 2] The multi-resolution evaluation in Sec. 6.2 selects subnetworks that minimize T_tracking + T_fr without applying any visual-quality constraint. For example, at 720p the optimal depth-3 subnetwork has a P95 gaze error of 7.35 degrees and a maximum error of 15.70 degrees, which by the paper's own analysis in Sec. 3.2 (Fig. 5(b)) would produce a noticeable increase in JND and degrade foveated output. The claim that the system selects the "optimal" configuration is therefore incomplete unless a perceptual-quality constraint is imposed or the JND is reported for each selected configuration.
minor comments (6)
  1. [Table 1] The column header "P0" should be "P90"; the text also states that P0 represents the 90th percentile, which is inconsistent.
  2. [Sec. 5.2] The FLOPs comparison percentages are inconsistent: FovealNet (0.2) at 2.08B versus ResNet-based at 3.6B is approximately 42% lower, not 70% lower; versus Seg at 2.6B it is approximately 20% lower, not 27% lower.
  3. [Abstract] The phrase "13% increase in perceptual quality" is misleading; Sec. 5.4 reports a 13% increase in discriminability for the ResNet baseline relative to FovealNet, which is a decrease in quality. The abstract should state a reduction in JND.
  4. [Fig. 6(a) caption and Table 1] The Fig. 6(a) caption says NVGaze results were excluded due to high tracking error and inconsistent performance, but Table 1 reports NVGaze results; the text should be reconciled.
  5. [Sec. 4.3] The text refers to "T_tf", which appears to be a typo for "T_fr" (foveated rendering latency).
  6. [Sec. 6.1] The claim that Tsensing and Tcomm are small cites Fig. 2, but Fig. 2 is a system configuration diagram; the latency breakdown appears in Fig. 4(b). Please correct the reference.

Circularity Check

1 steps flagged · score 6.0 of 10

System-latency evaluation reuses the same fitted latency function U(.) that defines the training loss, so the reported speedup is partly a re-statement of the training objective; the gaze-accuracy claim is independent.

  1. fitted input called prediction [Sec. 4.3 Eq. (4) and Sec. 6.1 (Fig. 11)]
    "we profile rendering latencies across different VR devices and develop a piecewise linear function U(.) that links the gaze tracking error to the corresponding rendering latency ... Thus, the training objective becomes: ... The Tf r values are derived by determining latency based on the latency analysis outlined in Sec. 3.3 based on gaze tracking error. ... For FovealNet, we train it using the performance-aware loss function described in Eq. (4), where U(.) represents the processing latency under different eccentricity angle θ f on either the HMD or mobile GPU."

    The same fitted piecewise-linear U(.) is both the training loss in Eq. (4) and the source of the T_fr numbers in Sec. 6.1. The paper derives T_fr by evaluating U at θ_i + P90/P95 gaze error for each model, so the reported latency reductions and the 1.42x speedup are values of the exact regression the model was trained to minimize, not independent end-to-end TFR measurements. Since U was fit on only 50 DCS frames (Quest Pro) and four scenes (mobile GPU) and never validated against a real TFR pipeline, the system-level claim is partly in-sample with respect to U. The held-out OpenEDS2020 gaze-error reduction is independent and not circular; the circularity is confined to the system-latency/speedup claim.

full rationale

The core tracking-accuracy result (P95 error reduction from 8.21 to 2.31 degrees on OpenEDS2020, and the JND 0.07 perceptual result via FovVideoVDP) is evaluated on held-out data with external metrics and is not circular. The circularity is in the system-level speedup claim: Sec. 4.3 defines the loss as U applied to a soft-max approximation of the maximum gaze error, and Sec. 6.1 computes T_fr by applying the same U to each model's P90/P95 error; Fig. 11 is thus a re-evaluation of the training objective rather than a measured TFR latency. T_tracking is simulated with GPGPU-sim rather than measured on the device, which further removes the 1.42x claim from hardware validation, but that is a correctness/validation concern, not circularity. Separately, Sec. 5.1 contains a dangling '??' in place of the reference for the cropping parameters β1 and β2, an incompleteness that does not affect the circularity verdict. No load-bearing self-citation or uniqueness-imported-from-authors pattern was found. Overall: one partial circularity in the headline system-latency claim, with independent gaze-accuracy content, so score 6.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central mechanism depends heavily on a small set of hand-chosen numbers and modeling assumptions. Most load-bearing are the soft-max temperature N and the fitted latency model U(.), which appears in both the training loss and the evaluation of the claimed speedup. The remaining axioms are domain assumptions about how TFR systems enlarge the foveal region in response to gaze error and about the representativeness of OpenEDS2020 and the profiling scenes.

free parameters (3)
  • N (soft-max temperature) = 100
    Chosen empirically; Table 4 shows N=10 gives poor P95 error (6.92 degrees), while N=150 and N=200 cause training overflow. The loss in Eq. (3)-(4) depends critically on this value.
  • Cropping parameters beta1, beta2 = 0.2 and 500
    Mentioned in Sec. 5.1 with a broken reference ('??'); the exact roles in the region cropping algorithm are not defined in the visible text, so they are effectively hand-set constants.
  • Piecewise-linear latency model U(.) = Device-specific fitted curve for Quest Pro and mobile GPU
    Fitted to rendering-latency measurements over a small set of scenes and frames (Sec. 3.3) and used both as the training loss in Eq. (4) and as the evaluation oracle for the speedup claims in Sec. 6.1.
assumptions (5)
  • domain assumption The foveal region radius is enlarged by exactly the gaze tracking error: r_f = rho*d*tan(theta_i + delta_theta), so a tracking error delta_theta directly inflates the high-resolution rendering region and thus latency.
    Used to motivate the loss and to compute T_fr from gaze error distributions in Sec. 3.3 and Sec. 6.1. If a renderer used a different compensation policy, the relationship between error distribution and latency would differ.
  • domain assumption The rendering latency measurements used to fit U(.) are representative of real TFR systems across scenes, resolutions, and devices.
    U(.) is the backbone of both the training objective and the reported system speedups; it is fitted on 50 DCS frames for the Quest Pro and four scenes for the mobile GPU (Sec. 3.3).
  • standard math The soft-max approximation max(x1,x2) approximately (1/N) ln(e^{N*x1}+e^{N*x2}) with finite tuned N is a valid surrogate for the maximum error.
    Standard analytic approximation, but its quality depends on N, which is a fitted parameter with a narrow usable range (Sec. 5.6, Table 4).
  • domain assumption Attention scores in the ViT indicate task-relevant tokens and can be pruned without harming gaze accuracy.
    The token-pruning method (Sec. 4.2) relies on this; it is empirically validated in Table 1 but has no formal guarantee.
  • domain assumption The OpenEDS2020 dataset (32 training and 8 validation participants) is representative of the VR eye-tracking population for the claimed system benefits.
    All accuracy results come from OpenEDS2020; no cross-dataset or on-device evaluation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FovealNet: Advancing AI-Driven Gaze Tracking Solutions for Optimized Foveated Rendering System Performance in Virtual Reality." pith.science (2026). https://pith.science/paper/4QCZVHYO

@misc{pith2026241210456,
  author       = {Pith},
  title        = {Pith review of: FovealNet: Advancing AI-Driven Gaze Tracking Solutions for Optimized Foveated Rendering System Performance in Virtual Reality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4QCZVHYO}},
  note         = {Machine review of arXiv:2412.10456}
}
abstract

Leveraging real-time eye-tracking, foveated rendering optimizes hardware efficiency and enhances visual quality virtual reality (VR). This approach leverages eye-tracking techniques to determine where the user is looking, allowing the system to render high-resolution graphics only in the foveal region-the small area of the retina where visual acuity is highest, while the peripheral view is rendered at lower resolution. However, modern deep learning-based gaze-tracking solutions often exhibit a long-tail distribution of tracking errors, which can degrade user experience and reduce the benefits of foveated rendering by causing misalignment and decreased visual quality. This paper introduces \textit{FovealNet}, an advanced AI-driven gaze tracking framework designed to optimize system performance by strategically enhancing gaze tracking accuracy. To further reduce the implementation cost of the gaze tracking algorithm, FovealNet employs an event-based cropping method that eliminates over $64.8\%$ of irrelevant pixels from the input image. Additionally, it incorporates a simple yet effective token-pruning strategy that dynamically removes tokens on the fly without compromising tracking accuracy. Finally, to support different runtime rendering configurations, we propose a system performance-aware multi-resolution training strategy, allowing the gaze tracking DNN to adapt and optimize overall system performance more effectively. Evaluation results demonstrate that FovealNet achieves at least $1.42\times$ speed up compared to previous methods and 13\% increase in perceptual quality for foveated output.

Figures

Figures reproduced from arXiv: 2412.10456 by the authors.

Figure 1
Figure 1. FovealNet for efficient gaze-tracked foveated rendering system operation. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) TFR system configuration. (b) Foveated rendering in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 4
Figure 4. (a) TFR system configuration. (b) Latency breakdown (nor [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figures from the paper (7 more)
Figure 6
Figure 6. Figure 6: (a) Predicted gaze error distributions on the OpenEDS2020 dataset, showing mean, 5th, 95th percentiles, min, and max angular [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Overall architecture of FovealNet. Eye Images CC Maps Pupil Maps [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 8
Figure 8. Figure 8: Examples of pupil-centered cropping with intermediate re [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]
Figure 9
Figure 9. Figure 9: (a) Multi-resolution training framework. (b) An example of [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]
Figure 10
Figure 10. Figure 10: (a) Distribution of gaze tracking error of FovealNet trained [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: Evaluation on overall processing latency. (a) Measurement on Quest Pro at a resolution of 1080P (1080 [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: The stacked bars show the distribution of the gaze errors [PITH_FULL_IMAGE:figures/full_fig_p009_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 70 canonical work pages

  1. [1]

    Perceptually- based foveated virtual reality

    Anjul Patney, Joohwan Kim, Marco Salvi, Anton Kaplanyan, Chris Wyman, Nir Benty, Aaron Lefohn, and David Luebke. Perceptually- based foveated virtual reality. In ACM SIGGRAPH 2016 Emerging Technologies, SIGGRAPH ’16, New York, NY , USA, 2016. Associa- tion for Computing Machinery. 1

    work page 2016

  2. [2]

    La- tency requirements for foveated rendering in virtual reality

    Rachel Albert, Anjul Patney, David Luebke, and Joohwan Kim. La- tency requirements for foveated rendering in virtual reality. ACM Transactions on Applied Perception (TAP) , 14(4):1–13, 2017. 1, 2, 3, 4

    work page 2017

  3. [3]

    Time-warped foveated rendering for virtual reality head- sets

    Linus Franke, Laura Fink, Jana Martschinke, Kai Selgrad, and Marc Stamminger. Time-warped foveated rendering for virtual reality head- sets. In Computer Graphics Forum, volume 40, pages 110–123. Wiley Online Library, 2021. 1

    work page 2021

  4. [4]

    Deepfovea: Neural recon- struction for foveated rendering and video compression using learned statistics of natural videos

    Anton S Kaplanyan, Anton Sochenov, Thomas Leimk ¨uhler, Mikhail Okunev, Todd Goodall, and Gizem Rufo. Deepfovea: Neural recon- struction for foveated rendering and video compression using learned statistics of natural videos. ACM Transactions on Graphics (TOG) , 38(6):1–13, 2019. 1

    work page 2019

  5. [5]

    On the interplay of foveated rendering and video encod- ing

    Gazi Karam Illahi, Matti Siekkinen, Teemu K ¨am¨ar¨ainen, and Antti Yl¨a-J¨a¨aski. On the interplay of foveated rendering and video encod- ing. In Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology, pages 1–3, 2020. 1

    work page 2020

  6. [6]

    Foveated streaming of real-time graphics

    Gazi Karam Illahi, Matti Siekkinen, Teemu K ¨am¨ar¨ainen, and Antti Yl¨a-J¨a¨aski. Foveated streaming of real-time graphics. In Proceed- ings of the 12th ACM Multimedia Systems Conference , pages 214– 226, 2021. 1

    work page 2021

  7. [7]

    Content-aware video encoding for cloud gaming

    Mohamed Hegazy, Khaled Diab, Mehdi Saeedi, Boris Ivanovic, Ihab Amer, Yang Liu, Gabor Sines, and Mohamed Hefeeda. Content-aware video encoding for cloud gaming. In Proceedings of the 10th ACM multimedia systems conference, pages 60–73, 2019. 1

    work page 2019

  8. [8]

    Cloud gaming with foveated video encoding

    Gazi Karam Illahi, Thomas Van Gemert, Matti Siekkinen, Enrico Masala, Antti Oulasvirta, and Antti Yl ¨a-J¨a¨aski. Cloud gaming with foveated video encoding. ACM Transactions on Multimedia Comput- ing, Communications, and Applications (TOMM), 16:1–24, 2020. 1

    work page 2020

Show all 73 references
  1. [9]

    Enhancing quality of experience for cloud virtual reality gaming: An object-aware video encoding

    Wenjie Zou, Shixuan Feng, Xionghui Mao, Fuzheng Yang, and Zhibin Ma. Enhancing quality of experience for cloud virtual reality gaming: An object-aware video encoding. In 2021 IEEE International Confer- ence on Multimedia & Expo Workshops (ICMEW), pages 1–6. IEEE,

    2021

  2. [10]

    Flanagin, Peter zu Eulenburg, and Seyed-Ahmad Ahmadi

    Yuk-Hoi Yiu, Moustafa Aboulatta, Theresa Raiser, Leoni Ophey, Vir- ginia L. Flanagin, Peter zu Eulenburg, and Seyed-Ahmad Ahmadi. Deepvog: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning. Journal of Neuroscience Methods, 324:108307, 2019....

    2019

  3. [11]

    Real-time gaze tracking with event-driven eye segmenta- tion

    Yu Feng, Nathan Goulding-Hotta, Asif Khan, Hans Reyserhove, and Yuhao Zhu. Real-time gaze tracking with event-driven eye segmenta- tion. In 2022 IEEE Conference on Virtual Reality and 3D User Inter- faces (VR), pages 399–408, 2022. 1, 2, 3, 4, 6, 7

    2022

  4. [12]

    Etracker: A mobile gaze-tracking system with near-eye display based on a combined gaze- tracking algorithm

    Bin Li, Hong Fu, Desheng Wen, and WaiLun LO. Etracker: A mobile gaze-tracking system with near-eye display based on a combined gaze- tracking algorithm. Sensors, 18(5), 2018. 1, 2

    2018

  5. [13]

    Angelopoulos, Julien N.P

    Anastasios N. Angelopoulos, Julien N.P. Martel, Amit P. Kohli, J ¨org Conradt, and Gordon Wetzstein. Event-based near-eye gaze tracking beyond 10,000 hz. IEEE Transactions on Visualization and Computer Graphics, 27(5):2577–2586, 2021. 1

    2021

  6. [14]

    Bailey, Jeff B

    Rakshit Kothari, Aayush Kumar Chaudhary, Reynold J. Bailey, Jeff B. Pelz, and Gabriel J. Diaz. Ellseg: An ellipse segmentation frame- work for robust gaze tracking. IEEE Transactions on Visualization and Computer Graphics, 27:2757–2767, 2020. 1

    2020

  7. [15]

    In the eye of the beholder: A survey of models for eyes and gaze

    Dan Witzner Hansen and Qiang Ji. In the eye of the beholder: A survey of models for eyes and gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3):478–500, 2010. 2

    2010

  8. [16]

    Evaluation of appearance-based methods and implications for gaze-based applica- tions

    Xucong Zhang, Yusuke Sugano, and Andreas Bulling. Evaluation of appearance-based methods and implications for gaze-based applica- tions. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, May 2019. 2

    2019

  9. [17]

    Lech ´Swirski and Neil A. Dodgson. A fully-automatic, temporal ap- proach to single camera, glint-free 3d eye model fitting [abstract]. In 9 © 2025 IEEE. This is the author’s version of the article that has been published in the proceedings of IEEE Visualization conference. Th...

    2025

  10. [18]

    Real time eye gaze tracking with 3d de- formable eye-face model

    Kang Wang and Qiang Ji. Real time eye gaze tracking with 3d de- formable eye-face model. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 1003–1011, 2017. 2, 3

    2017

  11. [19]

    A 3d morphable eye region model for gaze estimation

    Erroll Wood, Tadas Baltru ˇsaitis, Louis-Philippe Morency, Peter Robinson, and Andreas Bulling. A 3d morphable eye region model for gaze estimation. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016 , pages 297– 313, Cham, 2016. Spr...

    2016

  12. [20]

    Neural 3d gaze: 3d pupil localization and gaze tracking based on anatomical eye model and neural refraction cor- rection

    Conny Lu, Praneeth Chakravarthula, Kaihao Liu, Xixiang Liu, Siyuan Li, and Henry Fuchs. Neural 3d gaze: 3d pupil localization and gaze tracking based on anatomical eye model and neural refraction cor- rection. In 2022 IEEE International Symposium on Mixed and Aug- mented Reali...

    2022

  13. [21]

    Tongyu Zhang, Yiran Shen, Guangrong Zhao, Lin Wang, Xiaoming Chen, Lu Bai, and Yuanfeng Zhou. Swift-eye: Towards anti-blink pupil tracking for precise and robust high-frequency near-eye move- ment analysis with event cameras.IEEE Transactions on Visualization and Computer Grap...

    2024

  14. [22]

    Chaudhary, Rakshit Kothari, Manoj Acharya, Shusil Dangi, Nitinraj Nair, Reynold Bailey, Christopher Kanan, Gabriel Diaz, and Jeff B

    Aayush K. Chaudhary, Rakshit Kothari, Manoj Acharya, Shusil Dangi, Nitinraj Nair, Reynold Bailey, Christopher Kanan, Gabriel Diaz, and Jeff B. Pelz. Ritnet: Real-time semantic segmentation of the eye for gaze tracking. In 2019 IEEE/CVF International Confer- ence on Computer Vi...

    2019

  15. [23]

    Appearance-based gaze estimation in the wild

    Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. Appearance-based gaze estimation in the wild. In 2015 IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR) , pages 4511–4520, 2015. 2, 6

    2015

  16. [24]

    Mpiigaze: Real-world dataset and deep appearance-based gaze esti- mation, 2017

    Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. Mpiigaze: Real-world dataset and deep appearance-based gaze esti- mation, 2017. 2

    2017

  17. [25]

    A head pose-free approach for appearance-based gaze estimation

    Feng Lu, Takahiro Okabe, Yusuke Sugano, and Yoichi Sato. A head pose-free approach for appearance-based gaze estimation. In British Machine Vision Conference, 2011. 2

    2011

  18. [26]

    Learning- by-synthesis for appearance-based 3d gaze estimation

    Yusuke Sugano, Yasuyuki Matsushita, and Yoichi Sato. Learning- by-synthesis for appearance-based 3d gaze estimation. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 1821–1828, 2014. 2

    2014

  19. [27]

    Learning an appearance-based gaze estimator from one million synthesised images

    Erroll Wood, Tadas Baltru ˇsaitis, Louis-Philippe Morency, Peter Robinson, and Andreas Bulling. Learning an appearance-based gaze estimator from one million synthesised images. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Appli- cations, ETRA ’...

    2016

  20. [28]

    One eye is all you need: Lightweight ensembles for gaze estimation with single encoders, 2022

    Rishi Athavale, Lakshmi Sritan Motati, and Rohan Kalahasty. One eye is all you need: Lightweight ensembles for gaze estimation with single encoders, 2022. 2, 3, 6, 7

    2022

  21. [29]

    Deep learning based eye gaze estimation and prediction

    Pier Luigi Mazzeo, Dilan D’Amico, Paolo Spagnolo, and Cosimo Dis- tante. Deep learning based eye gaze estimation and prediction. In2021 6th International Conference on Smart and Sustainable Technologies (SpliTech), pages 1–6, 2021. 2, 3, 4, 6, 7

    2021

  22. [30]

    An image is worth 16x16 words: Transform- ers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transform- ers for image recognition at scale. arXiv preprint arXiv:2010.11929,

    2010 arXiv

  23. [31]

    Nvgaze: An anatomically-informed dataset for low-latency, near-eye gaze estimation

    Joohwan Kim, Michael Stengel, Alexander Majercik, Shalini De Mello, David Dunn, Samuli Laine, Morgan McGuire, and David Luebke. Nvgaze: An anatomically-informed dataset for low-latency, near-eye gaze estimation. In Proceedings of the 2019 CHI Conference on Human Factors in Com...

    2019

  24. [32]

    Towards foveated rendering for gaze-tracked virtual reality

    Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. Towards foveated rendering for gaze-tracked virtual reality. ACM Trans. Graph., 35(6), 2016. 2, 4

    2016

  25. [33]

    Duchowski, Donald H

    Andrew T. Duchowski, Donald H. House, Jordan Gestring, Rui I. Wang, Krzysztof Krejtz, Izabela Krejtz, Radosław Mantiuk, and Bar- tosz Bazyluk. Reducing visual discomfort of 3d stereoscopic displays with gaze-contingent depth-of-field. In Proceedings of the ACM Sym- posium on A...

    2014

  26. [34]

    Gaze- dependent depth-of-field effect rendering in virtual environments

    Rados Mantiuk, Bartosz Bazyluk, and Anna Tomaszewska. Gaze- dependent depth-of-field effect rendering in virtual environments. In Proceedings of the Second International Conference on Serious Games Development and Applications , Berlin, Heidelberg, 2011. Springer-Verlag. 2

    2011

  27. [35]

    Rectangular mapping-based foveated rendering

    Jiannan Ye, Anqi Xie, Susmija Jabbireddy, Yunchuan Li, Xubo Yang, and Xiaoxu Meng. Rectangular mapping-based foveated rendering. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 756–764, 2022. 2

    2022

  28. [36]

    Weier, M

    M. Weier, M. Stengel, T. Roth, P. Didyk, E. Eisemann, M. Eisemann, S. Grogorick, A. Hinkenjann, E. Kruijff, M. Magnor, K. Myszkowski, and P. Slusallek. Perception-driven accelerated rendering. Computer Graphics Forum, 36(2):611–643, 2017. 3

    2017

  29. [37]

    Turing variable rate shading in vrworks

    NVIDIA Corporation. Turing variable rate shading in vrworks. https://developer.nvidia.com/blog/ turing-variable-rate-shading-vrworks/ , 2018. 3, 4

    2018

  30. [38]

    An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weis- senborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  31. [39]

    Exploring efficiency of vision transformers for self-supervised monocular depth estimation

    Aleksei Karpov and Ilya Makarov. Exploring efficiency of vision transformers for self-supervised monocular depth estimation. In 2022 IEEE International Symposium on Mixed and Augmented Reality (IS- MAR), pages 711–719, 2022. 3

    2022

  32. [40]

    Spvit: Enabling faster vision transformers via soft token pruning, 2022

    Zhenglun Kong, Peiyan Dong, Xiaolong Ma, Xin Meng, Mengshu Sun, Wei Niu, Xuan Shen, Geng Yuan, Bin Ren, Minghai Qin, Hao Tang, and Yanzhi Wang. Spvit: Enabling faster vision transformers via soft token pruning, 2022. 3

    2022

  33. [41]

    Chasing sparsity in vision transformers: An end- to-end exploration, 2021

    Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, and Zhangyang Wang. Chasing sparsity in vision transformers: An end- to-end exploration, 2021. 3

    2021

  34. [42]

    Evo-vit: Slow-fast token evolution for dynamic vision transformer, 2021

    Yifan Xu, Zhijie Zhang, Mengdan Zhang, Kekai Sheng, Ke Li, Weim- ing Dong, Liqing Zhang, Changsheng Xu, and Xing Sun. Evo-vit: Slow-fast token evolution for dynamic vision transformer, 2021. 3

    2021

  35. [43]

    The mipi c-phy standard: A generalized multiconductor signaling scheme

    Philippe Lancheres and Mohamed Hafed. The mipi c-phy standard: A generalized multiconductor signaling scheme. IEEE Solid-State Cir- cuits Magazine, 11(2):69–77, 2019. 3

    2019

  36. [44]

    Eyecod: Eye tracking system acceleration via flatcam-based algorithm and hardware co-design

    Haoran You, Yang Zhao, Cheng Wan, Zhongzhi Yu, Yonggan Fu, Jiayi Yuan, Shang Wu, Shunyao Zhang, Yongan Zhang, Chaojian Li, et al. Eyecod: Eye tracking system acceleration via flatcam-based algorithm and hardware co-design. IEEE Micro, 43(4):88–97, 2023. 3

    2023

  37. [45]

    A 4.6µm, 512× 512, ultra-low power stacked digital pixel sensor with triple quantization and 127db dynamic range

    Chiao Liu, Lyle Bainbridge, Andrew Berkovich, Song Chen, Wei Gao, Tsung-Hsun Tsai, Kazuya Mori, Rimon Ikeno, Masayuki Uno, Toshiyuki Isozaki, et al. A 4.6µm, 512× 512, ultra-low power stacked digital pixel sensor with triple quantization and 127db dynamic range. In 2020 IEEE I...

    2020

  38. [46]

    Event based, near eye gaze tracking beyond 10,000 hz

    Anastasios N Angelopoulos, Julien NP Martel, Amit PS Kohli, Jorg Conradt, and Gordon Wetzstein. Event based, near eye gaze tracking beyond 10,000 hz. arXiv preprint arXiv:2004.03577, 2020. 3

    2004 arXiv

  39. [47]

    Estimating power, performance, and area for on-sensor deploy- ment of ar/vr workloads using an analytical framework

    Xiaoyu Sun, Xiaochen Peng, Sai Zhang, Jorge Gomez, Win-San Khwa, Syed Sarwar, Ziyun Li, Weidong Cao, Zhao Wang, Chiao Liu, et al. Estimating power, performance, and area for on-sensor deploy- ment of ar/vr workloads using an analytical framework. ACM Trans- actions on Design A...

    2024

  40. [48]

    A 6.84 gbps/lane mipi c-phy transceiver bridge chip with level-dependent equalization

    Pil-Ho Lee and Young-Chan Jang. A 6.84 gbps/lane mipi c-phy transceiver bridge chip with level-dependent equalization. IEEE Transactions on Circuits and Systems II: Express Briefs, 67(11):2672– 2676, 2019. 3

    2019

  41. [49]

    What is Mobile Industry Processor Interface (MIPI) Protocol? 3

  42. [50]

    Power, performance, and image quality tradeoffs in foveated rendering

    Rahul Singh, Muhammad Huzaifa, Jeffrey Liu, Anjul Patney, Hashim Sharif, Yifan Zhao, and Sarita Adve. Power, performance, and image quality tradeoffs in foveated rendering. In 2023 IEEE Conference Vir- tual Reality and 3D User Interfaces (VR) , pages 205–214, 2023. 3, 4

    2023

  43. [51]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 3

    2004

  44. [52]

    This is the author’s version of the article that has been published in the proceedings of IEEE Visualization conference

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and 10 © 2025 IEEE. This is the author’s version of the article that has been published in the proceedings of IEEE Visualization conference. The final version of this record is available at: xx.xxxx/TVCG.201x.xxxxxxx...

    2025

  45. [53]

    Pea-pods: Perceptual evaluation of algorithms for power optimization in xr displays.ACM Transactions on Graphics (TOG), 43(4):1–17, 2024

    Kenneth Chen, Thomas Wan, Nathan Matsuda, Ajit Ninan, Alexandre Chapiro, and Qi Sun. Pea-pods: Perceptual evaluation of algorithms for power optimization in xr displays.ACM Transactions on Graphics (TOG), 43(4):1–17, 2024. 3

    2024

  46. [54]

    Fovvideovdp: A visible difference predictor for wide field-of-view video

    Rafał K Mantiuk, Gyorgy Denes, Alexandre Chapiro, Anton Ka- planyan, Gizem Rufo, Romain Bachy, Trisha Lian, and Anjul Patney. Fovvideovdp: A visible difference predictor for wide field-of-view video. ACM Transactions on Graphics (TOG), 40(4):1–19, 2021. 3, 7

    2021

  47. [55]

    Image features influence reaction time: A learned probabilistic perceptual model for saccade latency

    Budmonde Duinkharjav, Praneeth Chakravarthula, Rachel Brown, Anjul Patney, and Qi Sun. Image features influence reaction time: A learned probabilistic perceptual model for saccade latency. ACM Transactions on Graphics (TOG), 41(4):1–15, 2022. 3

    2022

  48. [56]

    Virtual reality telepresence: 360-degree video streaming with edge-compute assisted static foveated compression

    Xincheng Huang, James Riddell, and Robert Xiao. Virtual reality telepresence: 360-degree video streaming with edge-compute assisted static foveated compression. IEEE Transactions on Visualization and Computer Graphics, 2023. 3

    2023

  49. [57]

    Fov-nerf: Foveated neural radiance fields for virtual reality

    Nianchen Deng, Zhenyi He, Jiannan Ye, Budmonde Duinkharjav, Pra- neeth Chakravarthula, Xubo Yang, and Qi Sun. Fov-nerf: Foveated neural radiance fields for virtual reality. IEEE Transactions on Visu- alization and Computer Graphics, 28(11):3854–3864, 2022. 3

    2022

  50. [58]

    Fovolnet: Fast volume ren- dering using foveated deep neural networks

    David Bauer, Qi Wu, and Kwan-Liu Ma. Fovolnet: Fast volume ren- dering using foveated deep neural networks. IEEE transactions on visualization and computer graphics, 29(1):515–525, 2022. 3

    2022

  51. [59]

    Estimating the just noticeable dif- ference of tactile feedback in oculus quest 2 controllers

    Dixuan Cui and Christos Mousas. Estimating the just noticeable dif- ference of tactile feedback in oculus quest 2 controllers. In2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 1–7, 2022. 3

    2022

  52. [60]

    Komogortsev, and Sachin S

    Cristina Palmero, Abhishek Sharma, Karsten Behrendt, Kapil Krish- nakumar, Oleg V . Komogortsev, and Sachin S. Talathi. Openeds2020: Open eyes dataset, 2020. 3, 6

    2020

  53. [61]

    Meta quest pro

    Meta Platform Inc. Meta quest pro. https://www.meta.com/ quest/quest-pro/, 2022. 4

    2022

  54. [62]

    https://www.nvidia.com/en-us/ design-visualization/rtx/, 2022

    Quadro rtx. https://www.nvidia.com/en-us/ design-visualization/rtx/, 2022. 4

    2022

  55. [63]

    Digital Combat Simulator

    Eagle Dynamics. Digital Combat Simulator. https://www. digitalcombatsimulator.com/en/, 2008. 4

    2008

  56. [64]

    Exploring the effects of image persistence in low frame rate virtual environments

    David J Zielinski, Hrishikesh M Rao, Mark A Sommer, and Regis Kopper. Exploring the effects of image persistence in low frame rate virtual environments. In2015 IEEE Virtual Reality (VR), pages 19–26. IEEE, 2015. 4

    2015

  57. [65]

    Apple Vision Pro, 2024

    Apple Inc. Apple Vision Pro, 2024. 4

    2024

  58. [66]

    Sprague, Zachary Helft, Jared Parnell, J

    W. Sprague, Zachary Helft, Jared Parnell, J. Schmoll, G. Love, and Martin Banks. Pupil shape is adaptive for many species. Journal of Vision, 13:607–607, 07 2013. 4

    2013

  59. [67]

    Roundness

    Wikipedia contributors. Roundness. https://en.wikipedia.org/ wiki/Roundness, 2024. 4

    2024

  60. [68]

    Edgaze: Efficient gaze tracking and deepvog: Deep learning for eye tracking in vr

    Horizon Research and PyDSGZ. Edgaze: Efficient gaze tracking and deepvog: Deep learning for eye tracking in vr. https:// github.com/horizon-research/edgaze and https://github. com/pydsgz/DeepVOG, 2022. Accessed: 2024-09-14. 6

    2022

  61. [69]

    Mi- crosoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C Lawrence Zitnick. Mi- crosoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6- 12, 2014, Procee...

    2014

  62. [70]

    Gaze-contingent real-time simu- lation of arbitrary visual fields

    Jeffrey S Perry and Wilson S Geisler. Gaze-contingent real-time simu- lation of arbitrary visual fields. In Human vision and electronic imag- ing VII, volume 4662, pages 57–69. SPIE, 2002. 7

    2002

  63. [71]

    Sal- icon: Saliency in context

    Ming Jiang, Shengsheng Huang, Juanyong Duan, and Qi Zhao. Sal- icon: Saliency in context. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1072–1080, 2015. 7

    2015

  64. [72]

    https://github.com/gpgpu-sim/gpgpu-sim_ distribution

    Gpgpu-sim. https://github.com/gpgpu-sim/gpgpu-sim_ distribution. 8

  65. [73]

    https://www.notebookcheck.net/ Qualcomm-Adreno-650-GPU-Benchmarks-and-Specs.448196

    Adreno gpu. https://www.notebookcheck.net/ Qualcomm-Adreno-650-GPU-Benchmarks-and-Specs.448196. 0.html. 8 11

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.