{"id":"3f977fdd-5879-42a7-b4eb-9602a5a4c295","arxiv_id":"2412.10456","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FovealNet reduces the long-tail errors of gaze tracking for foveated VR rendering by training with a soft-max of per-sample errors mapped through a fitted rendering-latency function, plus input cropping and token pruning.","lead":"FovealNet is a gaze-tracking system for virtual reality that trains its neural network to minimize the worst-case tracking errors that break foveated rendering, while cropping eye images and pruning unneeded tokens to reduce computation. It reports lower 95th-percentile gaze error, a 1.42x latency speedup over earlier gaze trackers, and better perceptual quality on simulated foveated output.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 1.42x speedup claim rests on the same unvalidated latency model used as the training loss; end-to-end TFR latency was never independently measured on the target device.","rationale":"The reader's weakest assumption identifies precisely the load-bearing concern: the unvalidated latency model U(.), used both as the training objective and as the evaluation oracle, together with simulated rather than measured tracking latency, underpins the headline speedup. The accuracy-side contribution is more solid: FovealNet's tail-error reduction on OpenEDS2020 is a concrete, reproducible-in-principle result, and the perceptual-quality simulation with FovVideoVDP is a sensible methodology. However, the system-level claim is not independently established without a direct end-to-end latency measurement. Since the reader already recommends conditional acceptance, my stress test does not change the verdict; it sharpens the condition: provide on-device latency measurements and validate U(.) against real TFR rendering latency, or release artifacts so others can run the check.","tokens_in":20917,"tokens_out":4893,"duration_ms":52689,"concrete_test":"Implement FovealNet and one representative baseline (e.g., the ResNet-based method) on a Quest Pro or equivalent XR2+ device; profile T_tracking with on-device counters and measure T_fr end-to-end by rendering foveated frames at 1080x1920 across at least 50 frames from multiple scenes not used to fit U(.). Recompute T_tracking+T_fr and the speedup ratio relative to the fastest prior method. If the ratio falls below 1.42x, or if total latency exceeds the 16.7 ms 60 FPS budget, the headline system claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central system-level claim (abstract; Sec. 6.1) is that FovealNet yields at least a 1.42x speedup in per-frame TFR latency. That claim depends on two quantities: T_tracking, obtained from GPGPU-sim of an Adreno 650 instead of an on-device profiler, and T_fr, read from the piecewise-linear function U(.) fitted to rendering-latency measurements on 50 DCS frames for the Quest Pro and four scenes for the mobile GPU. Critically, the same U(.) is used as the performance-aware training loss in Eq. (4). The evaluation in Sec. 6.1 therefore measures system benefit through the same surrogate that the model was explicitly trained to minimize, so the reported reduction in T_fr is at least partly an in-sample consequence of the chosen loss. If the fitted U(.) does not accurately predict real rendering cost across scenes, resolutions, or system load, the reported T_tracking+T_fr values and the 1.42x ratio are not end-to-end validated. The missing crop-parameter definition ('??' in Sec. 5.1) and the absence of released code/weights further prevent independent reproduction of the reported numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FovealNet, a ViT-based gaze tracker for foveated rendering in VR, with three main contributions: an event-based cropping method that removes background pixels, a token-pruning strategy for the ViT, and a \"performance-aware\" training loss (Eq. (4)) that applies a fitted piecewise-linear rendering-latency function U(.) to a soft-max approximation of the maximum gaze error. The authors evaluate tracking accuracy on OpenEDS2020, report reductions in the 95th-percentile gaze error (e.g., for FovealNet 0.0, from 8.21 to 2.31 degrees), a foveated perceptual-quality improvement in JND, and system-level latency figures on a Meta Quest Pro (simulated with GPGPU-sim) and a mobile GPU, claiming at least a 1.42x speedup over prior methods.","tokens_in":21012,"tokens_out":8656,"duration_ms":81671,"significance":"If the reported accuracy numbers hold, the work provides a potentially useful observation: training a gaze tracker with a soft-max maximum-error loss substantially reduces the long tail of gaze errors, which is directly relevant to foveated rendering quality. The cropping and token-pruning techniques are sensible and appear to reduce FLOPs, and the OpenEDS2020 evaluation is a standard benchmark. However, the central system-performance contribution is not established: the latency model U(.) used to define the training loss is also used to compute the reported speedups, and the loss is mathematically equivalent to a monotone transform of the soft-max error if U(.) is monotonic, which weakens the claim that the training is genuinely \"system performance-aware.\" The paper also omits code/weights and at least one key definition (the cropping parameters), limiting reproducibility.","major_comments":[{"comment":"The performance-aware loss in Eq. (4) applies the piecewise-linear function U(.) to the soft-max approximation of the maximum gaze error. Since rendering latency U(.) is monotonically increasing in the eccentricity angle, as shown in Fig. 6(c), minimizing U(softmax_error) is equivalent to minimizing the soft-max error itself; the specific shape of U(.) cannot affect the training optimum. Thus the claim that the loss \"directly optimizes TFR system latency\" is not supported, and the improvements in Fig. 10 should be attributed to the maximum-error objective of Eq. (3) rather than to the fitted latency model. To substantiate the system-awareness claim, the authors should either use a non-monotonic U(.) or demonstrate that different fitted latency curves yield different optimal models.","section":"Sec. 4.3, Eq. (4)"},{"comment":"The reported T_fr values and the \"at least 1.42x speedup\" claim in the abstract are computed from the same piecewise-linear function U(.) that defines the training loss in Eq. (4), and the Quest Pro T_tracking numbers are obtained from GPGPU-sim rather than on-device measurements. Because FovealNet is explicitly trained to minimize U(.), the resulting T_fr reduction is at least partly an in-sample outcome of the loss, and the paper provides no independent validation of U(.) against end-to-end TFR latency across scenes, resolutions, or system load. The speedup claim should be backed by direct end-to-end measurement on the target devices, or explicitly re-labeled as an estimate under the fitted latency model with a sensitivity analysis.","section":"Sec. 6.1"},{"comment":"The cropping parameters beta1=0.2 and beta2=500 are introduced with a reference to \"??\", but the formula or section that defines them is missing. Since event-based cropping is one of the three main contributions, the omitted definition prevents readers from reproducing the reported results.","section":"Sec. 5.1"},{"comment":"The multi-resolution evaluation in Sec. 6.2 selects subnetworks that minimize T_tracking + T_fr without applying any visual-quality constraint. For example, at 720p the optimal depth-3 subnetwork has a P95 gaze error of 7.35 degrees and a maximum error of 15.70 degrees, which by the paper's own analysis in Sec. 3.2 (Fig. 5(b)) would produce a noticeable increase in JND and degrade foveated output. The claim that the system selects the \"optimal\" configuration is therefore incomplete unless a perceptual-quality constraint is imposed or the JND is reported for each selected configuration.","section":"Sec. 6.2, Tab. 2"}],"minor_comments":[{"comment":"The column header \"P0\" should be \"P90\"; the text also states that P0 represents the 90th percentile, which is inconsistent.","section":"Table 1"},{"comment":"The FLOPs comparison percentages are inconsistent: FovealNet (0.2) at 2.08B versus ResNet-based at 3.6B is approximately 42% lower, not 70% lower; versus Seg at 2.6B it is approximately 20% lower, not 27% lower.","section":"Sec. 5.2"},{"comment":"The phrase \"13% increase in perceptual quality\" is misleading; Sec. 5.4 reports a 13% increase in discriminability for the ResNet baseline relative to FovealNet, which is a decrease in quality. The abstract should state a reduction in JND.","section":"Abstract"},{"comment":"The Fig. 6(a) caption says NVGaze results were excluded due to high tracking error and inconsistent performance, but Table 1 reports NVGaze results; the text should be reconciled.","section":"Fig. 6(a) caption and Table 1"},{"comment":"The text refers to \"T_tf\", which appears to be a typo for \"T_fr\" (foveated rendering latency).","section":"Sec. 4.3"},{"comment":"The claim that Tsensing and Tcomm are small cites Fig. 2, but Fig. 2 is a system configuration diagram; the latency breakdown appears in Fig. 4(b). Please correct the reference.","section":"Sec. 6.1"}],"recommendation":"major_revision","confidential_remarks":"The accuracy results for the tail-error reduction appear solid and could be a useful contribution. The main concern is conceptual: the performance-aware loss is equivalent to a monotone transform of the soft-max error if U(.) is monotonic, so the system-awareness novelty is not yet established. The system-speedup numbers also depend on the same unvalidated surrogate used as the training loss. I would ask the authors to address these two points head-on and to add the missing definition of beta1/beta2. If they can provide independent end-to-end latency measurements or substantially re-frame the system claims as estimates, the paper may become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is real and worth engaging with: training a gaze-tracking ViT with a loss that approximates worst-case gaze error and maps it through a fitted rendering-latency function is a sensible, novel way to align the tracker with the actual TFR objective. The accuracy experiments on OpenEDS2020 are the strongest part. The held-out test set shows a large, credible reduction in the long tail: P95 error drops from 8.21 degrees to 2.31 degrees for FovealNet 0.0, and the perceptual evaluation with FovVideoVDP is a reasonable way to connect gaze error to perceived quality. Those results alone justify a serious look.\n\nThe soft spots are mostly around the system-level claims, and they are real. The 1.42x speedup is not an end-to-end measurement. Tracking latency on the Quest Pro comes from GPGPU-sim, and the rendering latency is read off the same piecewise-linear U(.) that defines the training loss. So the reported T_fr reduction is partly in-sample: the model was explicitly trained to minimize that surrogate, and the evaluation then uses the same surrogate to claim the benefit. The paper should label these as estimated latencies and ideally validate U(.) against a real TFR pipeline on the target device, or at least against a broader set of scenes and resolutions. Without that, the absolute latency numbers and the speedup ratio are suggestive, not demonstrated.\n\nSmaller issues, in proportion: no code or weights released, the cropping parameter definition is missing (broken '??' in Sec. 5.1), accuracy tables lack error bars, and the NVGaze baseline is handled inconsistently (excluded from Fig. 6a but present in Table 1). None of these undercut the central accuracy mechanism.\n\nWho is this for? People working on gaze tracking for VR/AR, and more broadly anyone thinking about training DNNs with system-level objectives rather than pure accuracy. It deserves a serious referee. My recommendation: send it out, and expect the reviewers to push for either direct latency measurements or a clear re-framing of the speedup as simulated, plus release of the artifacts so the accuracy results can be reproduced.","headline":"A genuinely new performance-aware gaze-tracking loss that credibly cuts long-tail error on OpenEDS2020; the headline 1.42x system speedup is not actually measured end-to-end and rests on the same latency model used for training.","tokens_in":21704,"tokens_out":1235,"would_cite":true,"duration_ms":14555,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VR gaze-tracking long-tail error shrinks from 8.21 to 2.31 degrees.","keywords":["foveated rendering","gaze tracking","performance-aware training","event-based cropping","token pruning","vision transformer","virtual reality","latency optimization"],"falsifier":"Measure the actual end-to-end latency of a TFR system with FovealNet running on a Meta Quest Pro or comparable headset, including real on-device gaze tracking plus foveated rendering, and compare the P90/P95-driven $T_{\\text{tracking}} + T_{\\text{fr}}$ against the paper's simulated figures; if real total latency is not at least 1.42x lower than the best baseline under the same scene and resolution conditions, the central speedup claim fails. A simpler check is whether the fitted $U(\\cdot)$ curve, taken from the paper, predicts a measured rendering latency on a held-out scene within 10%.","tokens_in":20561,"feed_emoji":"🥽","tokens_out":10274,"duration_ms":86942,"temperature":0.7,"pith_summary":"This paper claims that gaze-tracking errors for foveated rendering should be optimized against the downstream task, not against average accuracy alone. It introduces FovealNet, a vision-transformer gaze tracker trained with a performance-aware loss that maps gaze error to expected rendering latency. On the OpenEDS2020 dataset, that loss cuts the 95th-percentile gaze error from 8.21° to 2.31° for the unpruned model, removing the long tail of large errors that previously forced foveated renderers to enlarge the high-resolution region. A reader should care because the long tail is what breaks foveated rendering: when the predicted gaze misses the true fovea, visual quality drops and rendering latency rises.","feed_headline":"VR gaze-tracking long-tail error shrinks from 8.21 to 2.31 degrees","feed_subtitle":"Training the tracker to minimize rendering latency, not average error, keeps foveated VR sharp and fast.","key_machinery":"The load-bearing mechanism is the performance-aware loss of Eq. (4): a soft-max approximation of the maximum per-batch gaze error, passed through a fitted piecewise-linear latency function $U(\\cdot)$ that converts an angular tracking error into the rendering latency needed to keep the foveal region aligned. $U(\\cdot)$ is constructed from rendering-latency measurements on a Meta Quest Pro (50 frames from one scene) and a mobile GPU (four scenes) at three resolutions. Because the same $U(\\cdot)$ appears in the training loss and in the reported system-speedup numbers, the argument reduces to a single identity: if $U(\\cdot)$ predicts real TFR latency from gaze error, then minimizing $U$ of the worst-case error directly lowers the component of total latency that gaze error controls. Around this core sit two efficiency mechanisms: an event-based cropping algorithm that locates the pupil as the largest connected dark component and crops a 450 × 200 region, removing over 64.8% of pixels, and a token-pruning pass that drops low-attention ViT tokens on the fly.","core_discovery":"FovealNet's central claim is that a gaze-tracking DNN can be trained to directly minimize the total latency of a gaze-tracked foveated-rendering (TFR) system by replacing the usual mean gaze error with a loss that emphasizes the worst errors and weighs them by their rendering cost. The proposed loss, Eq. (4), is $U\\left(\\frac{1}{N}\\ln\\left(\\sum_{d\\in D_b} e^{N\\|\\theta_d - \\theta_d^g\\|_2}\\right)\\right)$, where $U(\\cdot)$ is a piecewise-linear function fitted to measured rendering latency versus foveal eccentricity, and the soft-max inside approximates the maximum error in a batch. Training with this loss reduces the 95th-percentile error from 8.21° to 2.31° for the full FovealNet (0.0) and the maximum error from 24.2° to 5.22° on OpenEDS2020. The paper also claims that this training objective, combined with event-based pupil-centered cropping and token-wise pruning of the ViT, yields a per-frame latency $T_{\\text{tracking}} + T_{\\text{fr}}$ of about 10.2 ms in the P90 scenario, at least 1.42x faster than prior gaze trackers, and a foveated-rendering perceptual JND of 0.07, which the paper reads as imperceptible degradation.","pith_inferences":["Because $U(\\cdot)$ is just a mapping from error to downstream cost, the same training principle should transfer to other perception-to-action loops, such as autofocus, gaze-contingent depth of field, or adaptive bitrate streaming, whenever a measurable cost curve exists.","The paper's speedup numbers inherit $U(\\cdot)$'s accuracy: if $U(\\cdot)$ were refitted with real on-device tracking latency and more rendering scenes, the relative standing of FovealNet versus baselines like Seg or NVGaze could change, since those baselines were not retrained under the performance-aware objective.","A testable extension is to train FovealNet with separate cost functions for power instead of latency, since the paper says the framework can be extended to other system performance metrics.","The multi-resolution selection rule could be made fully automatic by predicting $T_{\\text{fr}}$ from current scene complexity and resolution, rather than choosing subnet depth by render resolution alone."],"forward_implications":["Training a gaze tracker with a performance-aware loss directly shrinks the worst-case errors that force foveated renderers to enlarge the high-resolution region, so the reported gains are latency gains, not just accuracy gains.","The unpruned FovealNet cuts 95th-percentile gaze error from 8.21° to 2.31° and maximum error from 24.2° to 5.22°, which the paper ties to a foveated-image JND of 0.07.","Event-based cropping removes over 64.8% of input pixels and token-wise pruning keeps FovealNet in the 2.08 to 2.80 billion FLOP range, making the ViT cheap enough for real-time headset use.","The multi-resolution framework yields one ViT with six subnetworks of depths 3 through 8; under 720P rendering the depth-3 subnetwork achieves $T_{\\text{tracking}} + T_{\\text{fr}}$ of 6.19 ms, while under 1440P rendering the depth-6 subnetwork achieves 16.4 ms, demonstrating runtime depth selection."],"supporting_citations":[{"why":"Supplies the OpenEDS2020 eye images and ground-truth gaze vectors on which every reported error number is computed.","marker":"[60]"},{"why":"Defines the TFR latency decomposition and the eccentricity-based rendering-latency relationship that $U(\\cdot)$ is fitted to represent.","marker":"[2]"},{"why":"Establishes that gaze-tracking overhead can outweigh foveated-rendering savings, motivating the latency-aware training objective.","marker":"[50]"},{"why":"Baseline gaze tracker whose long-tailed errors are compared; its 95th-percentile error drives a rendering latency above 20 ms in the paper's analysis.","marker":"[11]"},{"why":"Baseline appearance-based tracker with high average error used to motivate the need for performance-aware training.","marker":"[31]"},{"why":"Vision Transformer architecture that FovealNet extends with early-exit branches and token pruning.","marker":"[30]"},{"why":"Provides the FovVideoVDP metric used to measure foveated-image quality in JND units.","marker":"[54]"},{"why":"GPU simulator used to estimate gaze-tracking latency on the Quest Pro's Adreno 650 when no on-device ViT run is available.","marker":"[72]"}],"fun_headline_variants":["FovealNet slashes VR gaze error from 8.21° to 2.31°","Gaze tracker trained to minimize rendering latency, not error","FovealNet's loss targets worst gaze errors, boosts VR speed 1.42x","FovealNet prunes 64.8% of pixels, keeps VR gaze sharp","FovealNet makes foveated rendering imperceptible: JND 0.07"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the piecewise-linear function $U(\\cdot)$, fitted to rendering-latency measurements on a Meta Quest Pro (50 frames from one scene) and a mobile GPU (four scenes), accurately predicts real end-to-end TFR latency, and it uses that same model both to define the training loss and to compute the reported speedups; tracking latency on the Quest Pro is simulated, not measured on the device.","fun_headline_variants_meta":{"raw":{"variants":["FovealNet slashes VR gaze error from 8.21° to 2.31°","Gaze tracker trained to minimize rendering latency, not error","FovealNet's loss targets worst gaze errors, boosts VR speed 1.42x","FovealNet prunes 64.8% of pixels, keeps VR gaze sharp","FovealNet makes foveated rendering imperceptible: JND 0.07"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00085,"raw_usage":{"total_tokens":3788,"prompt_tokens":1128,"completion_tokens":2660,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":744,"completion_tokens_details":{"reasoning_tokens":2560}},"tokens_in":744,"tokens_out":2660,"duration_ms":19089,"temperature":1.0,"reasoning_tokens":2560,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:21:16.329178+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the actual end-to-end latency of a TFR system with FovealNet running on a Meta Quest Pro or comparable headset, including real on-device gaze tracking plus foveated rendering, and compare the P90/P95-driven $T_{\\text{tracking}} + T_{\\text{fr}}$ against the paper's simulated figures; if real total latency is not at least 1.42x lower than the best baseline under the same scene and resolution conditions, the central speedup claim fails. A simpler check is whether the fitted $U(\\cdot)$ curve, taken from the paper, predicts a measured rendering latency on a held-out scene within 10%.","supporting_citations":[{"cited_title":"Komogortsev, and Sachin S","cited_arxiv_id":null,"evidence_quote":"Supplies the OpenEDS2020 eye images and ground-truth gaze vectors on which every reported error number is computed."},{"cited_title":"La- tency requirements for foveated rendering in virtual reality","cited_arxiv_id":null,"evidence_quote":"Defines the TFR latency decomposition and the eccentricity-based rendering-latency relationship that $U(\\cdot)$ is fitted to represent."},{"cited_title":"Power, performance, and image quality tradeoffs in foveated rendering","cited_arxiv_id":null,"evidence_quote":"Establishes that gaze-tracking overhead can outweigh foveated-rendering savings, motivating the latency-aware training objective."},{"cited_title":"Real-time gaze tracking with event-driven eye segmenta- tion","cited_arxiv_id":null,"evidence_quote":"Baseline gaze tracker whose long-tailed errors are compared; its 95th-percentile error drives a rendering latency above 20 ms in the paper's analysis."},{"cited_title":"Nvgaze: An anatomically-informed dataset for low-latency, near-eye gaze estimation","cited_arxiv_id":null,"evidence_quote":"Baseline appearance-based tracker with high average error used to motivate the need for performance-aware training."},{"cited_title":"Fovvideovdp: A visible difference predictor for wide field-of-view video","cited_arxiv_id":null,"evidence_quote":"Provides the FovVideoVDP metric used to measure foveated-image quality in JND units."},{"cited_title":"https://github.com/gpgpu-sim/gpgpu-sim_ distribution","cited_arxiv_id":null,"evidence_quote":"GPU simulator used to estimate gaze-tracking latency on the Quest Pro's Adreno 650 when no on-device ViT run is available."}],"review_version":1}