REVIEW 2 major objections 2 minor 14 references
Measuring Browser Webcam Gaze Honestly: A Capture-Clock Methodology and Open Reference Implementation
T0 review · 2 major / 2 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Common timestamping hides webcam gaze latency; a capture-clock method measures honest medians of 22–34 ms.
desk verdict The capture-clock method is a real fix for a real bug, but the opaque-engine 'lower bound' for WebGazer is likely mislabeled and flips the paper's headline latency claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrier of the argument is the rVFC frame clock: a per-frame timestamp the browser attaches to each decoded video frame, provided as $captureTime$ for local camera streams and as $presentationTime$ otherwise. It supplies the missing $t_c$ in the latency identity $\ell_I = t_e - t_c$. Exact pairing uses a FIFO queue of frame clocks, dequeued at each gaze emission, so each sample is tagged with its true source frame's clock when the engine processes frames in arrival order. For opaque engines, a single scalar keeps the most recently observed frame clock, and the paper proves the inequality $\tilde{t}_c \ge t_c^{\mathrm{true}}$: a sample cannot be derived from a frame that arrived after the latest observed frame clock at emission time, so $t_e - \tilde{t}_c$ is a verifiable lower bound on the true inference latency.
What would settle it
Instrument an engine to log, in the same process, both the rVFC callback firings and the frame identity each gaze sample consumed; a single sample whose source frame's callback fires after emission falsifies the ordering premise behind the lower bound.
Extended reading notes
Core claim
The paper's central claim is that the ~0 ms inference latencies commonly reported by browser webcam gaze trackers are a measurement artifact: the source frame's capture time is never recorded at the gaze callback, so the emit timestamp silently doubles as the missing capture timestamp, making $\ell_I = t_e - t_c$ read zero for every sample. The fix is to recover a per-frame capture clock from the requestVideoFrameCallback (rVFC) API, using $captureTime$ for local camera streams and $presentationTime$ otherwise. When the engine exposes its per-frame pipeline, a FIFO queue of frame clocks pairs each gaze sample with its exact source frame. When the engine is opaque, as with WebGazer, the most recently observed frame clock gives a verifiable lower bound: $\tilde{\ell}_I = t_e - \tilde{t}_c \le t_e - t_c^{\mathrm{true}}$. On one commodity-laptop session, honest medians are 22–34 ms with p95 of 27–52 ms, a 20–50 ms gap against the naive ~0 ms reading.
Load-bearing premise
The load-bearing premise is that a gaze sample can never be derived from a frame whose rVFC callback fires after the sample is emitted, since otherwise the reported lower bounds on latency would not be guaranteed.
Editorial extensions
If this is right
- A reported all-zero inference-latency column should be read as evidence that the capture timestamp is missing, not as proof that the engine is fast.
- With exact pairing, the FaceMesh+KRR engine shows 22.0–22.8 ms median and 26.8–27.0 ms p95 inference latency; with lower-bound pairing, WebGazer shows 32.8–34.0 ms median and 50.6–52.0 ms p95.
- Against a 50 ms interactive-latency budget, FaceMesh+KRR's p95 pipeline latency of 27–28 ms clears the target on 30 Hz video, while WebGazer's lower bound already exceeds it at 51–52 ms p95, so its true latency fails by at least that margin.
- Spatial spread and temporal jitter are separate quantities: radial p95 is nearly identical between engines (6.13° vs 6.21°) while within-fixation $v_{p99}$ differs by 1.6–3.5×, so reporting only one precision number under-describes an engine.
- With a published weak-supervision segmentation pipeline held fixed, expert eye-tracker gaze trains a usable polyp segmenter (test Dice 0.679) while webcam gaze does not (Dice ≈0), an upper bound on the hardware-only penalty because annotator expertise and viewing instruction changed along with the tracker.
Reading between the lines
- Because the method only needs a video element and rVFC, it should transfer to any browser perception engine that consumes camera frames — face tracking, hand pose, object detection — wherever emission-time timestamps hide per-frame cost.
- For opaque engines, the gap between the reported lower bound and the true latency is the engine's internal queue depth times the frame interval (~33 ms at 30 Hz); instrumenting the engine to expose queue depth would turn the floor into an estimate.
- The $presentationTime$ fallback is tight on the tested hardware (median 0.6 ms behind $captureTime$), but the gap is platform-dependent, so runs should keep recording which clock served; the harness's $capture\_clock\_source$ header already does this.
- The paper's own ablation suggests that accuracy differences below the roughly 4.6° between-run band are not interpretable at N=1; a replicate-per-condition protocol with per-kernel regularization tuning would settle whether webcam gaze can reach lesion-level prompting.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a capture-clock methodology for measuring browser webcam-gaze inference latency. It distinguishes capture time, emission time, and render-handoff time, uses requestVideoFrameCallback as a per-frame capture clock, and pairs gaze samples to source frames exactly for engines that expose their inference loop (FaceMesh+KRR) while applying a claimed lower-bound tag for opaque engines such as WebGazer. The authors report that naive timestamps collapse to ~0 ms median inference latency, whereas the corrected measurements give 22-34 ms medians (27-52 ms p95), a gap that can change whether a 50 ms interactive-latency budget is met. They also feed the gaze into a published weakly-supervised polyp-segmentation pipeline, where expert EyeLink gaze trains a usable segmenter (Dice 0.68) and non-expert webcam gaze does not (Dice ~0).
Significance. If the methodology is correct, this is a useful and simple contribution: the FIFO pairing via rVFC cleanly eliminates the zero-latency artifact for engines whose pipeline is observable, and the open TypeScript implementation and raw CSVs are concrete reproducibility assets. The paper is also admirably explicit about N=1, fixed run order, excluded degraded runs, and confounds in the clinical probe. The exact-pairing part (Section 3.2) is sound under its stated assumptions. However, the claimed lower bound for opaque engines rests on an unproven scheduling-order property (Section 3.3), and because that property supports the WebGazer latency columns and the budget-failure conclusion in Section 4.2, the central claim is not yet fully supported.
major comments (2)
- [§3.3, Eq. (4)] The inequality tilde_t_c >= t_true_c is not established. The justification that a source frame 'can never have arrived after the moment the engine emitted a sample derived from it' establishes only t_true_c <= t_e, not t_true_c <= tilde_t_c. Because WebGazer reads the video element directly (for example, via drawImage inside its own rAF loop), it can consume frame F_k before the rVFC callback for F_k has executed; the most recent observed rVFC clock at emission can then be the clock of an earlier frame, so tilde_t_c < t_true_c(F_k). In that case Eq. (4) yields an upper bound rather than a verifiable lower bound. The paper explicitly states that WebGazer's queue depth is unobservable, so no internal check can verify the ordering. The FaceMesh+KRR exact-pairing column is unaffected because the harness controls frame handoff through the FIFO queue.
- [§4.2, Table 1] Because the WebGazer latency columns depend on Eq. (4), the statements that 'WebGazer reports 32.8-34.0 ms median' and that 'WebGazer's lower bound already exceeds the budget at 51-52 ms p95, so its true latency fails it by at least that margin' are unsupported as written. If Eq. (4) is actually an upper bound, the true WebGazer p95 could be below 50 ms, and the abstract's '20-50 ms gap' would not be established for opaque engines. The paper should validate the rVFC-before-consumer ordering empirically on the target browser, instrument WebGazer's frame access point, or relabel the opaque-engine numbers and remove the budget-failure inference for WebGazer.
minor comments (2)
- [§3.2] The text says captureTime was present on every frame in the 30 s clock probe, yet the runs reported in the paper used the presentationTime fallback; please clarify whether captureTime was unavailable in the engine's rVFC integration or explain why the fallback was deliberately chosen, since the tightness claim is otherwise confusing.
- [Table 1] The dagger for WebGazer latency cites 'lower-bound capture clock (§3.4)', but the lower-bound pairing methodology is described in §3.3, not §3.4; also, FaceMesh+KRR latencies inherit the small presentationTime-vs-captureTime offset, so both columns should carry appropriate annotations or the asymmetry should be explained.
Circularity Check
No circularity: the latency claims are measured against an external rVFC capture clock and the downstream Dice contrast is a fixed-pipeline external benchmark.
full rationale
The paper's load-bearing derivation is the capture-clock measurement chain in Section 3: Eq. (2) pairs each gaze sample with a rVFC frame clock through a FIFO queue, and Eq. (4) tags opaque-engine samples with the most recent observed frame clock. These are operational measurement recipes, not models whose predictions are fitted to their own inputs. No engine parameter, kernel hyperparameter, calibration value, or downstream metric enters the latency equations; FaceMesh+KRR latency is computed as t_e minus the rVFC timestamp of the exact source frame, and the WebGazer value is an independently measured clock difference, not a restatement of a fit. The weakest point, the assertion in Eq. (4) that the source frame cannot have arrived after the emission-derived maximum observed frame clock, is a scheduling assumption about rVFC ordering relative to WebGazer's internal frame consumption. That is a correctness or validity risk, not a circular reduction: even if the inequality tilde_t_c >= t_true_c fails, the reported quantity is still an externally referenced clock difference rather than a quantity equivalent by construction to an input. The downstream GazeMedSeg experiment holds the published pipeline fixed end-to-end and swaps only the gaze CSV; the Dice contrast is an external benchmark, and the paper explicitly frames the gap as an upper bound because annotator expertise changes along with hardware, which is a conservative interpretation rather than a circular derivation. Accuracy comparisons are explicitly not ranked, and the 100ms smooth-pursuit lag is cited to external prior work. No self-citation is load-bearing, and no fitted parameter is renamed as a prediction. The paper is self-contained against the external rVFC clock and external benchmark, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (7)
- RBF kernel width gamma =
median-pairwise-distance heuristic; ranged 8.88e-2 to 1.31e-1 across ablation runs
- Ridge regularization lambda =
1e-3
- One-Euro filter minCutoff =
1.0
- One-Euro filter beta =
0.007 default, swept from 0.003 to 0.030
- I-VT velocity threshold =
1200 px/s, about 18 degrees/s under the paper's geometry
- I-VT stability parameters =
2 consecutive above-threshold frames; at least 3 samples per fixation centroid
- Pursuit calibration delay compensation =
100 ms
assumptions (4)
- domain assumption requestVideoFrameCallback provides captureTime for local camera streams and presentationTime for compositor submission, and capture precedes presentation.
- domain assumption For opaque engines, the source frame of an emitted gaze sample cannot be captured after the most recent rVFC frame clock observed at emission time.
- domain assumption The FaceMesh+KRR engine processes frames in arrival order and emits exactly one gaze sample per processed frame.
- domain assumption WebGazer exposes no per-frame inference entry point, so its queue depth and ordering cannot be observed from outside the library.
Cite this review
Pith. "Pith review of Measuring Browser Webcam Gaze Honestly: A Capture-Clock Methodology and Open Reference Implementation." pith.science (2026). https://pith.science/paper/XTABYPEU
@misc{pith2026260811566,
author = {Pith},
title = {Pith review of: Measuring Browser Webcam Gaze Honestly: A Capture-Clock Methodology and Open Reference Implementation},
year = {2026},
howpublished = {\url{https://pith.science/paper/XTABYPEU}},
note = {Machine review of arXiv:2608.11566}
}
abstract
Browser-based webcam gaze trackers are increasingly used for crowd-scale data collection and in clinical settings where lab eye trackers are impractical, but the reported latency numbers may not represent real world functionality. The common practice of timestamping each gaze sample when it is emitted, rather than when its source frame was captured, makes the measured inference latency read about $0\,$ms no matter how slow the engine really is. We show how to measure it honestly, recovering a per-frame capture clock from the browser's \texttt{re\-quest\-Video\-Frame\-Call\-back} (rVFC) API (\texttt{captureTime} where the browser exposes it for local camera streams, else \texttt{presentationTime}, in which case every recovered latency is a verifiable lower bound): exact source-frame pairing through a per-frame queue for engines that expose their inference pipeline, and a further lower bound for engines that do not, such as WebGazer. We release an open TypeScript implementation and benchmark harness, demonstrated on two interchangeable engines: WebGazer and a new FaceMesh+KRR pipeline.
Figures
Figures from the paper (12 more)
Reference graph
Works this paper leans on
-
[1]
In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI)
Casiez, G., Roussel, N., Vogel, D.: One-Euro filter: A simple speed-based low- pass filter for noisy input in interactive systems. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). pp. 2527–2530 (2012). https://doi.org/10.1145/2207676.2208639, original title uses the euro currency symbol as the “1”-suffix; spelled-out f...
arXiv 2012
-
[2]
The Annals of Statistics36(3), 1171–1220 (2008).https://doi.org/10.1214/ 009053607000000677
Hofmann, T., Schölkopf, B., Smola, A.J.: Kernel methods in machine learning. The Annals of Statistics36(3), 1171–1220 (2008).https://doi.org/10.1214/ 009053607000000677
work page 2008
-
[3]
Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., de Lange, T., Johansen, D., Johansen, H.D.: Kvasir-SEG: A segmented polyp dataset. In: MultiMedia Modeling (MMM). Lecture Notes in Computer Science, vol. 11962, pp. 451–462. Springer (2020).https://doi.org/10.1007/978-3-030-37734-2_37
-
[4]
Kartynnik, Y., Ablavatski, A., Grishchenko, I., Grundmann, M.: Real-time fa- cial surface geometry from monocular video on mobile GPUs. In: CVPR Work- shop on Computer Vision for Augmented and Virtual Reality (CV4ARVR) (2019), arXiv:1907.06724
arXiv 2019
-
[5]
Khosravan, N., Celik, H., Turkbey, B., Cheng, R., McCreedy, E., McAuliffe, M., Bednarova, S., Jones, E., Chen, X., Choyke, P.L., Wood, B.J., Bagci, U.: Gaze2Segment: A pilot study for integrating eye-tracking technology into medi- cal image segmentation. In: Medical Computer Vision and Bayesian and Graph- ical Models for Biomedical Imaging (MCV & BAMBI), ...
-
[6]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., Dollár, P., Girshick, R.: Segment anything. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 4015–4026 (2023)
work page 2023
-
[7]
In: Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI)
Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., Hays, J.: We- bGazer: Scalable webcam eye tracking using user interactions. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI). pp. 3839–3845 (2016) Measuring Browser Webcam Gaze Honestly 11
work page 2016
-
[8]
In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST)
Pfeuffer, K., Vidal, M., Turner, J., Bulling, A., Gellersen, H.: Pursuit calibration: Making gaze calibration less tedious and more flexible. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST). pp. 261–270 (2013).https://doi.org/10.1145/2501988.2501998
arXiv 2013
Show all 14 references
-
[9]
In: Proceedings of the Symposium on Eye Tracking Research and Ap- plications (ETRA)
Salvucci, D.D., Goldberg, J.H.: Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the Symposium on Eye Tracking Research and Ap- plications (ETRA). pp. 71–78 (2000).https://doi.org/10.1145/355017.355028
2000
-
[10]
IEEE Transactions on Medical Imaging36(1), 86–97 (2017).https://doi.org/ 10.1109/TMI.2016.2593957, the Cholec80 dataset was introduced in this paper
Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., de Mathelin, M., Padoy, N.: EndoNet: A deep architecture for recognition tasks on laparoscopic videos. IEEE Transactions on Medical Imaging36(1), 86–97 (2017).https://doi.org/ 10.1109/TMI.2016.2593957, the Cholec80 datase...
2017
-
[11]
In: Proceedings of The 2nd Gaze Meets ML Workshop
Wang, B., Aboah, A., Zhang, Z., Pan, H., Bagci, U.: GazeSAM: Interactive im- age segmentation with eye gaze and segment anything model. In: Proceedings of The 2nd Gaze Meets ML Workshop. Proceedings of Machine Learning Research, vol. 226, pp. 254–265. PMLR (2024), earlier arXi...
2024 arXiv
-
[12]
W3C WICG Editor’s Draft (2024), editor: T
Web Incubator Community Group: HTMLVideoElement.requestVideoFrameCall- back() specification. W3C WICG Editor’s Draft (2024), editor: T. Guilbert.https: //wicg.github.io/video-rvfc/, accessed 2026
2024
-
[13]
arXiv preprint arXiv:1504.06755 (2015)
Xu, P., Ehinger, K.A., Zhang, Y., Finkelstein, A., Kulkarni, S.R., Xiao, J.: Turk- erGaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755 (2015)
2015 arXiv
-
[14]
engine+its default calibration,
Zhong, Y., Tang, C., Yang, Y., Qi, R., Zhou, K., Gong, Y., Heng, P.A., Hsiao, J.H., Dou, Q.: Weakly-supervised medical image segmentation with gaze anno- tations. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. Lecture Notes in Computer Science, v...
2024 arXiv
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.