Pith. sign in

REVIEW 3 major objections 6 minor 25 references

MAGE: A Multi-task Architecture for Gaze Estimation with an Efficient Calibration Module

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MAGE claims that complete 6-DoF gaze—direction plus position—can be predicted from a single RGB image and face bounding box, and that a screen-free calibration from roughly 50 frames adapts it to each user.

desk verdict MAGE has a sound multi-task gaze architecture and a practical screen-free calibration idea, but the calibration labels rely on a face-center proxy that likely swamps the reported gain—read the IMRGaze results with caution. read the letter →

arxiv 2505.16384 v1 pith:2QC7W6SW submitted 2025-05-22 cs.CV cs.HC

Haoming Huang , Musen Zhang , Jianxin Yang , Zhen Li , Jinkai Li , Yao Guo This is my paper · ORCID
classification cs.CVcs.HC
keywords gazeestimation6-DoFpointofmulti-tasklearningperson-specificcalibrationEasy-Normhuman-robotinteractionRGB-onlytracking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MAGE is a gaze-estimation network that takes a single RGB image plus a face bounding box and predicts both the 3D direction of gaze and where the gaze ray crosses the camera's XY-plane, a quantity the paper calls PoGz. Together these give the full six-degree-of-freedom gaze (origin and direction) needed for human-robot interaction. To handle person-to-person differences in eye anatomy, the paper adds Easy-Calibration, a screen-free procedure in which a user simply looks at the camera lens while moving his or her head; about 50 frames per person are enough to fine-tune the network. On MPIIFaceGaze, EYEDIAP, and the authors' new IMRGaze dataset, the method reports lower gaze-direction and point-of-gaze errors than the specialized direction-only and PoG-only baselines.

What carries the argument

The load-bearing mechanism is Easy-Norm plus Easy-Calibration, coordinated through multi-task decoders. Easy-Norm standardizes camera intrinsics and rotates the camera z-axis toward the face center using only the facial bounding box, so the network sees a normalized face without requiring head pose or depth. Easy-Calibration generates ground-truth labels without a screen: while the subject gazes at the camera lens and moves the head, Eq. 8 defines the gaze vector as pointing from the projected face-center to the lens center, and these frames fine-tune the model. PoGz—the intersection of the gaze ray with the XY-plane of the camera coordinate system—is the device-independent positional output that lets the same model be evaluated on different screens and merges directional and positional supervision.

What would settle it

Run a controlled comparison on the same subjects between Easy-Calibration and a standard screen-based calibration: if the screen-based method yields materially lower gaze error, the Eq. 8 face-center/lens-center assumption is biased. A more direct check is to simulate an eye model with known kappa angle and verify that Eq. 8's ground-truth vector coincides with the simulated visual axis.

Watch

Extended reading notes

Core claim

The central claim is that gaze direction and point of gaze should be estimated jointly, and that the joint task can be made accurate across people with a bounding-box-only normalization and a screen-free personalization step. MAGE's Easy-Norm rotates the camera coordinate system so its optical axis points at the face center, using only the face bounding box, and then feeds the normalized face through separate gaze, pose, and box encoders. Decoders jointly output the normalized gaze vector, the original gaze vector, the intersection of the gaze ray with the camera's XY-plane (PoGz), the normalization rotation, and the face center, all supervised by a multi-task loss. On MPIIFaceGaze the paper reports 3.54° direction error and 32.73 mm point-of-gaze error; on EYEDIAP-Screen, 4.64° and 73.07 mm; on EYEDIAP-Float, 5.11°; and on its own IMRGaze dataset, 5.15° after Easy-Calibration with about 100 calibration frames.

Load-bearing premise

The screen-free calibration labels rely on the assumption that when a person looks at the camera lens, the gaze ray is exactly the line from the face bounding box center to the lens center; a large or systematic kappa angle, or a face-center proxy that does not match the true gaze origin, would bias the fine-tuning.

Editorial extensions

If this is right

  • A depth sensor is unnecessary for complete 6-DoF gaze: one RGB camera plus a face detector supplies both direction and position.
  • Because PoGz is device-independent, the same trained model can serve both screen-pointer tasks and free-space human-robot interaction without retraining for each screen.
  • Easy-Calibration makes per-user adaptation practical outside the lab, requiring about 50 frames and roughly 17 seconds of video, with no screen-based annotations.
  • Supervising the model with PoGz improves its gaze-direction accuracy: removing the PoGz branch raises IMRGaze direction error from 5.38° to 5.83°.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • PoGz's device independence suggests the model could be trained on multiple camera geometries and deployed on a new camera with no retraining; the paper does not test this cross-device transfer explicitly.
  • The calibration protocol could be folded into normal human-robot interaction, since moments when a user deliberately looks at the robot's camera are exactly the frames Eq. 8 needs; this would allow continuous, passive personalization.
  • Because the network also predicts the normalization rotation and gaze origin, a self-supervised consistency loss linking go, PoGz, and the rotation could further tighten predictions; the paper lists this as future work.
  • A testable extension is to apply Easy-Calibration with a kappa-angle correction to isolate how much of the reported 0.23° improvement is true individual-geometry adaptation rather than generic fine-tuning.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes MAGE, a multi-task architecture that takes a single RGB image and a facial bounding box and predicts a 6-DoF gaze representation composed of gaze direction (gn and go) and a gaze point PoGz on the camera XY-plane. It introduces Easy-Norm, a normalization procedure that uses only the bounding box, and Easy-Calibration, a screen-free personalization module whose calibration labels are generated by having the subject look at the camera lens. Experiments on MPIIFaceGaze, EYEDIAP, and a new IMRGaze dataset report state-of-the-art or improved results, including a 0.23° calibration gain on IMRGaze.

Significance. If the claims held, MAGE would offer a practical contribution to HRI by providing both direction and position of gaze from a single RGB image with a lightweight calibration procedure, and the IMRGaze dataset could be a useful benchmark. The multi-task design is sensible, the Easy-Norm idea of avoiding head-pose/depth requirements is useful, and the ablation study gives a concrete account of the contribution of each branch. However, the paper does not ship code or a public release of IMRGaze, and the current evidence for the state-of-the-art and calibration claims is weakened by protocol inconsistencies and an unclear label-origin definition. The screen-free calibration idea is interesting but needs validation with explicitly consistent ground-truth definitions.

major comments (3)
  1. [Section II-E (Eq. 8) and Section III-A] The calibration ground truth in Eq. (8) is the vector from the face-bounding-box center projection to the camera origin. The paper does not state whether the IMRGaze general-data ground-truth gaze vector is defined from the same face-center origin or from the eye centers. The Introduction defines gaze direction as originating from the ocular center, and public datasets such as MPIIFaceGaze and EYEDIAP use eye-based gaze vectors. If the IMRGaze evaluation labels are eye-based, then Eq. (8) labels carry a systematic bias from the face-eye offset and the kappa angle, which is likely larger than the reported 0.23° calibration improvement. If the evaluation labels are also face-center-based, then the reported calibration gain is for a nonstandard definition and cannot be compared with the public-dataset baselines. The authors must specify and justify the gaze-origin definition in both label sets and demonstrate that the calibration and evaluation labels are consistent.
  2. [Table II and Section IV-A] The state-of-the-art claim relies on comparisons to numbers taken from other papers, including iTracker and AFF-Net quoted from the review [6]. There is no demonstration that the baselines were evaluated under the same normalization, input format (face vs. eyes), and cross-validation splits. The reported margins, e.g., 3.54° vs. 3.82° on MPIIFaceGaze, may be within protocol differences, and no error bars or statistical significance tests are reported. Please either re-evaluate all baselines under a unified protocol or restrict the claim to competitive performance with an explicit caveat that protocol details differ.
  3. [Section II-C (Eq. 3)] The transformation of the gaze direction vector into the screen coordinate system is incorrect. In Eq. (3), g_s = R_s^c · g_o + t_s^c treats a direction vector as a point; a unit direction transforms as g_s = R_s^c · g_o without the translation term. If this equation is used for the PoG conversion that underlies the PoG errors in Table II, the reported PoG results are not trustworthy. This should be corrected and the PoG numbers recomputed.
minor comments (6)
  1. [Abstract] "real-word HRI" should be "real-world HRI".
  2. [Section II] The gaze origin is defined as "face center" in the methodology, but the Introduction states that gaze direction originates from the ocular center. Please align the terminology and clarify which origin the model actually predicts.
  3. [Section III-A] The description of the IMRGaze general-data label computation is underspecified; please provide the exact formula used to convert RGB-D and MediaPipe outputs into the gaze-vector ground truth.
  4. [Figure 3 and Eq. (3)] The notation involving O_PoGz and O_s_PoGz is confusing, and "intersection point of the original gaze vector go" should read "intersection point of the gaze ray associated with go".
  5. [Section III-D] There is a missing space in "λPoGz = 0and freezing".
  6. [Section III-A] The paper does not state whether the IMRGaze dataset will be released; please clarify data and code availability for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MAGE's outputs are supervised against external ground truth, and Easy-Calibration is a standard fine-tuning protocol with an explicitly stated geometric approximation.

full rationale

The paper's derivation chain is self-contained and does not reduce to its inputs. Easy-Norm is a deterministic preprocessing transform (Eqs. 1-2) whose parameters are computed from the input bounding box, not learned from the targets. The supervised outputs gn, go, PoGz, rno, and Oface are each trained against ground truth from public datasets (MPIIFaceGaze, EYEDIAP) or from camera geometry (IMRGaze), so no output is defined in terms of the model's own predictions. PoGz is the intersection of the ground-truth gaze ray with the CCS XY-plane, and Eq. 3 provides an invertible geometric link to PoG; this is a coordinate relationship, not a renamed empirical result. Easy-Calibration labels in Eq. 8 use an explicit approximation (the gaze ray from the face-center projection to the camera lens center, neglecting kappa), but the module is then fine-tuned on held-out frames and evaluated on separate IMRGaze test frames, which is a standard personal-calibration protocol rather than a fitted parameter masquerading as a prediction. The self-citation [20] motivates the calibration module but is not load-bearing: the equations, data collection, and ablations are all given in the present paper. Concerns about the face-center proxy injecting systematic bias are validity and accuracy risks for real gaze, not circularity in the derivation.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

The central claim rests on standard deep learning assumptions and on a geometric proxy for gaze origin. The main free parameters are training loss weights and the unspecified standardized focal length. The invented entities, PoGz and '6-DoF gaze', are mathematical descriptions rather than new physical objects, and neither has independent empirical support outside the paper.

free parameters (2)
  • Multi-task loss weights (lambda_gn, lambda_go, lambda_PoGz, lambda_rno, lambda_face) = lambda_gn=1, lambda_go=0.5, lambda_PoGz=0.1, lambda_rno=0.5, lambda_face=0.1
    Chosen by hand in Section III-C; no sensitivity analysis is provided, so the reported gains may depend on these values.
  • Standardized focal length fs in Easy-Norm
    Easy-Norm standardizes the camera intrinsic to a focal length fs (Section II-A), but the actual value is not specified, making the normalization not fully reproducible.
assumptions (4)
  • domain assumption The center of the facial bounding box approximates the 3D gaze origin (Oface defined as face center in this study).
    Used throughout Section II-D to define gaze origin and to derive calibration labels; individual anatomy makes this approximation imperfect.
  • domain assumption A subject looking at the camera lens center yields a gaze vector exactly along the line from the face center projection to the camera origin (Eq. 8).
    Assumes negligible kappa angle and that face center projection lies on the visual axis; this is the labeling rule for calibration data.
  • standard math Standard pinhole camera projection and coordinate transforms between CCS and SCS (Eqs. 1-3).
    Used to derive normalization rotation and PoG conversion.
  • domain assumption Deep networks with L1 losses and multi-task constraints can learn the gaze mapping from limited data.
    The entire training relies on the standard deep learning assumption that the architecture and losses generalize to unseen subjects and datasets.
invented entities (2)
  • PoGz (point of gaze on the XY-plane of the camera coordinate system)
    purpose: Provides a device-independent positional gaze target to unify direction and position prediction across datasets.
    The paper defines PoGz as a new output representation (Section II-C) and claims it enables cross-dataset training, but no cross-dataset experiment is shown; it is a mathematical construct, not a measured quantity.
  • 6-DoF gaze
    purpose: Describes the complete gaze information as six degrees of freedom.
    A gaze ray has only 4 to 5 degrees of freedom; the term appears in the abstract and Section I but is not formally defined, and the paper even says 'a 3D line requires six Degrees of Freedom', which is geometrically inaccurate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MAGE: A Multi-task Architecture for Gaze Estimation with an Efficient Calibration Module." pith.science (2026). https://pith.science/paper/2QC7W6SW

@misc{pith2026250516384,
  author       = {Pith},
  title        = {Pith review of: MAGE: A Multi-task Architecture for Gaze Estimation with an Efficient Calibration Module},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2QC7W6SW}},
  note         = {Machine review of arXiv:2505.16384}
}
read the original abstract

Eye gaze can provide rich information on human psychological activities, and has garnered significant attention in the field of Human-Robot Interaction (HRI). However, existing gaze estimation methods merely predict either the gaze direction or the Point-of-Gaze (PoG) on the screen, failing to provide sufficient information for a comprehensive six Degree-of-Freedom (DoF) gaze analysis in 3D space. Moreover, the variations of eye shape and structure among individuals also impede the generalization capability of these methods. In this study, we propose MAGE, a Multi-task Architecture for Gaze Estimation with an efficient calibration module, to predict the 6-DoF gaze information that is applicable for the real-word HRI. Our basic model encodes both the directional and positional features from facial images, and predicts gaze results with dedicated information flow and multiple decoders. To reduce the impact of individual variations, we propose a novel calibration module, namely Easy-Calibration, to fine-tune the basic model with subject-specific data, which is efficient to implement without the need of a screen. Experimental results demonstrate that our method achieves state-of-the-art performance on the public MPIIFaceGaze, EYEDIAP, and our built IMRGaze datasets.

Figures

Figures reproduced from arXiv: 2505.16384 by the authors.

Figure 1
Figure 1. The overview of MAGE, our proposed multi-task network architecture for human gaze estimation. The model takes the normalized RGB image [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of Easy-Norm, involving two steps: (1) Standardize [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 1
Figure 1. The main idea of the structure is to extract the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Illustration of IMRGaze dataset. (a) General data in IMRGaze are [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 4
Figure 4. Figure 4: The pipeline of the Easy-Calibration module. Subjects gaze at the [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 6
Figure 6. Figure 6: Qualitative results. The row 1-2, 3-4, 5-6 show the subjects and the predicted gaze directions from the MPIIFaceGaze, EYEDIAP and IMRGaze [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 22 canonical work pages

  1. [6]

    Appearance-based gaze estimation with deep learning: A review and benchmark,

    Y . Cheng, H. Wang, Y . Bao, and F. Lu, “Appearance-based gaze estimation with deep learning: A review and benchmark,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    work page 2024

  2. [1]

    Medical robotics: opportunities in china,

    Y . Guo, W. Chen, J. Zhao, and G.-Z. Yang, “Medical robotics: opportunities in china,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 5, no. 1, pp. 361–383, 2022

    work page 2022

  3. [2]

    Socially assistive robots’ de- ployment in healthcare settings: a global perspective,

    L. Aymerich-Franch and I. Ferrer, “Socially assistive robots’ de- ployment in healthcare settings: a global perspective,” International Journal of Humanoid Robotics , vol. 20, no. 01, p. 2350002, 2023

    work page 2023

  4. [3]

    Human–robot interaction for rehabilitation robotics,

    Y . Guo, X. Gu, and G.-Z. Yang, “Human–robot interaction for rehabilitation robotics,” Digitalization in Healthcare: Implementing Innovation and Artificial Intelligence , pp. 269–295, 2021

    work page 2021

  5. [4]

    Engagement esti- mation of the elderly from wild multiparty human–robot interaction,

    Z. Zhang, J. Zheng, and N. Magnenat Thalmann, “Engagement esti- mation of the elderly from wild multiparty human–robot interaction,” Computer Animation and Virtual Worlds , vol. 33, no. 6, p. e2120, 2022

    work page 2022

  6. [5]

    Eye-tracking for performance evaluation and workload estimation in space telerobotic training,

    Y . Guo, D. Freer, F. Deligianni, and G.-Z. Yang, “Eye-tracking for performance evaluation and workload estimation in space telerobotic training,” IEEE Transactions on Human-Machine Systems , vol. 52, no. 1, pp. 1–11, 2021

    work page 2021

  7. [7]

    Gaze estimation from color image based on the eye model with known head pose,

    J. Li and S. Li, “Gaze estimation from color image based on the eye model with known head pose,” IEEE Transactions on Human-Machine Systems, vol. 46, no. 3, pp. 414–423, 2015

    work page 2015

  8. [8]

    Mpiigaze: Real- world dataset and deep appearance-based gaze estimation,

    X. Zhang, Y . Sugano, M. Fritz, and A. Bulling, “Mpiigaze: Real- world dataset and deep appearance-based gaze estimation,” IEEE transactions on pattern analysis and machine intelligence , vol. 41, no. 1, pp. 162–175, 2017

    work page 2017

Show all 25 references
  1. [9]

    Appearance-based gaze estimation via evaluation-guided asymmetric regression,

    Y . Cheng, F. Lu, and X. Zhang, “Appearance-based gaze estimation via evaluation-guided asymmetric regression,” in Proceedings of the European Conference on Computer Vision (ECCV) , September 2018

    2018

  2. [10]

    Recurrent cnn for 3d gaze estimation using appearance and shape cues,

    C. Palmero, J. Selva, M. A. Bagheri, and S. Escalera, “Recurrent cnn for 3d gaze estimation using appearance and shape cues,” arXiv preprint arXiv:1805.03064, 2018

    2018 arXiv

  3. [11]

    Spatio-temporal attention and gaussian processes for personalized video gaze estimation,

    S. Jindal, M. Yadav, and R. Manduchi, “Spatio-temporal attention and gaussian processes for personalized video gaze estimation,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024, pp. 604–614

    2024

  4. [12]

    Eye tracking for every- one,

    K. Krafka, A. Khosla, P. Kellnhofer, et al. , “Eye tracking for every- one,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    2016

  5. [13]

    Adaptive feature fusion network for gaze tracking in mobile tablets,

    Y . Bao, Y . Cheng, Y . Liu, and F. Lu, “Adaptive feature fusion network for gaze tracking in mobile tablets,” in 2020 25th International Conference on Pattern Recognition (ICPR) . IEEE, 2021, pp. 9936– 9943

    2020

  6. [14]

    It’s written all over your face: Full-face appearance-based gaze estimation,

    X. Zhang, Y . Sugano, M. Fritz, and A. Bulling, “It’s written all over your face: Full-face appearance-based gaze estimation,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . IEEE, Jul. 2017. [Online]. Available: http://dx.doi.org/10.1109...

    2017 doi

  7. [15]

    Training person- specific gaze estimators from user interactions with multiple devices,

    X. Zhang, M. X. Huang, Y . Sugano, and A. Bulling, “Training person- specific gaze estimators from user interactions with multiple devices,” in Proceedings of the 2018 CHI conference on human factors in computing systems, 2018, pp. 1–12

    2018

  8. [16]

    Two-eye model-based gaze esti- mation from a kinect sensor,

    X. Zhou, H. Cai, Y . Li, and H. Liu, “Two-eye model-based gaze esti- mation from a kinect sensor,” in 2017 IEEE International Conference on Robotics and Automation (ICRA) , 2017, pp. 1646–1653

    2017

  9. [17]

    A differential approach for gaze estimation,

    G. Liu, Y . Yu, K. A. F. Mora, and J.-M. Odobez, “A differential approach for gaze estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 43, no. 3, pp. 1092–1099, 2021

    2021

  10. [18]

    Revisiting data normalization for appearance-based gaze estimation,

    X. Zhang, Y . Sugano, and A. Bulling, “Revisiting data normalization for appearance-based gaze estimation,” in Proceedings of the 2018 ACM symposium on eye tracking research & applications , 2018, pp. 1–9

    2018

  11. [19]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015. [Online]. Available: https://arxiv.org/abs/ 1512.03385

    2015 arXiv

  12. [20]

    Easygaze3d: Towards effective and flexible 3d gaze estimation from a single rgb camera,

    J. Li, J. Yang, Y . Liu, et al. , “Easygaze3d: Towards effective and flexible 3d gaze estimation from a single rgb camera,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 6537–6543

    2023

  13. [21]

    Gaze estimation using transformer,

    Y . Cheng and F. Lu, “Gaze estimation using transformer,” in 2022 26th International Conference on Pattern Recognition (ICPR). IEEE, 2022, pp. 3341–3347

    2022

  14. [22]

    Agent-guided gaze estimation network by two-eye asymmetry exploration,

    Y . Shi, F. Zhang, W. Yang, G. Wang, and N. Su, “Agent-guided gaze estimation network by two-eye asymmetry exploration,” in 2024 IEEE International Conference on Image Processing (ICIP) . IEEE, 2024, pp. 2320–2326

    2024

  15. [23]

    L2cs-net: Fine-grained gaze estimation in unconstrained environments,

    A. A. Abdelrahman, T. Hempel, A. Khalifa, A. Al-Hamadi, and L. Dinges, “L2cs-net: Fine-grained gaze estimation in unconstrained environments,” in 2023 8th International Conference on Frontiers of Signal Processing (ICFSP) . IEEE, 2023, pp. 98–102

    2023

  16. [24]

    Eyediap: A database for the development and evaluation of gaze estimation algorithms from rgb and rgb-d cameras,

    K. A. Funes Mora, F. Monay, and J.-M. Odobez, “Eyediap: A database for the development and evaluation of gaze estimation algorithms from rgb and rgb-d cameras,” in Proceedings of the symposium on eye tracking research and applications , 2014, pp. 255–258

    2014

  17. [25]

    Mediapipe: A framework for per- ceiving and processing reality,

    C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. Yong, J. Lee, W.-T. Chang, W. Hua, M. Georg, and M. Grundmann, “Mediapipe: A framework for per- ceiving and processing reality,” inThird Workshop on Computer Vision for AR/VR at IEEE C...

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.