{"id":"50f7f988-c4ff-4ec6-9687-149654b56885","arxiv_id":"2504.15751","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A 0.02M-parameter model that combines grouped Deep Sets with multihead attention reaches competitive head pose accuracy while being hundreds to thousands of times smaller than recent state-of-the-art models.","lead":"This paper introduces GADS, a very small neural network that estimates head pose from groups of facial landmarks, plus a hybrid variant that also reads the face image. The model is far smaller and faster than previous head pose systems, which matters for phones and low-power devices, though some of the comparison claims are overstated.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Accuracy comparison is undermined by FAN-filtered test subsets: GADS is scored only on frames where landmark detection succeeds, while Table 2 baselines are full-dataset numbers from LwPosr.","rationale":"I read the central claim as the lightest-fastest-competitive triad. Parameter counts (0.02M vs 0.15M LwPosr) and raw latency (Table 4) are internally consistent and plausible. The vulnerability is the accuracy comparison. The reader identified unverified baseline comparability; I sharpen this to a demonstrable mismatch: GADS's test sets are FAN-surviving subsets, while copied baseline numbers are full-set. Table 1 quantifies the filtering. Since high-angle and occluded frames are where pose error is largest, subset scoring likely improves GADS's numbers. This does not require assuming bad faith; it is visible from the manuscript's own tables. The false \"top-3\" claim in Section 5.1.2 is a separate red flag that supports caution but is not the main attack. The size and speed results are plausible and should be credited. The conditional verdict remains appropriate: the authors should report filtered-subset re-evaluation and full-set GADS evaluation before the competitive-accuracy claim is accepted.","tokens_in":18840,"tokens_out":6171,"duration_ms":53987,"concrete_test":"Re-run one lightweight baseline checkpoint (preferably LwPosr, 0.15M params, whose numbers were copied) on the exact 1,869 AFLW2000 and 14,954 BIWI images for which FAN succeeded, and compute per-angle MAE on that subset. Compare with GADS's reported values (AFLW2000 MAE 5.30, BIWI MAE 3.90). If baseline subset MAE drops to GADS's level, the competitive-accuracy conclusion fails. As a complementary check, obtain landmarks for the 131 missing AFLW2000 images with an alternative 68-point detector and evaluate GADS on all 2,000 images; if GADS full-set MAE rises materially, the reported accuracy is an artifact of test-set filtering.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is \"competitive accuracy with SOTA at 20K parameters.\" The architecture and parameter counts support the size claim, but the accuracy claim depends on an apples-to-oranges comparison. Section 4 states that all inputs require MTCNN face detection plus FAN landmark extraction, and Table 1 reports that FAN loses 4.62% of BIWI frames, 13.86% of 300W-LP frames, and 6.55% of AFLW2000 frames (131 of 2,000, leaving 1,869). GADS's Table 2 MAE values are therefore computed on these surviving subsets only. The SOTA values in Tables 2 and 3 are copied from LwPosr (caption of Table 2) and are not re-run; they are conventionally reported on the full test sets. The missing frames are explicitly the hard cases: Section 5.5 says landmark detection fails on occlusions, warped images, and extremely high angles, and HPE error is largest at high angles. If the baselines were evaluated on the same 1,869/14,954-image subsets, their MAE would likely improve, narrowing or eliminating GADS's claimed competitive margin. This is a concrete comparability defect, not just a hypothetical difference in training recipes; it is visible from the paper's own Table 1.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces GADS, a landmark-only head-pose estimator with about 20K parameters, together with a hybrid variant (GADS-Hybrid) that additionally consumes a 64x64 RGB crop. The vanilla model applies five parallel Deep Set layers to region-wise groups of 3D facial landmarks, passes the group embeddings through a four-head self-attention layer, and regresses yaw, pitch, and roll with a small MLP. The authors train on 300W-LP and on a 70% split of BIWI, evaluate under two protocols on AFLW2000 and BIWI, report MAE values, an ablation over architecture choices, and CPU/GPU execution times. The central claims are that GADS is the smallest published HPE model (about 7.5x smaller than LwPosr and 4321x smaller than TokenHPE), executes tens of times faster than lightweight CNN baselines, and achieves accuracy competitive with state-of-the-art methods.","tokens_in":19033,"tokens_out":21098,"duration_ms":166384,"significance":"The strongest assets of the paper are its genuinely tiny architecture and the clarity of the size and single-forward-pass timing measurements: a 20K-parameter regressor with sub-3ms per-forward latency is a plausible deployable component for landmark-based pipelines, and the Deep Set plus grouped-attention design is simple and extensible. The ablation study is broad, and the authors are transparent about the MTCNN/FAN dependency. If the accuracy comparison were re-run on matched evaluation conditions, the 'competitive with SOTA' claim would be a useful data point for the HPE community. As it stands, however, the evidence for that claim is not yet reliable: the SOTA numbers in Tables 2-3 are copied from a single prior study, GADS is scored on FAN-filtered easy subsets rather than the full test sets, and hyperparameters were selected using the test partitions themselves. The paper provides no code and re-runs no baselines, so the quantitative claims are not independently verifiable at this stage.","major_comments":[{"comment":"The headline accuracy comparison is not apples-to-apples. Table 1 shows that GADS is evaluated only on frames where FAN landmark detection succeeded (BIWI 14,954/15,678; 300W-LP 105,484/122,450; AFLW2000 1,869/2,000), while the SOTA values in Tables 2 and 3 are copied from the LwPosr study (per the captions) and correspond to the full test sets. Section 5.5 states that FAN fails on occlusions, warped images, and extreme angles, and that pose error is largest at high angles. The comparison therefore omits exactly the hard cases for the competing methods while evaluating GADS only on the easy cases, which biases the comparison in GADS's favor. This is a concrete comparability defect, not merely a difference in training recipes. The authors must re-run or re-evaluate all baselines on the identical surviving frame subsets, or evaluate GADS on the full test sets with a fallback mechanism, and report both sets of numbers.","section":"Section 3.3, Table 1, Tables 2-3"},{"comment":"The model-selection procedure leaks the test sets into the reported numbers. The ablation study is conducted under Protocol 1, i.e., with BIWI and AFLW2000 as evaluation sets, and Section 4.0.3 states that all architecture hyperparameters were determined from this ablation. The configuration so selected is then the one whose MAE values are reported in Tables 2-3. Since no separate validation split is described, the headline accuracies are the result of selection on the test partitions themselves and are optimistically biased. Please redo the hyperparameter selection on a held-out validation split (for example, a subset of the training data) and report the final test-set numbers for the single configuration chosen without test-set feedback.","section":"Section 5.4, Table 5, Section 4.0.3"},{"comment":"Several textual ranking claims are contradicted by the paper's own tables. In Protocol 2, GADS-Hybrid has MAE 3.46 and ranks 6th behind TokenHPE (2.49), 6DRepNet (2.66), TriNet (2.80), Martin (2.90), and VGG16-RNN (3.07), so the claim that it 'secures a position within the top 3 among SOTA models' is unsupported. In Protocol 1 on AFLW2000, GADS's pitch (7.06) ranks 13th of 18 and its roll (5.00) ranks 10th of 18, contradicting the statement that GADS 'ranks within the top 5' in pitch and roll. The stated roll margin of 0.18 in Section 5.1.1 also does not match Table 2 (GADS 3.04 versus TokenHPE 2.71 is a difference of 0.33). All ranking and margin statements must be recomputed from the tables and corrected.","section":"Section 5.1.2, Table 3; Section 5.1.1, Table 2"},{"comment":"The speed claims are misattributed and measure only the regressor, not the full pipeline. The abstract says GADS is '25x faster than the current lightest state-of-the-art model', which the paper identifies as LwPosr (Section 2.1), but Table 4 reports execution times only for HopeNet and FSA-Net; no LwPosr timing is given, so that specific claim is unsubstantiated. Section 5.5 says GADS exhibits a processing speed '2x faster than FSANET', which contradicts Table 4 (about 44x faster on CPU and 39x faster on GPU). In addition, the GADS timing of 2.04 ms is for the model alone ('for a given set of landmarks', Section 5.2), whereas HopeNet and FSA-Net are end-to-end models; the MTCNN face detector and FAN landmark extractor add latency, as Section 5.5 itself acknowledges. Please report end-to-end pipeline latency for all methods and correct the inconsistent speedup factors.","section":"Abstract, Section 5.2, Table 4, Section 5.5"},{"comment":"The hybrid model's stated purpose is robustness ('To make GADS robust', Section 3.2), but no experiment evaluates the model on the failure cases identified in Section 5.5 (occlusions, warped images, extreme angles). Those cases are precisely the frames excluded from Tables 2-3 by FAN failures, so the paper never demonstrates that GADS-Hybrid recovers accuracy where vanilla GADS fails. An evaluation on the excluded frames or on a hard-angle subset would directly test the robustness motivation and should be added.","section":"Section 3.2, Section 5.5"}],"minor_comments":[{"comment":"The subsection 'Implementation Details' is empty; the relevant content appears in Section 4.0.3. The empty subsection should be removed or filled.","section":"Section 3.6"},{"comment":"The AFLW2000 row is misformatted as 'AFLW 2000 1869 6.55', which can be misread as a dataset name followed by only two numbers; please write 'AFLW2000 2000 1869 6.55' so the Before, After, and Loss columns are unambiguous.","section":"Table 1"},{"comment":"The Protocol P2 paragraph ends with the stray words 'improve this', which appears to be a leftover editing note and should be removed.","section":"Section 3.4"},{"comment":"The model name 'LowPosr' is a typo for 'LwPosr'; please correct it for consistency with Tables 2-3 and the references.","section":"Section 5.1.2"},{"comment":"The first reference entry is incomplete: it reads 'Self-Attention with Relative Position Representations' with no author names and an in-text citation key '(sha, 2018)' that is not a standard author-year key. Please complete the entry and fix the citation.","section":"Section 2.3 and References"},{"comment":"The captions read 'AFLW200' instead of 'AFLW2000', and the text contains frequent typographical errors such as 'it's' for 'its' and 'di fferent' for 'different'; a careful proofreading pass is needed.","section":"Figures 7 and 8"},{"comment":"Since GADS is presented as an improvement over the authors' prior DS-HPE model, including DS-HPE's MAE, parameter count, and execution time in Tables 2-4 would substantiate the claimed improvement and better situate the contribution; this baseline is currently absent from all comparison tables.","section":"Introduction and Tables 2-3"}],"recommendation":"major_revision","confidential_remarks":"The paper makes a modest but real architectural contribution (a 20K-parameter landmark-set regressor), and the size and per-forward-pass latency numbers are credible. My main concerns, which agree with the stress-test analysis, are that the accuracy claims rest on FAN-filtered test subsets, on SOTA values copied from a single prior study without re-running, and on hyperparameters selected using the test partitions; all three favor GADS, and several textual claims are also contradicted by the paper's own tables. These issues are fixable in revision, but they require the authors to redo the evaluation control and release per-frame results rather than reword the text. I would ask the editor to require the exact frame indices used for evaluation and the per-frame error files, since no baseline was re-run and no code is currently available."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper has a real efficiency result and the architecture is simple and sensible, but the accuracy comparison is not apples-to-apples, and the paper's own Table 1 shows it. GADS is a 20K-parameter landmark-only head pose regressor built from grouped Deep Set layers plus multihead attention. That combination is new for HPE as far as I know, and the parameter count and the measured 2-4 ms inference are credible. If you need a tiny landmark regressor, this is a reasonable starting point.\n\nThe problem is the headline claim of 'competitive accuracy with SOTA'. Section 4 says all inputs go through MTCNN then FAN, and Table 1 reports FAN drops 4.62% of BIWI, 13.86% of 300W-LP, and 6.55% of AFLW2000 (2000 to 1869). The MAE values in Tables 2 and 3 are computed on the surviving frames only, for the simple reason that GADS needs landmarks. The SOTA numbers are copied from LwPosr (Dhingra, 2022) and are full-dataset numbers. The dropped frames are the hard ones: Section 5.5 says landmark detection fails on occlusions, warped images, and extreme angles, and the figures show error grows with angle. So the comparison is biased in GADS's favor. This is not a training-recipe nitpick; it is visible from the paper's own tables. To fix it, the authors need to evaluate baselines on the same filtered subsets, or evaluate GADS on the full sets with fallback landmarks, and report both.\n\nOther issues are smaller. The 'top 3' claim for GADS-Hybrid under Protocol 2 is contradicted by Table 3; by MAE it is somewhere between 5th and 6th. Hyperparameters were selected using the ablation study on the test sets, which is selection on the test data. There are no error bars, no seeds, and no code. The execution time table only reports the regressor, which is fine if labeled as such, but the abstract's '25x faster' refers to the regressor only, not the full pipeline.\n\nNone of this kills the paper. The efficiency claim is real and the architecture is reasonable. But the accuracy claim as stated is not supported. I would send this to peer review, with a clear request to fix the evaluation protocol, rerun or clearly bound the baselines, and correct the overstatements. After that, it could be a useful baseline for edge HPE.","headline":"A plausible 20K-parameter HPE regressor whose efficiency claims hold up, but whose 'competitive accuracy' is not established due to evaluation on FAN-filtered test subsets versus full-set baselines.","tokens_in":19642,"tokens_out":3341,"would_cite":false,"duration_ms":27810,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 20,000-parameter landmark-only model matches head-pose accuracy of networks thousands of times larger.","keywords":["head pose estimation","facial landmarks","Deep Sets","multihead attention","lightweight model","edge computing","permutation invariance","geometric deep learning"],"falsifier":"Run LwPosr, FSA-Net, WHENet, EVA-GCN, and TokenHPE under the exact Protocol 1 pipeline used for GADS, with the same face crops, landmark detector, and test splits, and compare mean absolute errors; if the re-measured baselines are more than about 1.5 degrees better than the borrowed numbers, the competitive-accuracy claim is unsupported. Separately, measure end-to-end CPU latency including face detection and landmark extraction to test the deployment-speed claim.","tokens_in":18581,"feed_emoji":"🎯","tokens_out":12832,"duration_ms":100741,"temperature":0.7,"pith_summary":"This paper proposes GADS, a head-pose estimator built from facial landmarks that is much smaller and faster than existing models while staying close to their accuracy. The intended contribution is a deployable baseline for resource-constrained settings: about 20,000 parameters, a reported CPU latency near 2 ms for the landmark-only variant, and mean absolute errors within roughly 0.2 to 1.4 degrees of the best values in the compared tables. The model groups 27 facial landmarks into five regions, reduces each group with a permutation-invariant Deep Set layer, and uses multihead attention to combine the group summaries before regressing yaw, pitch, and roll. A hybrid variant that also consumes the RGB image is evaluated as well. If the claims hold, GADS is the lightest head-pose regression stage reported to date.","feed_headline":"20K-parameter model rivals head-pose networks 4,000x larger","feed_subtitle":"Grouping facial landmarks with attention keeps GADS within a degree or two of far larger models, for edge devices.","key_machinery":"The load-bearing object is the Grouped Attention Deep Set block. A Deep Set layer is a permutation-invariant encoder-aggregator-decoder that maps an unordered set of landmark coordinates to a fixed vector, so the first stage of the model does not depend on landmark ordering. GADS runs five such layers in parallel, one per facial region, aggregates each group with a max operation, and then applies multihead self-attention to the five group summaries so the final regressor sees both within-region geometry and cross-region relationships. This reduces the input from 67x3 coordinates to five 32-dimensional vectors before the final decoder, which is what keeps the parameter count near 20,000.","core_discovery":"On its own terms, the paper's central discovery is that a pose regressor can be reduced to a twenty-thousand-parameter landmark-only network without leaving the accuracy band of state-of-the-art models. GADS partitions 68 FAN landmarks (27 used) into left eye, right eye, left cheek, right cheek, and chin; each group is processed by a dedicated Deep Set encoder-decoder with a max invariant operator, yielding a fixed-size summary per region. Multihead attention with four heads then mixes these summaries, and a two-layer decoder regresses yaw, pitch, and roll. Trained on 300W-LP with MAE loss, GADS reports a BIWI MAE of 3.90 and an AFLW2000 MAE of 5.30 under Protocol 1, including the best yaw error on BIWI at 3.61, while using 0.02 million parameters; the authors state this is 7.5x smaller than LwPosr and 4321x smaller than TokenHPE, with measured CPU latency of 2.04 ms for the landmark-only model. The hybrid variant, adding a small CNN on the 64x64 RGB crop, reports a BIWI Protocol 2 MAE of 3.46, outperforming vanilla GADS on that split.","pith_inferences":["The semantic grouping into eyes, cheeks, and chin may itself be a large part of the inductive bias; a controlled experiment with landmarks randomly permuted into five arbitrary groups would reveal whether the region grouping or merely the set partitioning drives accuracy.","The reported 2 ms latency covers only the pose regressor, not face detection or landmark extraction; an end-to-end measurement that includes the upstream detector would show what share of the real-time budget remains on a given device.","If the same 20K-parameter budget transfers, the grouped-set design should produce comparable accuracy on other landmark regression problems such as hand pose or gaze estimation, which would be a direct test of the paper's generality claim.","Re-running the baseline models under the exact same preprocessing pipeline would refine the competitive-accuracy claim, since the comparison currently depends on numbers taken from earlier studies."],"forward_implications":["Because GADS is landmark-only and adds about 2 ms and 20,000 parameters, it can be inserted as a near-free head-pose stage into systems that already compute facial landmarks for other tasks.","The 0.02-million-parameter size makes the model a candidate for edge and compute-poor deployments where RGB models in the hundreds of thousands or millions of parameters are impractical.","On the cross-dataset protocol, GADS reports the best yaw error on BIWI among the compared methods, with pitch and roll slightly behind, suggesting the geometric grouping captures orientation information unevenly across angles.","The same grouped Deep Set plus attention design is presented as extensible to any landmark-based analysis, not only head pose estimation."],"supporting_citations":[{"why":"Defines the Deep Set permutation-invariant architecture that GADS instantiates as one layer per landmark group.","marker":"(Zaheer et al., 2017)"},{"why":"Supplies the scaled dot-product multihead self-attention used in Block 2 to combine the five group summaries.","marker":"(Vaswani et al., 2017)"},{"why":"Provides LwPosr, the lightest prior SOTA model, and the published MAE and parameter values used as the comparison baseline in Tables 2 and 3.","marker":"(Dhingra, 2022)"},{"why":"Provides the FAN detector whose 68 3D landmarks are the model's input modality.","marker":"(Bulat and Tzimiropoulos, 2017)"},{"why":"Contributed the 300W-LP training set and the AFLW2000 evaluation set used in Protocol 1.","marker":"(Zhu et al., 2016)"},{"why":"Provides the BIWI dataset used for both the cross-dataset evaluation and the 70/30 train-test protocol.","marker":"(Fanelli et al., 2013)"},{"why":"Earlier DS-HPE work that contributes the preprocessing pipeline and the Deep Set baseline GADS extends with grouping and attention.","marker":"(Menan et al., 2023)"},{"why":"TokenHPE is the recent SOTA model whose parameter count anchors the 4321x size-ratio claim.","marker":"(Zhang et al., 2023)"}],"fun_headline_variants":["GADS: 20K-parameter head pose model rivals 4000x-larger nets","Super lightweight GADS: 20K params, 25x faster, near-SOTA accuracy","Grouped attention deep sets cut head pose model to 20K parameters","Head pose with 20K weights: GADS is 4321x smaller, stays accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central comparison numbers for competing methods are taken from a previous study's tables rather than re-measured, so the claim that a 20,000-parameter model is competitive depends on those external mean absolute errors having been produced under the same training and evaluation conditions as GADS.","fun_headline_variants_meta":{"raw":{"variants":["GADS: 20K-parameter head pose model rivals 4000x-larger nets","Super lightweight GADS: 20K params, 25x faster, near-SOTA accuracy","Grouped attention deep sets cut head pose model to 20K parameters","Head pose with 20K weights: GADS is 4321x smaller, stays accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000456,"raw_usage":{"total_tokens":2317,"prompt_tokens":1004,"completion_tokens":1313,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":1219}},"tokens_in":620,"tokens_out":1313,"duration_ms":9716,"temperature":1.0,"reasoning_tokens":1219,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:18:34.840554+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run LwPosr, FSA-Net, WHENet, EVA-GCN, and TokenHPE under the exact Protocol 1 pipeline used for GADS, with the same face crops, landmark detector, and test splits, and compare mean absolute errors; if the re-measured baselines are more than about 1.5 degrees better than the borrowed numbers, the competitive-accuracy claim is unsupported. Separately, measure end-to-end CPU latency including face detection and landmark extraction to test the deployment-speed claim.","supporting_citations":[{"cited_title":", author Lei, Z","cited_arxiv_id":null,"evidence_quote":"Contributed the 300W-LP training set and the AFLW2000 evaluation set used in Protocol 1."},{"cited_title":", author Dantone, M","cited_arxiv_id":null,"evidence_quote":"Provides the BIWI dataset used for both the cross-dataset evaluation and the 70/30 train-test protocol."},{"cited_title":", author Liu, H","cited_arxiv_id":null,"evidence_quote":"TokenHPE is the recent SOTA model whose parameter count anchors the 4321x size-ratio claim."}],"review_version":1}