{"id":"33563e38-128b-42b6-98fa-0d362a262aa3","arxiv_id":"1908.02116","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"TS3 uses a learned teacher network to select reliable pseudo-labeled images from two student detectors, improving semi-supervised facial landmark detection on three benchmarks.","lead":"This paper introduces TS3, where a teacher network filters pseudo-labeled face images produced by two student detectors, so retraining uses only high-quality pseudo-labels. It reports state-of-the-art semi-supervised facial landmark detection on 300-W, AFLW, and 300-VW using just 10-20% of labels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's core mechanism assumes the teacher's quality model, learned on labeled data where pseudo-labels are nearly perfect and its regression target is near-constant, transfers to ranking unlabeled pseudo-labels; Section 3.4 concedes this overfitting risk, leaving the central novelty unverified.","rationale":"The reader's weakest_assumption correctly identifies the load-bearing risk, and my pass sharpens the mechanism: because the students are optimized on L, the teacher's training targets are near-constant, so the teacher's supervision signal is a narrow, nearly-degenerate distribution of (image, heatmap) to detection-loss pairs. The method's central novelty requires the learned quality function to extend to U's much wider loss range and to attribute error to the heatmap rather than to image difficulty. Section 3.4 concedes both the overfitting risk and the absence of bad pseudo-labels during training, which is the strongest possible acknowledgment that this assumption is the paper's weakest link. The paper does earn credit elsewhere: Figure 4 directly compares teacher selection against SPL and SPaCo at controlled pseudo-label counts, Table 4 isolates the contribution of the second student, and the model code is publicly released, making the mechanism empirically testable rather than merely asserted. However, Figure 4 is a single configuration with no variance estimates, and no experiment measures the teacher's ranking quality on unlabeled data, so the decisive evidence is missing. The missing 30%-label result claimed in the abstract and the absence of standard deviations are real reportability defects but are secondary and do not change the verdict. Overall, the conditional verdict stands: the method is plausible and well-ablated, but the central mechanism rests on a transfer assumption the authors themselves flag as unverified, and the paper should supply either the rank-correlation diagnostic or the random-selection ablation before the state-of-the-art claim is taken at face value.","tokens_in":13201,"tokens_out":9423,"duration_ms":91252,"concrete_test":"Take a held-out annotated set (e.g., the 300-W test set, or an annotated 200-image subset of U), treat it as unlabeled, generate ensemble pseudo-labels with the trained students, and compute the Spearman rank correlation between the teacher's quality scores and the true detection losses computed against ground-truth heatmaps. Repeat for each training round. If the correlation is near zero, or if the mean true loss of the top-ranked decile is not significantly below the bottom decile, the teacher's filtering is vacuous. As a complementary check, re-run the Figure 4 protocol with the teacher replaced by a random selector while holding pseudo-label counts, dual students, and rounds fixed; if random selection matches teacher selection, the mechanism contributes nothing beyond the ensemble and progressive scheme.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central novelty is that the teacher's quality scores filter unlabeled pseudo-labels better than unsupervised criteria such as SPL and SPaCo. For this to hold, the teacher must rank (image, heatmap) pairs on U by true detection loss. But by construction (Algorithm 1, steps 8 and 13), the teacher is trained only on L via Eq. (4), regressing q to −||H^M − H*||²_F. Because the students are themselves trained on L, their predictions on L are near-perfect, so the regression targets are concentrated near zero with very little spread; the teacher's effective training distribution contains almost no bad pseudo-labels and almost no loss variation. The mechanism then applies this teacher to U, where the true loss distribution is much wider. Two concrete failure modes follow: (i) the teacher learns a near-constant predictor, making the top-r selection essentially random; (ii) the teacher attributes loss variation to image difficulty rather than heatmap error, systematically preferring easy images regardless of pseudo-label correctness. The authors explicitly concede the premise in Section 3.4: 'the teacher's judgment could suffer from the over-fitting problem... it might only learn what a good pseudo labeled sample is, but overlook what a bad one is.' Figure 4 is the only direct evidence that teacher selection beats loss-based selection, and it is a single configuration (10% labels, 300-W, no variance bars). If the teacher's ranking on U is no better than random, the reported gains must be attributed to the dual-student ensemble and progressive retraining alone, and the paper's central claim that the teacher selects qualified pseudo-labeled samples collapses.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies semi-supervised facial landmark detection and proposes TS3, an interaction mechanism between one teacher network and two student detectors (CPM and stacked hourglass). The students generate pseudo-label heatmaps for unlabeled images; the teacher, trained on labeled images to regress the negative detection loss of student predictions, scores each pseudo-labeled sample, and the top-ranked samples are progressively added to the training set. The final prediction is an ensemble of the two students. Experiments on 300-W, AFLW, and 300-VW report improved NME/AUC over the semi-supervised baseline RCN+ and over loss-based selection strategies SPL and SPaCo, and the authors claim state-of-the-art semi-supervised performance.","tokens_in":13501,"tokens_out":4980,"duration_ms":55663,"significance":"If the central mechanism works as claimed, the paper makes a useful contribution: a learned, data-driven quality filter for pseudo-labels in a structured-prediction task, with public code and results on three standard benchmarks. The ablations partially isolate the two claimed components (teacher-based selection and dual-student interaction), and the comparison against external baselines means the empirical claims are not circular. However, the core assumption that a teacher trained on nearly correct labeled-data predictions can rank the much noisier pseudo-labels on unlabeled data is not directly verified, and the reported results lack variance information. The current evidence supports competitiveness but falls short of fully establishing the stated mechanism and the abstract's 30%-label claim.","major_comments":[{"comment":"The teacher is trained only on the labeled set L, where student predictions are close to the ground truth, so the regression target q = -||H^M_i - H*_i||^2_F is concentrated near zero and the teacher rarely sees bad pseudo-labels. Section 3.4 explicitly concedes this over-fitting risk: the teacher 'might only learn what a good pseudo labeled sample is, but overlook what a bad one is.' The central claim that teacher filtering is more reliable than loss-based selection (SPL/SPaCo) therefore rests on an unverified transfer assumption. Please provide direct evidence that teacher scores on U correlate with actual detection error, for example a rank-correlation analysis on a held-out subset or a comparison of the error of selected versus rejected pseudo-labels, and supplement Figure 4 with variance across the three repeats. Without this, the advantage over SPL/SPaCo could arise from near-random selection or from input-difficulty bias rather than from the teacher's ability to identify incorrect pseudo-labels.","section":"§3.2, Eq. (4), Algorithm 1 steps 8 and 13"},{"comment":"The abstract and introduction state that using only 30% labels the method achieves competitive results with full supervision on 300-W and AFLW, but no 30%-label experiment appears in Tables 1 or 2, which report only 100%, 20%, 10%, and 5% ratios. This is a missing support for a specific claimed result. Either add the 30% experiments or remove/qualify the claim.","section":"Abstract and Section 1"},{"comment":"The paper states that each experiment is repeated three times and the mean is reported, yet no standard deviations, confidence intervals, or per-repeat values are given. Tables 1, 2, and 4 show single numbers, and Figure 4 shows single trajectories without error bars. Because L and U are randomly sampled and the teacher and students are randomly initialized, the differences used to support the state-of-the-art claim (for example, 5.64 vs. 6.32 NME at 10% labels in Table 1) could be within run-to-run noise. Please report variance or a significance measure for the main comparisons and for the Figure 4 curves.","section":"§4.2 and Tables 1, 2, 4; Figure 4"},{"comment":"The ablation on the 'effect of the interaction between students' compares single-student CPM and HG rows with the TS3 row, but the TS3 row uses the Eq. (5) ensemble of two detectors at evaluation time, whereas the single-student rows use only one detector. This conflates the mutual-training interaction with test-time ensembling, so the reported 30% and 9% improvements could be due to ensemble averaging alone. Please add a control in which two detectors are trained independently without interaction and then ensembled by Eq. (5), or report the individual student errors inside TS3, to isolate the contribution of the student-student interaction.","section":"§4.4, Table 4"}],"minor_comments":[{"comment":"The selection ratio r and maximum step S are said to be set 'based on cross-validation,' but no details are given about the validation split or the validation criterion. Please specify what was validated and on which data.","section":"§4.2"},{"comment":"The symbol '⊿' and the wording 'update the first student' are unclear; also, because Eq. (5) uses both f1 and f2, the temporal ordering of the two students' predictions when updating a single student should be stated explicitly.","section":"Algorithm 1, lines 7 and 12"},{"comment":"The notation is inconsistent: Eq. (3) defines q_i while Eq. (4) writes q without an index, and it is not stated whether the quality scalar is computed from the last stage only or from all M stages. Please clarify.","section":"Eq. (4)"},{"comment":"RCN+ is the key semi-supervised baseline but its citation appears only in the surrounding text; adding [16] in the table caption or row would improve readability.","section":"Table 1"},{"comment":"Please clarify whether DGCM and SBR use the same 300-W training labels in addition to the 300-VW labels, and how the 300-VW unlabeled training frames are selected and processed for TS3, since the comparison depends on the exact data split.","section":"Table 3"},{"comment":"The caption's references to 'the third line' and 'the fifth line' do not match the row descriptions in the text (second row ground truth heatmaps, third row ground truth landmarks, fourth row predicted heatmaps, fifth row predicted coordinates). Please make the caption consistent.","section":"Figure 5 caption"},{"comment":"The statement that the teacher 'will fail' at 256x256 input is presented without experiments; please mark it as a conjecture or provide supporting evidence.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The main risk is whether the teacher's quality estimates transfer from the labeled set to the much noisier unlabeled pseudo-labels. This is explicitly acknowledged in Section 3.4, so the authors are aware of it, but the current paper does not supply the correlation or ablation evidence needed to resolve it. If the authors can add that evidence and variance reporting, the contribution is likely acceptable for publication. The 30%-label claim in the abstract should also be reconciled with the tables."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper proposes TS3, a teacher network that learns to score the quality of pseudo-labeled heatmaps and filters which unlabeled samples get fed back to two student detectors (CPM and HG). The teacher is trained on labeled data by regressing the negative detection loss, and students are retrained iteratively on selected pseudo-labels. The new piece is applying a learned quality judge to structured outputs in an iterative semi-supervised loop, as opposed to unsupervised loss/confidence heuristics used in SPL and SPaCo. That is a genuine novelty, and the paper is honest about its own mechanism: Section 3.4 explicitly flags the overfitting risk that the teacher may only learn what good pseudo-labels look like and miss bad ones.\n\nThe experiments are reasonably comprehensive: three benchmarks, ablations that separate the teacher's contribution from the dual-student ensemble, and a 300-VW result where TS3 beats supervised video-based methods while using no 300-VW labels. The ablation in Figure 4 does show TS3 outperforming SPL and SPaCo with the same number of pseudo-labels, which is the key evidence that the teacher is doing something better than a loss-based criterion.\n\nThe soft spots are real but not fatal. First, all numbers are means from three runs with no variance bars, so it is impossible to tell if the 0.3-0.8 NME gains are meaningful. Second, the abstract claims competitive results with 30% labels, but the tables only report 10% and 20%; the 30% result is missing. Third, the teacher's transfer to unlabeled data is the crux, and the only direct evidence is Figure 4, a single configuration with no error bars. The authors' own discussion concedes the teacher may overfit to the easy labeled set, and the stress-test is right that this could make selection near-random. That said, the paper is not hiding the issue, and the ablation does suggest the teacher helps in at least one setting. The 64x64 resolution and CycleGAN-style teacher architecture also limit the approach to low-res tasks, which the authors acknowledge.\n\nWho is this for? People working on semi-supervised facial landmark detection or other structured-output tasks with limited labels. It is a solid, honest paper, not a paradigm shift. I would cite it if I were doing follow-up work on learned pseudo-label selection, and I would not desk reject it.\n\nI recommend sending this to peer review. A good reviewer would ask for error bars, the missing 30% experiment, and a deeper look at what the teacher actually learns on unlabeled data, but those are fixable concerns, not grounds for rejection.","headline":"A solid semi-supervised landmark detection paper with a real teacher-based pseudo-label filtering mechanism, but the evidence for the teacher's transfer is thinner than the claims suggest.","tokens_in":14073,"tokens_out":1901,"would_cite":false,"duration_ms":23430,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TS3: a learned teacher filter improves semi-supervised face landmark detection by selecting only reliable pseudo labels, achieving state-of-the-art accuracy with far fewer labels.","keywords":["semi-supervised learning","facial landmark detection","pseudo labels","teacher-student networks","self-training","co-training","heatmap regression","convolutional pose machine"],"falsifier":"Train TS3 exactly as described, then on a holdout of unlabeled images with known landmarks compare the teacher's quality scores to the true detection loss of the student pseudo labels; if the correlation is near zero, or if replacing the teacher with random selection at the same ratio keeps the reported NME unchanged, the teacher filter is not doing the work.","tokens_in":12954,"feed_emoji":"👤","tokens_out":6353,"duration_ms":58270,"temperature":0.7,"pith_summary":"The paper proposes TS3, an interaction mechanism in which one teacher network scores the quality of pseudo labels produced by two student detectors and keeps only the trustworthy ones for retraining. It claims this learned filter makes semi-supervised facial landmark detection markedly better than prior self-training and co-training approaches, because selection is supervised instead of a heuristic loss. On 300-W, TS3 reports 5.64 NME with 10% labels and 5.03 with 20%, against 6.32 and 5.88 for RCN+. If the claim holds, detectors can approach fully supervised accuracy with a fraction of the landmark annotation cost, and the same teacher-filtering idea transfers to other structured prediction tasks.","feed_headline":"Teacher filter lifts face landmark detection with 10% labels","feed_subtitle":"A learned teacher scores pseudo labels, cutting 300-W NME from 6.32 to 5.64.","key_machinery":"The load-bearing mechanism is the teacher network: a CycleGAN-style discriminator that concatenates the input face image with a student's predicted heatmap and outputs a scalar quality. The teacher is trained on labeled data by L1 regression to match that scalar to the negative squared Frobenius detection loss $\\|H^M_i - H^*_i\\|_F^2$ (Eq. 4), so a higher quality value is taken to indicate a more reliable pseudo label. This learned filter replaces the heuristic unsupervised loss used by self-paced learning and self-paced co-training baselines. The second mechanism is the dual-student ensemble: two structurally distinct detectors trained on different sample subsets, whose averaged heatmaps define the pseudo label (Eq. 5) and the final prediction.","core_discovery":"The central claim is that what limits semi-supervised landmark detection is the reliability of pseudo labels, not their quantity, and that reliability can be learned. TS3 trains a teacher network on the labeled set to regress the negative detection loss of a student's heatmap prediction (Eq. 4), then uses that teacher to score every unlabeled sample and admits only the top $r \\times i \\times |U|$ samples into the student's retraining set. Two structurally different students, a convolutional pose machine and a stacked hourglass network, alternate so that each generates pseudo labels for the other, and their averaged heatmaps form both the pseudo label (Eq. 5) and the final prediction. The authors report state-of-the-art normalized mean error on 300-W (5.64 and 5.03 with 10% and 20% labels), 1.99 NME on AFLW-Full with 20% labels, and 59.65 AUC on 300-VW category C without using any 300-VW labels.","pith_inferences":["A direct test of the weakest point: retrain the teacher on a small validation set drawn from the unlabeled pool with known ground truth, as the authors suggest, and measure whether teacher quality scores correlate with true detection loss; if not, the filter is no better than random.","The reported 64x64 resolution limit suggests the teacher's quality function is local; a patch-based or multi-scale teacher might extend the mechanism to high-resolution segmentation while keeping the same training signal.","The dual-student ensemble and the teacher filter are confounded in the headline numbers; ablating each separately (e.g., teacher with a single student, ensemble without teacher) would tell which component carries the gain."],"forward_implications":["At 10% of 300-W labels, TS3 reports 5.64 NME versus 6.32 for RCN+; at 20%, 5.03 versus 5.88, so the learned filter yields lower error with fewer labels than the prior semi-supervised method.","With only 20% of AFLW-Full labels, TS3 reaches 1.99 NME, competitive with fully supervised detectors that use all labels.","On 300-VW category C, TS3 obtains 59.65 AUC without using any 300-VW labels, above DGCM (59.38) and SBR (59.39) that do use them.","In the five-round comparison against self-paced learning baselines, SPL falls into a local trap at round 4 and worsens, while TS3 keeps improving, supporting the claim that teacher-filtered data steers retraining.","Because the teacher consumes an image paired with a structured prediction, the same mechanism should apply to segmentation and pose estimation, not just facial landmarks."],"supporting_citations":[{"why":"Supplies the main semi-supervised landmark baseline RCN+ that TS3 must beat in Tables 1 and 2.","marker":"[16]"},{"why":"Defines the convolutional pose machine used as the first student detector and the heatmap loss in Eq. (2).","marker":"[41]"},{"why":"Defines the stacked hourglass network used as the second student detector and supplies p(y_i) for ideal heatmap generation.","marker":"[30]"},{"why":"Provides the CycleGAN discriminator architecture that the teacher network is built from.","marker":"[46]"},{"why":"One of the heuristic pseudo-label selection baselines (self-paced learning) compared in the round-by-round experiment of Figure 4.","marker":"[23]"},{"why":"The self-paced co-training baseline TS3 is compared against, representing heuristic unsupervised selection.","marker":"[28]"},{"why":"Prior model-communication method that selects pseudo labels by loss and score, which the teacher-based selection is designed to replace.","marker":"[12]"},{"why":"Data distillation approach whose augmented ensembling the authors cite as a complementary way to improve pseudo-label quality.","marker":"[31]"},{"why":"Fully supervised SAN baseline used for context in the 300-W and AFLW comparisons.","marker":"[9]"}],"fun_headline_variants":["Teacher grades student pseudo labels for face landmarks","Two students, one teacher: better face landmark training","Teacher filter improves face landmark detection with few labels","Reliable pseudo labels via teacher scoring for face landmarks","TS3: Teacher-scored pseudo labels for facial landmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole gain depends on the teacher, trained only on labeled images where student predictions are mostly correct, being able to recognize bad pseudo labels on unlabeled images it has never seen.","fun_headline_variants_meta":{"raw":{"variants":["Teacher grades student pseudo labels for face landmarks","Two students, one teacher: better face landmark training","Teacher filter improves face landmark detection with few labels","Reliable pseudo labels via teacher scoring for face landmarks","TS3: Teacher-scored pseudo labels for facial landmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000294,"raw_usage":{"total_tokens":1738,"prompt_tokens":999,"completion_tokens":739,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":665}},"tokens_in":615,"tokens_out":739,"duration_ms":8536,"temperature":1.0,"reasoning_tokens":665,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:53:18.677246+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train TS3 exactly as described, then on a holdout of unlabeled images with known landmarks compare the teacher's quality scores to the true detection loss of the student pseudo labels; if the correlation is near zero, or if replacing the teacher with random selection at the same ratio keeps the reported NME unchanged, the teacher filter is not doing the work.","supporting_citations":[{"cited_title":"Improving landmark localization with semi-supervised learning","cited_arxiv_id":null,"evidence_quote":"Supplies the main semi-supervised landmark baseline RCN+ that TS3 must beat in Tables 1 and 2."},{"cited_title":"Self- paced learning for latent variable models","cited_arxiv_id":null,"evidence_quote":"One of the heuristic pseudo-label selection baselines (self-paced learning) compared in the round-by-round experiment of Figure 4."},{"cited_title":"Self-paced co-training","cited_arxiv_id":null,"evidence_quote":"The self-paced co-training baseline TS3 is compared against, representing heuristic unsupervised selection."},{"cited_title":"Few-example object detection with model communi- cation","cited_arxiv_id":null,"evidence_quote":"Prior model-communication method that selects pseudo labels by loss and score, which the teacher-based selection is designed to replace."},{"cited_title":"Data distillation: Towards omni- supervised learning","cited_arxiv_id":null,"evidence_quote":"Data distillation approach whose augmented ensembling the authors cite as a complementary way to improve pseudo-label quality."},{"cited_title":"Style aggregated network for facial landmark detection","cited_arxiv_id":null,"evidence_quote":"Fully supervised SAN baseline used for context in the 300-W and AFLW comparisons."}],"review_version":1}