{"id":"d8dd703d-7b0c-4bd3-8019-c16331d0e19d","arxiv_id":"2411.13024","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"POI improves facial expression recognition by generating emotion soft labels from facial action unit priors and using uncertainty-based weighting to combat annotation ambiguity.","lead":"This paper introduces POI, a facial expression recognition system that uses knowledge of facial muscle movements to create softer, more varied emotion labels, then trains a recognition network on both the original labels and these soft labels. It reports accuracy gains on several standard face datasets and under simulated label noise.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'objective' soft labels are not independent of the subjective labels: Eq. (4) derives AU pseudolabels deterministically from the emotion label y_i via Table I, so the PIN's output inherits annotation bias; the central claim that POI recovers objective distributions is therefore untested.","rationale":"The reader's weakest assumption is on target: the fixed AU-emotion table and pseudolabel rule are the hinge of the 'objective inference' claim. I agree. The concern is not that Table I is empirically imperfect; it is that the construction makes the supposedly objective signal a function of the very label being corrected. Eq. (4) uses y_i, not image evidence or external AU annotations, to set AU targets. Therefore PIN's soft labels are not an independent measurement; they are a learned re-encoding of the subjective label through a fixed prior. This matters because the paper's contribution and framing rest on objectivity: if the soft labels are just label-derived, POI reduces to a multi-branch distillation or consistency regularizer, and the reported gains on random synthetic noise do not establish the ambiguity-resolution mechanism. The proposed FERPlus test directly checks whether the soft labels move toward the true annotator distribution; it uses a dataset already in the paper and requires no new annotation. Secondary concerns, such as test-set hyperparameter selection and missing error bars, are valid but less load-bearing: they affect the reliability of the accuracy claims, whereas the circularity concern affects the interpretation of the mechanism itself. Therefore the conditional verdict stands, with the objectivity check as a prerequisite for the strong claim.","tokens_in":21311,"tokens_out":5813,"duration_ms":62545,"concrete_test":"Use FERPlus, which provides per-image crowd vote distributions from 10 annotators as a proxy for the 'objective' expression distribution. Train POI on the majority-vote hard labels under the paper's protocol; then for each test image compare the model's soft distribution (PIN p~*_i or TRN q_i) with the crowd distribution using KL divergence, and compare it with KL(one-hot majority label || crowd distribution). If the model's soft labels are not closer to the crowd distribution than the one-hot labels are, the claim that PIN infers objective emotional distributions is unsupported, and the accuracy gains would be attributable to generic consistency or regularization rather than prior-based objective inference.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that PIN produces 'relatively objective' emotion distributions that correct annotation ambiguity. But in the paper's design, no AU annotations are used: every AU pseudolabel y_i,n,m in Eq. (4) is set to 1-epsilon or epsilon depending only on whether the AU appears in the row of Table I for the image's emotion label y_i. Thus the AU supervision, and hence the subregion predictions p_{i,n}, the intermediate prediction p~*_i in Eq. (7), and the soft labels distilled into TRN in Eq. (11), are all deterministic functions of the same subjective labels the method claims to correct. If y_i is wrong or systematically biased, the pseudolabel targets are wrong in the same direction. The synthetic-noise experiments only use random flips; random flips are partially absorbable by the epsilon smoothing and the mutual-learning loss, so robustness to them does not demonstrate objectivity. The 'w/o inpre' ablation in Table III removes the intermediate prediction but still trains on the same y_i-derived AU targets, so it cannot separate prior knowledge from label-derived regularization. The load-bearing assumption is that Table I plus Eq. (4) yields an independent signal; as written, it does not.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Prior-based Objective Inference (POI), a two-branch network for facial expression recognition (FER) that aims to mitigate annotation ambiguity. The Prior Inference Network (PIN) uses a fixed AU-expression correlation table (Table I) to generate AU pseudolabels from the emotion label (Eq. 4), learns per-subregion features, and produces an intermediate emotion distribution via gated aggregation and mutual learning (Eqs. 5--8). The Target Recognition Network (TRN) is trained with cross-entropy on the original labels and KL divergence toward the PIN's soft labels, weighted by an uncertainty estimate based on subregion prediction consistency (Eqs. 9--11). Only the TRN is used at inference. Experiments on CK+, RAF-DB, AffectNet, FERPlus, and SFEW 2.0, plus synthetic label-noise versions of three datasets, report competitive or state-of-the-art accuracy.","tokens_in":21635,"tokens_out":5766,"duration_ms":56536,"significance":"The paper's empirical scope is broad: comparisons on five public benchmarks, synthetic-noise experiments at 10--30% corruption, component ablations, and visualizations. The reported gains over prior noise-robust methods are consistent under synthetic random flips, and the commitment to release code is a strength. However, the central conceptual claim -- that POI infers 'relatively objective' emotional distributions that correct subjective annotation bias -- is undermined by the fact that the AU pseudolabels are derived deterministically from the very emotion labels the method aims to correct. If reframed as a AU-prior-regularized FER method, the engineering contribution is still interesting, but as written the paper overclaims its objectivity. The experimental methodology also lacks error bars, significance tests, and a clear separation between validation and test sets for hyperparameter selection, which tempers the strength of the benchmark claims.","major_comments":[{"comment":"The 'objective' soft labels are not independent of the subjective annotations. In Eq. (4), the AU pseudolabel \\hat{y}_{i,n,m} is set to 1-epsilon or epsilon depending only on whether the AU appears in the row of Table I for the image's emotion label y_i. Consequently, the AU supervision, the subregion emotion predictions p_{i,n} trained by Eq. (5), the intermediate prediction \\tilde{p}^*_i in Eq. (7), and the soft labels distilled into the TRN in Eq. (11) are all deterministic functions of the same y_i the method claims to correct. If y_i is wrong or systematically biased, the pseudolabel targets are wrong in the same direction. The synthetic-noise experiments only use random flips, which are partially absorbable by the epsilon smoothing and the mutual-learning KL loss, so robustness to random flips does not demonstrate objectivity. The 'w/o inpre' ablation in Table III removes the intermediate prediction but still trains on the same y_i-derived AU targets and the same CE loss on y_i, so it cannot separate the contribution of AU prior knowledge from label-derived regularization. The paper should either use genuinely independent AU supervision (e.g., an external AU detector or actual AU annotations) or substantially weaken the claim that the PIN produces 'relatively objective' distributions.","section":"III-C, Eq. (4), Table I; III-D, Eq. (11); Table III"},{"comment":"Section III-D states that the Uncertainty Estimation Module assesses the consistency of predictions across N_sub subregions 'in TRN,' but Eq. (10) defines w^au_i using \\tilde{p}_n, which are the PIN subregion soft predictions from Eq. (6), not TRN predictions. The text and the equation are therefore inconsistent. Please clarify whether the uncertainty weight is computed from PIN or TRN subregion predictions, and correct the description accordingly. This is not just a notation issue because it changes what quantity the confidence score measures.","section":"III-D, Eq. (10)"},{"comment":"The hyperparameters \\lambda_1, \\lambda_2, \\lambda_3, T, and L_sub are selected based on RAF-DB test-set accuracy in the ablation study, yet the paper then reports the resulting configuration as the default for all datasets. The text in Section IV-C says each experiment is run six times to determine the average accuracy, but no standard deviations or significance tests are reported anywhere, and the ablations do not indicate whether a held-out validation split was used. Given that some benchmark margins are small (e.g., 0.16% on RAF-DB and 0.18% on AffectNet in Tables VI and VII), test-set hyperparameter tuning without error bars makes the claimed improvements difficult to assess. Please report standard deviations, use a validation split for hyperparameter selection, and state which comparisons are statistically significant.","section":"IV-C, IV-G; Tables XI, XII; Fig. 12"}],"minor_comments":[{"comment":"Several blocks of text are duplicated in the manuscript: the paragraph beginning 'JOURNAL OF LATEX CLASS FILES...' and the full description of Section III-A appear twice, and the Fig. 2 caption is repeated. The manuscript should be cleaned up.","section":"Overall manuscript"},{"comment":"Equation (4) contains a typo: the second loss term reads '(1 − \\hat{y}_{i,n,m}) log(1 − log \\hat{p}_{i,n,m})', which should be '(1 − \\hat{y}_{i,n,m}) log(1 − \\hat{p}_{i,n,m})'.","section":"Eq. (4)"},{"comment":"In Table V, the bracket notation is inconsistent: POI* ResNet-18 is shown as '[99.55]' while POI ResNet-50 is shown as '99.89', and the text says 'the highest recognition accuracy (99.89%)' without clarifying that this comes from the ResNet-50 variant. Please explain the bracket convention and distinguish the configurations clearly.","section":"Table V"},{"comment":"The comparison with SFEW 2.0 refers to 'Table XII', but the corresponding table is labeled Table IX. Please correct the cross-reference.","section":"IV-F"},{"comment":"The learning-rate schedule is described as 'drops by 10 after 20 epochs'; this should read 'drops by a factor of 10'.","section":"IV-B"}],"recommendation":"major_revision","confidential_remarks":"This paper reports strong empirical results but the central 'objective inference' claim is conceptually problematic because the AU pseudolabels are derived from the same emotion labels the method is supposed to correct. The authors should be asked to either bring in an independent AU signal or reframe the contribution as AU-prior-regularized learning. The lack of error bars and test-set hyperparameter tuning also needs addressing. I would not reject outright because the method may still be a useful contribution once the claims are calibrated, but the current version overstates its novelty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hanwei — quick take on 2411.13024.\n\nThe engineering is real. POI packs AU-prior weak supervision, regional mutual learning, and uncertainty-weighted distillation into one network, and the empirical work is thorough: five datasets, synthetic noise, ablations, failure cases, even training times. The gains over RUL/DMUE under 10-30% random label noise are consistent, and the real-world numbers are competitive. If I were building a noisy-label FER pipeline, this architecture would be on my shortlist.\n\nThe soft spot is the central claim. The paper says the PIN infers 'relatively objective' emotion distributions, but the AU pseudolabels that supervise the PIN are read off Table I using the image's emotion label y_i. So the soft labels distilled into TRN are not independent of the subjective annotations; they inherit the same biases. The stress-test note says they are 'deterministic functions' of y_i—that's overstated, since the subregion predictions also depend on image features, so there is some signal beyond the label. But the supervision itself is label-derived, and the synthetic-noise experiments only flip labels uniformly at random, which the epsilon smoothing and mutual-learning losses can partially absorb. Nothing in the paper tests whether the method corrects systematic annotation bias. The 'w/o inpre' ablation can't separate prior from label-derived regularization, because both arms use the same AU targets. This is a load-bearing interpretation gap, not a refutation of the method: as a regularizer, it may still work, but as 'objective inference' it is untested.\n\nTwo smaller evaluation issues. Hyperparameters T, Lsub, and the lambdas are tuned on test accuracy, with no held-out validation; the text says the synthetic-noise numbers are averaged over six runs, but no standard deviations are given. And there is a real mismatch in the UEM: Section III-D says consistency is computed in TRN, but Eq. (10) uses the PIN's ~p_n. That needs fixing, as do a couple of typos in Eq. (4) and Eq. (11).\n\nBottom line: the paper deserves a serious referee and likely conditional acceptance after revisions. The experiments are honest and broad, the method is plausible, and the code is promised. But the authors should soften the objectivity language, add a held-out validation split, report variance, and ideally test on naturally noisy labels rather than only random flips.","headline":"A solid, well-ablated FER method whose 'objective inference' story is overstated, because the AU pseudolabels are derived from the very emotion labels it claims to correct.","tokens_in":22099,"tokens_out":3167,"would_cite":true,"duration_ms":31641,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-network design that infers objective emotion distributions from action-unit priors can resolve crowd-annotation ambiguity in facial expression recognition, the paper claims, reporting the highest accuracy in its comparisons on five…","keywords":["Facial expression recognition","Annotation ambiguity","Facial action units","Objective inference","Label noise robustness","Uncertainty estimation","Knowledge distillation"],"falsifier":"Permute the AU-emotion table or replace the pseudolabels with random labels while keeping everything else fixed, and retrain on RAF-DB with 30% flipped labels: if accuracy stays near the reported 85.30%, the prior is not carrying the gain, and if it falls toward the 75.12% baseline, the prior is load-bearing. A complementary check is to compare the PIN's inferred soft distribution with the human label distribution in FERPlus: the objective-inference claim stands or falls on whether the two match.","tokens_in":21145,"feed_emoji":"😊","tokens_out":10223,"duration_ms":86986,"temperature":0.7,"pith_summary":"The paper proposes POI, a two-branch network that treats the subjectivity of crowd-sourced expression labels as a missing objective signal and tries to recover that signal from facial action-unit (AU) priors. In the Prior Inference Network, four facial subregion features are supervised by fixed AU-emotion correlation pseudolabels, then pooled through a gated objective inference module into a soft emotion distribution. The Target Recognition Network is trained simultaneously on the original one-hot annotation and on that soft distribution, weighted by an uncertainty score that is high when the subregions disagree. The claim is that this reduces annotation ambiguity: POI reports accuracy of 90.51% on RAF-DB, 63.02% on AffectNet, 90.21% on FERPlus, 99.89% on CK+, and 59.86% on SFEW 2.0, and higher accuracy than SCN, DMUE, LRN, and RUL under 10% to 30% synthetic label noise on three datasets. At inference only the target network runs, so the extra machinery costs nothing at test time.","feed_headline":"AU priors beat subjective labels in expression recognition","feed_subtitle":"The two-network design hits 90.51% on RAF-DB and grows its lead as synthetic label noise rises to 30%.","key_machinery":"The load-bearing mechanism is the pair formed by the Prior Inference Network and the Uncertainty Estimation Module. The PIN splits the face into eye and mouth subregions, flips the right-side crops for symmetry, feeds them into two prior branches, and supervises each branch with AU pseudolabels generated from a fixed AU-expression correlation table, for example happiness associated with AU6 and AU12/AU26 and surprise with AU1/AU2/AU5/AU26. The Objective Inference Module temperature-smooths each subregion's emotion prediction, weights them with a learned gate, and sums them into an intermediate distribution, while a KL divergence term makes every subregion learn from this aggregate, reducing both over-reliance on uncertain priors and over-reliance on noisy labels. The Target Recognition Network's loss mixes cross-entropy on the one-hot annotation with a KL term toward the aggregate soft label, and the UEM sets the mixing weight per image from the variance of subregion predictions. The chain that carries the argument is: muscle-movement prior, then subregion AU features, then a gated soft label, then uncertainty-weighted knowledge transfer into the target network.","core_discovery":"The paper's central claim is that annotation ambiguity in facial expression recognition can be reduced by constructing a relatively objective emotion distribution from facial muscle-movement priors and then training a recognition network on both the subjective one-hot label and that inferred distribution. The Prior Inference Network (PIN) crops four facial subregions, learns action-unit features under pseudolabels derived from a fixed AU-emotion correlation table, and aggregates the subregion predictions through a gated Objective Inference Module into an intermediate soft label. The Target Recognition Network (TRN) is supervised by the original label and by this soft label, with the Uncertainty Estimation Module (UEM) scaling the soft-label term by the disagreement among subregion predictions: high disagreement means low confidence and a larger soft-label weight. The paper reports that this scheme yields the best accuracies in its comparisons on RAF-DB (90.51%), AffectNet (63.02%), FERPlus (90.21%), CK+ (99.89%), and SFEW 2.0 (59.86%), and that it degrades less than the compared noise-robust methods when 10-30% of training labels are flipped.","pith_inferences":["The same architecture pattern — prior-driven subregion inference yielding soft labels, then uncertainty-weighted transfer to a target network — is portable to other tasks where annotations are subjective and a mechanistic prior exists, such as pathology grading or content moderation.","The subregion-consistency confidence could be exposed at test time as a review flag: low-consistency faces are precisely the cases where a single emotion label is least trustworthy, and routing them to human adjudication is a direct deployment extension.","Because the soft labels are generated from a fixed action-unit/emotion table, learning or calibrating that mapping per dataset is a natural follow-up experiment that would separate the contribution of the prior itself from the contribution of the gated mutual-learning and uncertainty reweighting scheme.","Comparing the inferred soft distributions with the human label distributions available in FERPlus would directly test how objective the inferred labels are; the paper's reported accuracies do not by themselves show this agreement."],"forward_implications":["POI reports 90.51% on RAF-DB, 63.02% on AffectNet, 90.21% on FERPlus, 99.89% on CK+, and 59.86% on SFEW 2.0, the top numbers in the paper's comparisons on those benchmarks.","On RAF-DB, FERPlus, and AffectNet with 10-30% flipped labels, POI beats SCN, DMUE, LRN, and RUL; at 30% noise it leads RUL by 3.24, 1.68, and 2.85 points, respectively.","The uncertainty score is predictive of accuracy: on RAF-DB the 30% highest-confidence test images reach 99.12% while the 30% lowest-confidence reach 74.32%, a 24.8-point gap, and AffectNet shows a 21.73-point gap.","At deployment the PIN is removed, so the test-time cost is just the target network, and no AU annotations or AU detectors are needed at inference.","The objective inference module adds 1.24 to 2.67 points over directly using subregion predictions as soft labels under 10-30% synthetic noise."],"supporting_citations":[{"why":"Supplies the AU-expression correlation prior that Table I encodes and that generates the PIN pseudolabels.","marker":"[27]"},{"why":"Further source of the AU-expression correlation knowledge in Table I; the paper cites both [27] and [34] as the prior.","marker":"[34]"},{"why":"SCN is the re-labeling baseline; POI is compared against it under synthetic noise and on real-world benchmarks.","marker":"[19]"},{"why":"DMUE is the latent-distribution ambiguity baseline; POI reports higher accuracy on five datasets and under noise.","marker":"[21]"},{"why":"RUL is the relative-uncertainty baseline; the largest noise-robustness gains are reported against it.","marker":"[12]"},{"why":"RAF-DB is the main in-the-wild benchmark and the pretraining data for CK+ and SFEW fine-tuning.","marker":"[10]"},{"why":"AffectNet supplies the large in-the-wild test set used for the main accuracy comparison and confidence analysis.","marker":"[9]"},{"why":"FERPlus provides crowd-sourced label distributions; it is the natural dataset for validating ambiguity-resolving soft labels.","marker":"[8]"},{"why":"SFEW 2.0 is one of the five benchmarks used to support the generalization claim.","marker":"[51]"},{"why":"Inspires the epistemic-uncertainty formula used by the Uncertainty Estimation Module to weight soft labels.","marker":"[50]"}],"fun_headline_variants":["AU priors tame noisy labels in facial expression recognition","Objective inference from AU priors cuts FER label ambiguity","Muscle-movement priors reduce subjective label ambiguity in FER","Soft labels from AU priors beat one-hot FER annotations","Prior-based objective inference reduces facial expression ambiguity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the fixed action-unit/emotion correlation table and the smoothed AU pseudolabels derived from possibly noisy emotion labels are objective enough to yield soft labels closer to the true emotion distribution than the original one-hot annotations are; if the prior is wrong for in-the-wild faces, or the noise corrupts the pseudolabels, the soft labels simply inherit the annotation bias.","fun_headline_variants_meta":{"raw":{"variants":["AU priors tame noisy labels in facial expression recognition","Objective inference from AU priors cuts FER label ambiguity","Muscle-movement priors reduce subjective label ambiguity in FER","Soft labels from AU priors beat one-hot FER annotations","Prior-based objective inference reduces facial expression ambiguity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000806,"raw_usage":{"total_tokens":3579,"prompt_tokens":1025,"completion_tokens":2554,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":641,"completion_tokens_details":{"reasoning_tokens":2474}},"tokens_in":641,"tokens_out":2554,"duration_ms":52589,"temperature":1.0,"reasoning_tokens":2474,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:55:07.725349+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Permute the AU-emotion table or replace the pseudolabels with random labels while keeping everything else fixed, and retrain on RAF-DB with 30% flipped labels: if accuracy stays near the reported 85.30%, the prior is not carrying the gain, and if it falls toward the 75.12% baseline, the prior is load-bearing. A complementary check is to compare the PIN's inferred soft distribution with the human label distribution in FERPlus: the objective-inference claim stands or falls on whether the two match.","supporting_citations":[{"cited_title":"Exploring domain knowledge for facial expression-assisted action unit activation recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the AU-expression correlation prior that Table I encodes and that generates the PIN pseudolabels."},{"cited_title":"Dive into ambi- guity: Latent distribution mining and pairwise uncertainty estimation for facial expression recognition,","cited_arxiv_id":null,"evidence_quote":"DMUE is the latent-distribution ambiguity baseline; POI reports higher accuracy on five datasets and under noise."},{"cited_title":"Reliable crowdsourcing and deep locality- preserving learning for expression recognition in the wild,","cited_arxiv_id":null,"evidence_quote":"RAF-DB is the main in-the-wild benchmark and the pretraining data for CK+ and SFEW fine-tuning."},{"cited_title":"Training deep networks for facial expression recognition with crowd-sourced label distribution,","cited_arxiv_id":null,"evidence_quote":"FERPlus provides crowd-sourced label distributions; it is the natural dataset for validating ambiguity-resolving soft labels."},{"cited_title":"Collecting large, richly annotated facial-expression databases from movies,","cited_arxiv_id":null,"evidence_quote":"SFEW 2.0 is one of the five benchmarks used to support the generalization claim."}],"review_version":1}