{"id":"6c0c7ff5-66cf-49c3-b7e2-647e6a9cb25f","arxiv_id":"2506.14511","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MOL jointly trains micro-expression recognition with optical flow estimation and landmark detection via the F5C block, reporting improved accuracy on CASME II, SAMM, and SMIC.","lead":"This paper proposes MOL, a deep learning framework that jointly trains micro-expression recognition, optical flow estimation, and facial landmark detection from raw video frames. The authors report accuracy gains over existing methods on three public micro-expression datasets using a new feature block called F5C.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim is confounded by unablated Aff-Wild2 pretraining: no from-scratch or matched-pretraining baseline isolates whether F5C and joint auxiliary losses produce the reported gains.","rationale":"The paper describes a coherent architecture and reports useful ablations. I do not see an internal inconsistency in the F5C mechanism, and the auxiliary-task ablation in Table 8 gives genuine evidence that the two pseudo-labeled tasks help on CASME II, conditional on the chosen pretraining. However, the strongest empirical claim, namely SOTA MER without key frames or pre-extracted features, is not cleanly established because every result is preceded by Aff-Wild2 pretraining and no experiment removes or matches it. This is the single most load-bearing concern because it directly affects the external validity of Tables 4-7, whereas the reader's pseudo-label concern, while plausible, is partly answered by the fact that the auxiliary losses improve performance in the internal ablation; without evidence of Dlib/TV-L1 failure on ME frames it remains a secondary risk. The missing control is easy to specify and should be required before the SOTA claim is accepted. Since the suggested remedy is an additional experiment rather than a conceptual flaw, the appropriate verdict remains conditional; my read therefore does not change the reader's verdict.","tokens_in":21919,"tokens_out":5571,"duration_ms":58470,"concrete_test":"Retrain MOL and its two decisive variants (MOL w/o OFE&FLD and MOL w/o F5C) from random initialization on CASME II, SAMM, and SMIC using the exact hyperparameters of Sec. 4.1.3, and, as a matched control, give the closest competing methods (at least AU-GACN, MicroNet, and GACNN) the same Aff-Wild2 pretraining before evaluation. If the from-scratch MOL loses the SOTA margin, or if Aff-Wild2-pretrained baselines match or exceed MOL, the reported advantage is attributable to pretraining rather than to the proposed F5C/joint-learning design.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Every reported MOL result in Tables 4-7 uses the protocol stated in Sec. 4.1.3: \"Before training on ME datasets, we pre-train MOL on a popular in-the-wild macro-expression dataset Aff-Wild2 [59], [60].\" This pretraining stage is never ablated. Consequently, the central claim that the F5C block and joint optical-flow/landmark learning yield superior MER is underdetermined: the margin over prior methods could come entirely from Aff-Wild2 transfer, including its expression labels and the same Dlib/TV-L1 pseudo-supervision used to pre-train the auxiliary heads. The internal ablations in Table 8 compare MOL with variants that are all plausibly pretrained under the same protocol; they show the auxiliary tasks help conditional on Aff-Wild2 pretraining, but they do not quantify the contribution of pretraining itself. The problem is compounded by Table 4, where MERSiamC3D already exceeds MOL on 5-class CASME II (81.89 vs 79.23 Acc), so the abstract's \"outperforms the state-of-the-art\" wording is not supported even before the pretraining confound is addressed. This is a missing control, not an internal inconsistency; the architecture may be sound, but the empirical support for the headline claim is incomplete.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes MOL, an end-to-end framework for micro-expression recognition (MER) that jointly learns optical flow estimation and facial landmark detection as auxiliary tasks. The architecture uses a novel F5C block combining fully-connected convolution (FCC) and channel correspondence convolution (CCC), operating on raw frame sequences without key-frame annotations. Experiments on CASME II, SAMM, and SMIC show improvements over several prior MER methods in most settings, and ablations demonstrate the contribution of each proposed component and auxiliary task. The paper also reports competitive performance for optical flow and landmark prediction.","tokens_in":22186,"tokens_out":5025,"duration_ms":50269,"significance":"The work is a well-executed empirical study of a novel multi-task architecture for MER. Its strengths include a clean ablation design (Tables 8–12) that consistently shows the F5C block, FCC, CCC, and auxiliary tasks each contribute to accuracy; a clear end-to-end pipeline that does not require hand-crafted features or key frames; and public code release. The main claims are falsifiable through the reported tables. However, the unablated Aff-Wild2 pretraining and the use of pseudo-labels as ground truth for auxiliary-task evaluation leave some empirical questions unresolved, so the paper's significance depends on the authors' ability to address these points in revision.","major_comments":[{"comment":"The abstract and conclusion claim that MOL 'outperforms the state-of-the-art MER methods on CASME II, SAMM, and SMIC benchmarks,' but this wording is not supported by the reported numbers. In Table 4, MERSiamC3D achieves 81.89 Acc versus MOL's 79.23 on the five-class CASME II setting, and in Table 7 AU-GACN achieves 45.10 Acc versus MOL's 44.58 on SAMM→SMIC. Sec. 4.2.1 also concedes that GACNN, MERSiamC3D, and I2Transformer outperform MOL in a few cases. The claim should be qualified to state that MOL achieves state-of-the-art or competitive results on most settings, or the comparison should be restricted to the settings where MOL is strictly best.","section":"Abstract, Sec. 4.2.1, Tables 4 and 7"},{"comment":"Every reported experiment pre-trains MOL on Aff-Wild2, but the paper never ablates the pretraining stage. The internal ablations in Tables 8–11 show that the F5C block and auxiliary tasks improve accuracy conditional on this pretraining, but they do not quantify how much of the margin over prior methods comes from Aff-Wild2 transfer (including its Dlib/TV-L1 pseudo-supervision). Please add an experiment that trains MOL from scratch or with a matched pretraining protocol to isolate the contribution of the proposed architecture and joint losses; otherwise the headline SOTA claim is underdetermined by the current evidence.","section":"Sec. 4.1.3"},{"comment":"The optical flow and landmark evaluations use TV-L1 and Dlib outputs as ground truth, and MOL is trained to predict exactly these pseudo-labels. The comparisons against RAFT, UnsupFlownet, TCDCN, and HRNetV2 are therefore not apples-to-apples unless those baselines are fine-tuned on the same data and evaluated against the same pseudo-ground-truth. The numbers in Tables 14–15 should be framed as 'agreement with TV-L1/Dlib' rather than as general optical-flow/landmark accuracy, or the evaluation should be repeated on standard benchmarks with human-annotated ground truth.","section":"Sec. 4.1.1, Tables 14–15"}],"minor_comments":[{"comment":"The dataset name is misspelled as 'CAMSE II'; it should be 'CASME II'.","section":"Table 4 header"},{"comment":"The proposed block is referred to as 'FC5' instead of 'F5C' in the phrase 'one FC5 block is already sufficient'; please correct this typo.","section":"Sec. 4.3.2"},{"comment":"The statement that MOL is 'the only one DL based method with raw frame images as input' in the composite evaluation is misleading, since MicroNet, FRL-DGT, and SelfME also take raw images as input; the intended distinction appears to be that those methods require key frames. Please clarify the wording.","section":"Sec. 4.2.2"},{"comment":"The paper does not report the computational cost or parameter count of MOL; a brief efficiency comparison with prior methods would be useful, especially since the proposed FCC and CCC add global operations over the full feature map.","section":"Sec. 4.4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this paper has a genuinely new block and a sensible joint-training idea, and the internal ablations back it up. But the abstract's SOTA sentence is not supported by the paper's own tables, and the Aff-Wild2 pretraining is never ablated, so the real-world gains over prior work are not cleanly established.\n\nThe new thing is the F5C block: fully-connected convolution plus channel-correspondence convolution, meant to get local-global features without key frames. That's a reasonable combination, and the paper's ablations (Tables 8-11) show each component matters—removing FCC, CCC, or either auxiliary task drops accuracy on CASME II. The joint training of MER with optical flow and landmark detection as auxiliary tasks is also a sensible way to use motion and structure cues on small datasets. The paper releases code and gives enough detail to reproduce.\n\nThe soft spots, in order of importance.\n\nFirst, the headline. The abstract says MOL \"outperforms the state-of-the-art\" on all three benchmarks. Table 4 shows MERSiamC3D gets 81.89% Acc on 5-class CASME II while MOL gets 79.23%, and GACNN gets 88.72% on 3-class SAMM while MOL gets 88.36%. The paper itself concedes in Sec 4.2.1 that several methods beat it \"in a few cases.\" So the claim is just overbroad; it should be \"competitive or best on most settings\" not \"outperforms SOTA.\"\n\nSecond, the Aff-Wild2 pretraining confound. Every result in Tables 4-7 is from a model pretrained on Aff-Wild2 (Sec 4.1.3). There's no from-scratch run and no comparison with, say, a baseline that uses the same pretraining but no F5C block. The ablations in Tables 8-11 all use the same pretraining, so they show the components help conditional on pretraining, not that they cause the margin over prior methods. That's a missing control, not an internal contradiction—the architecture may be fine—but it needs to be run and reported.\n\nThird, the auxiliary supervision is pseudo-label based: Dlib for landmarks, TV-L1 for optical flow. That's a real limitation, not a killer. It would help to mention it explicitly and maybe show a sensitivity check if cleaner labels are available.\n\nMinor: no variance or standard errors across LOSO folds. On datasets with roughly 160-250 videos, a couple of percent could easily be noise. The cross-dataset comparison also includes only a handful of methods, so the generalization claim rests on a thin baseline set.\n\nBottom line: the paper is worth engaging with. The architecture is plausible, the ablation study is well designed, and the code availability is a plus. It needs a serious referee, but the authors should be asked to add an ablation without Aff-Wild2, temper the abstract, and report variance or significance. I'd bring it to a reading group interested in fine-grained action recognition.\n\nRecommendation: send to peer review, but expect major revisions.","headline":"A useful architecture with solid ablations, but the SOTA claim is overbroad and the Aff-Wild2 pretraining confound means the reported gains over prior work aren't cleanly attributed.","tokens_in":22697,"tokens_out":4003,"would_cite":true,"duration_ms":36702,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Jointly training micro-expression recognition with optical flow and landmark detection lets one network read subtle facial actions from raw video, beating prior methods without key frames or hand-crafted features.","keywords":["micro-expression recognition","multi-task learning","optical flow estimation","facial landmark detection","transformer-style convolution","graph convolution","F5C block","raw video end-to-end"],"falsifier":"Retrain MOL on the same benchmarks but replace the pseudo-label generators with human-annotated landmarks and optical flow, or with a different landmark and flow model; if the accuracy gap over the no-auxiliary-task variant shrinks or vanishes, the reported gains depend on the specific pseudo-label tools rather than on multi-task learning per se.","tokens_in":21734,"feed_emoji":"👁","tokens_out":5149,"duration_ms":43514,"temperature":0.7,"pith_summary":"The paper tries to show that facial micro-expression recognition can be solved end-to-end from raw frame sequences, without needing key frames or pre-extracted features, by training the recognition task together with two auxiliary tasks: optical flow estimation and facial landmark detection. The claim is that these auxiliary tasks teach the shared feature extractor about the subtle muscle movements that define micro-expressions, and that this shared learning compensates for the small, low-diversity training datasets in the field. On the three standard benchmarks (CASME II, SAMM, SMIC) the resulting framework, MOL, reports the best or second-best accuracy in most single-dataset, composite, and cross-dataset evaluations, with the best three-class accuracy on CASME II (91.26) and SMIC (80.71). A sympathetic reader would care because the method removes two practical bottlenecks of prior work, and the ablation study attributes the gains to the joint learning and the new feature block rather than to extra supervision.","feed_headline":"One network reads micro-expressions, motion, and landmarks together","feed_subtitle":"Joint training on raw frames outperforms earlier methods without key frames or hand-crafted features.","key_machinery":"The carrying mechanism is the F5C block, a plug-and-play feature extractor made of two operations. Fully-connected convolution (FCC) treats each row and each column of the feature map as a patch, adds positional embeddings, and applies circular convolution along both directions, so each output position sees the whole row or column, giving local detail with a global receptive field in the style of a transformer. Channel correspondence convolution (CCC) builds a k-nearest-neighbor graph whose vertices are feature-map channels, computes edge features from the global channel vector and the difference to each neighbor, and aggregates them with max pooling to model correlations among feature patterns. The F5C features are shared by the MER classifier, a FlowNet-style optical-flow decoder, and a landmark regressor; consecutive-frame feature concatenations are fed to a 3D CNN for the final classification.","core_discovery":"MOL's central claim is that multi-task learning with automatically generated motion and shape supervision is enough to make a deep network attend to micro-expression cues that are barely visible in raw pixels. The framework processes a clip of raw frames, extracts a shared local-global feature for each frame with the F5C block, and from those features simultaneously predicts the micro-expression category, the optical flow between consecutive frames, and the facial landmarks of each frame. On CASME II, SAMM, and SMIC the authors report that MOL outperforms state-of-the-art methods in most configurations, including the best three-class accuracy on CASME II (91.26) and SMIC (80.71), and that the two auxiliary tasks and the F5C block each contribute substantially in ablations. The paper also reports that the auxiliary tasks improve each other: adding MER and landmark detection lowers optical-flow error, and adding MER and optical flow lowers landmark error.","pith_inferences":["If the pseudo-label dependence is real, then replacing the automatic landmark and flow generators with human-refined or learned supervision could push accuracy higher, and worse pseudo-labels could erase the gain; this is testable by swapping label generators.","The pair-feature concatenation effectively treats every consecutive frame pair as a mini-clip; an untested corollary is that the same architecture may extend to micro-expression spotting, where onset and offset are unknown, by classifying all pairs and thresholding confidence.","The FCC's circular convolution with positional embeddings is a lightweight alternative to full self-attention; one could adapt the block to other fine-grained video tasks such as action unit detection or subtle gaze change, where local detail and global context both matter."],"forward_implications":["MER can be deployed on raw video clips directly, so pipelines no longer need onset/apex frame detectors or hand-crafted optical-flow inputs.","Because optical flow and landmark estimation improve in the joint framework, the same network can serve as a single model for several face-analysis tasks on small datasets.","The claimed results imply that pseudo-labeled auxiliary tasks can substitute for large-scale human annotation in micro-expression research, at least when the labels come from reliable off-the-shelf tools.","Cross-dataset gains (training on CASME II or SAMM, testing on SMIC) suggest the auxiliary tasks improve generalization to unseen subjects and recording conditions."],"supporting_citations":[{"why":"provides the CASME II benchmark and its five- and three-class evaluation protocol.","marker":"[39]"},{"why":"provides the SAMM benchmark used for single- and cross-dataset tests.","marker":"[40]"},{"why":"provides the SMIC benchmark and serves as test set for cross-dataset evaluation.","marker":"[41]"},{"why":"the face alignment library used to generate 68-landmark pseudo-labels for every frame.","marker":"[42]"},{"why":"the one-millisecond face alignment method behind the landmark pseudo-labels.","marker":"[43]"},{"why":"the TV-L1 optical flow algorithm that produces the flow pseudo-labels.","marker":"[44]"},{"why":"FlowNet supplies the encoder-decoder architecture of the optical flow estimation module.","marker":"[36]"},{"why":"transformer attention motivates the positional embedding and global receptive field of FCC.","marker":"[28]"},{"why":"dynamic graph convolution motivates the k-NN channel graph in CCC.","marker":"[33]"},{"why":"defines the AU-assisted graph attention baseline and the cross-dataset evaluation protocol.","marker":"[21]"}],"fun_headline_variants":["Joint model spots micro-expressions, motion, and landmarks from raw video","One network jointly predicts micro-expressions, flow, and landmarks","End-to-end model fuses micro-expression, motion, and shape cues","Joint training on raw frames boosts micro-expression, flow, and landmark tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the automatically generated pseudo-labels, Dlib's 68 landmarks and TV-L1 optical flow, are accurate enough to teach the shared features about real micro-expression motion even though these tools were not designed for micro-expression videos.","fun_headline_variants_meta":{"raw":{"variants":["Joint model spots micro-expressions, motion, and landmarks from raw video","One network jointly predicts micro-expressions, flow, and landmarks","End-to-end model fuses micro-expression, motion, and shape cues","Joint training on raw frames boosts micro-expression, flow, and landmark tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000778,"raw_usage":{"total_tokens":3465,"prompt_tokens":1000,"completion_tokens":2465,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":2388}},"tokens_in":616,"tokens_out":2465,"duration_ms":15810,"temperature":1.0,"reasoning_tokens":2388,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:17:03.112062+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain MOL on the same benchmarks but replace the pseudo-label generators with human-annotated landmarks and optical flow, or with a different landmark and flow model; if the accuracy gap over the no-auxiliary-task variant shrinks or vanishes, the reported gains depend on the specific pseudo-label tools rather than on multi-task learning per se.","supporting_citations":[{"cited_title":"Casme ii: An improved spontaneous micro-expression database and the baseline evaluation,","cited_arxiv_id":null,"evidence_quote":"provides the CASME II benchmark and its five- and three-class evaluation protocol."},{"cited_title":"Samm: A spontaneous micro-facial movement dataset,","cited_arxiv_id":null,"evidence_quote":"provides the SAMM benchmark used for single- and cross-dataset tests."},{"cited_title":"A spontaneous micro-expression database: Inducement, collection and baseline,","cited_arxiv_id":null,"evidence_quote":"provides the SMIC benchmark and serves as test set for cross-dataset evaluation."},{"cited_title":"One millisecond face alignment with an ensemble of regression trees,","cited_arxiv_id":null,"evidence_quote":"the one-millisecond face alignment method behind the landmark pseudo-labels."},{"cited_title":"A duality based approach for realtime tv-l 1 optical flow,","cited_arxiv_id":null,"evidence_quote":"the TV-L1 optical flow algorithm that produces the flow pseudo-labels."},{"cited_title":"Flownet: Learning optical flow with convolutional networks,","cited_arxiv_id":null,"evidence_quote":"FlowNet supplies the encoder-decoder architecture of the optical flow estimation module."},{"cited_title":"Attention is all you need,","cited_arxiv_id":null,"evidence_quote":"transformer attention motivates the positional embedding and global receptive field of FCC."},{"cited_title":"Dynamic graph cnn for learning on point clouds,","cited_arxiv_id":null,"evidence_quote":"dynamic graph convolution motivates the k-NN channel graph in CCC."},{"cited_title":"Au-assisted graph attention convolutional network for micro-expression recognition,","cited_arxiv_id":null,"evidence_quote":"defines the AU-assisted graph attention baseline and the cross-dataset evaluation protocol."}],"review_version":1}