{"id":"e4d3ef8c-d186-4c0a-997c-76b6a585543b","arxiv_id":"2507.21778","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"AU-LLM uses a large language model and a feature-fusion projector to detect micro-expression action units, reporting the best average F1 scores on CASME II and SAMM.","lead":"This paper combines a 3D video model with a large language model (LLM) to detect micro-expressions, tiny involuntary facial movements, on two standard datasets. The authors report a new state-of-the-art score by fusing local and global visual features into a single token that the LLM reasons over.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No ablation isolates the LLM: the reported SOTA gains could come entirely from the 3D-CNN+EFP features, leaving the central novelty claim unsupported.","rationale":"The reader correctly identified the missing LLM-free baseline as the weakest assumption. I see no reason to move away from a conditional verdict: the empirical within-domain results are plausible, but the causal attribution to LLM reasoning is untested, and the duplicated Table 4 values cast doubt on the cross-domain subclaim. The proposed ablation directly tests the central claim and is feasible given the reported setup, so the appropriate disposition is to require that experiment before accepting the LLM-specific novelty. I considered elevating Table 4 to the primary concern, but a duplicated table cell could be a transcription error, whereas the missing LLM-free baseline is a structural gap in the argument; the latter is more load-bearing for the paper's thesis.","tokens_in":8798,"tokens_out":5486,"duration_ms":58797,"concrete_test":"Run the following controlled experiment on CASME II and SAMM with the exact LOSO protocol and ASL loss: keep the visual backbone and EFP unchanged, but replace the LoRA-tuned LLM with a non-LLM classifier (e.g., a 2-layer MLP with hidden dimension comparable to the LLM embedding, or a linear layer) applied directly to the fused token Tv. Report average F1 for each AU. If the non-LLM head reaches 81.4% (CASME II) or 61.9% (SAMM) or within one standard deviation, the LLM is not the source of the SOTA gain. As a follow-up, also verify Table 4 by recomputing the CASME II to SAMM cells for ResNet-18 and LED SSSNet from raw predictions; the duplicated AU2 and AU7 values should not be identical.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that a pretrained, LoRA-tuned LLM adds reasoning power for micro-expression AU detection. Section 4.4 ablates fusion inputs (Fhigh, Fmid), the projector (EFP-), and prompting strategy, but every variant still ends with the same LoRA-tuned LLM classifier. No experiment replaces the LLM with a non-LLM head on the identical fused token Tv. Thus the 3.0-point gain over SSSNet LED on CASME II and the 4.6-point gain on SAMM cannot be attributed to the LLM rather than to the 3D-CNN+EFP representation. If a simple MLP or linear classifier on the same token matches these F1 values, the 'first LLM-based AU detection' claim and the paper's motivation collapse to a feature-engineering result. This is load-bearing because the title, abstract, and conclusion all assert the LLM's reasoning is responsible for the improvement. A secondary red flag in the same direction: Table 4 contains identical F1 values across different baselines (e.g., AU2=47.0 and AU7=41.5 for both ResNet-18 and LED SSSNet under CASME II to SAMM), which suggests the cross-domain comparison may be unreliable; this should be verified but is not the primary concern.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AU-LLM, a framework for micro-expression Action Unit (AU) detection that combines a 3D-CNN backbone (with LED temporal filtering and a Squeeze-and-Excitation layer) with a Large Language Model (LLM). Mid-level and high-level visual features are fused by an MLP-based Enhanced Fusion Projector (EFP) into a single visual token, which is prepended to a short text prompt and passed to a LoRA-tuned 1.5B LLM. The last token hidden state is fed to a linear classifier for multi-label AU prediction. Experiments on CASME II and SAMM report average F1 of 81.4% and 61.9% for the best configuration (DeepSeek-R1), outperforming the prior state of the art (78.4% and 57.3%). The paper also includes ablations of the EFP components and bidirectional cross-domain evaluations between CASME II and SAMM.","tokens_in":8832,"tokens_out":5163,"duration_ms":55701,"significance":"If the result holds, AU-LLM would be the first successful application of an LLM to micro-expression AU detection, and the EFP's compact single-token fusion is an interesting design for bridging the vision-language gap in fine-grained tasks. Strengths include the use of standard benchmarks and protocols (LOSO and cross-domain), the reported comparisons against several strong baselines, and the public code release. However, the central attribution of the improvement to the LLM's reasoning is currently unsupported because no experiment replaces the LLM with a non-LLM classifier on the same token. The absence of error bars and the suspicious duplicate values in the cross-domain table further weaken the quantitative claims. With the requested ablations and verification, the contribution could be a solid advance for the micro-expression analysis community.","major_comments":[{"comment":"The experiments do not include a baseline that replaces the LoRA-tuned LLM with a standard classifier (e.g., an MLP or a linear head) on the same fused token Tv. Every ablated variant reported in Table 3 still ends with the LLM, so the 3.0-point gain over SSSNet LED on CASME II and the 4.6-point gain on SAMM cannot be attributed to the LLM's reasoning rather than to the 3D-CNN+EFP representation. Because the title, abstract, and conclusion all credit the LLM reasoning for the improvement, this missing control is load-bearing and should be added.","section":"Section 4.4, Tables 1–3"},{"comment":"In the CASME II→SAMM block, ResNet-18 and LED SSSNet report identical F1 values for AU2 (47.0), AU7 (41.5), and AU12 (45.1), which is implausible for two different models and strongly suggests a copying or reporting error. This table is the basis for the cross-domain generalization claim (13.4 percentage-point improvement over LED SSSNet), so the numbers must be verified and corrected before the claim can be evaluated.","section":"Table 4"},{"comment":"No standard deviations, confidence intervals, or significance tests are reported for any F1 result, although the LOSO protocol produces multiple folds and the training procedure involves stochastic optimization. The reported margin of 3.0 points over SSSNet LED on CASME II may be within run-to-run or fold variation; the state-of-the-art claim requires variance estimates or statistical testing.","section":"Section 4.3"},{"comment":"The paper evaluates three LLM backbones (DeepSeek-R1, Qwen2, Qwen2.5) and then presents the best-performing one on the test set as the proposed AU-LLM result, without a pre-specified choice, a shared validation set, or a multiple-comparison correction. This post-hoc selection inflates the reported improvement and should be justified, for example by reporting all variants as the method's performance range or by selecting the backbone based on validation performance alone.","section":"Section 4.3, Tables 1–2"}],"minor_comments":[{"comment":"The formula for the LED filter is typeset incorrectly: the exponents and subscripts in the piecewise expression are garbled, making the equation unreadable. Please use the original formulation from [20] or fix the LaTeX.","section":"Equation (1)"},{"comment":"The text says evaluation on SAMM is 'cross-dataset validation,' but Table 2 does not state whether the models were trained on CASME II and tested on SAMM or trained on SAMM directly with LOSO. Please clarify the protocol and ensure that all compared methods use the identical setting.","section":"Section 4.1, Table 2"},{"comment":"The claim that this is the 'first' use of an LLM for AU detection should be supported by a more thorough search of recent literature, including concurrent or arXiv-only works; consider softening the novelty claim or adding the missing references.","section":"Abstract, Section 1"},{"comment":"The exact text prompt (e.g., 'Please analyze the features and classify AU') is only described in prose; include the full prompt template and tokenization details to improve reproducibility.","section":"Section 3.3"},{"comment":"The caption of Figure 2 does not explain what is plotted (e.g., per-AU F1 curves or bar charts), and the figure is only referenced in passing; please describe the visualization content in the text and caption.","section":"Table 3, Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a plausible and well-motivated architecture, but the central attribution of the performance gain to the LLM is not experimentally isolated, and the cross-domain table contains unexplainable duplicate numbers that need author verification. I recommend requiring the LLM-free ablation, error bars, and a corrected Table 4 before considering acceptance. The post-hoc selection of the best LLM among three variants is also a correctness-risk concern that should be addressed. The paper is within the journal's scope and the code release is a positive signal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the thing to know: this paper reports new best F1 numbers on micro-expression AU detection with an LLM-based classifier, but it never tests whether the LLM is actually contributing anything. Every ablation still ends with the same LoRA-tuned LLM, so the reported 3-5 point gains over SSSNet LED could come entirely from the 3D-CNN and the fusion MLP.\n\nWhat is genuinely new: this is the first application of an LLM to micro-expression AU detection, as far as I know. The design is clean: a LED-filtered 3D-CNN backbone, mid- and high-level features fused via an MLP into a single visual token, then fed to a 1.5B LLM with LoRA. The EFP is a reasonable variant of standard vision-language projectors. The experiments use standard LOSO and cross-domain protocols on CASME II and SAMM, and the ablations show that both mid-level and high-level features matter, and that the non-linear MLP fusion beats a linear projection. The three LLM variants give consistent results, which is a nice sanity check.\n\nThe soft spots, in order of severity. First, the missing LLM-free baseline is the load-bearing flaw. Without replacing the LLM with a linear classifier or MLP on the same fused token, the paper cannot support its central claim that LLM reasoning is the reason for the improvement. This is not a theoretical nit; on a dataset with 247 samples, a 3-point F1 gain is well within the range of classifier noise. Second, Table 4 has duplicated numbers: under CASME II to SAMM, AU2=47.0 and AU7=41.5 for both ResNet-18 and SSSNet, and AU12=45.1 for both. That looks like a copy-paste error and makes the cross-domain average comparison untrustworthy. Third, there are no error bars or significance tests anywhere. Fourth, the code link in the abstract is a placeholder, so the reproducibility claim is unverifiable from the manuscript alone. Finally, the 'first' framing is fine for micro-expression AU detection specifically, but the paper does not survey LLM-based AU detection in macro-expressions, so a broader 'first to use LLM for AU detection' would be an overstatement.\n\nWho should read this: anyone working on micro-expression analysis or trying to apply LLMs to fine-grained visual tasks. It is worth a serious referee, but the central claim needs a proper ablation and the cross-domain table needs to be fixed. I would send it to review with a request for an LLM-free classifier baseline and error bars.","headline":"A useful empirical first for micro-expression AU detection with an LLM, but the paper never isolates the LLM's contribution, so the SOTA claim rests on a missing ablation.","tokens_in":9579,"tokens_out":4272,"would_cite":false,"duration_ms":41452,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a large language model, fed a single fused visual token that combines mid-level texture and high-level semantic features, can outperform prior methods on micro-expression Action Unit detection, reaching 81.4% average…","keywords":["Micro-expression","Action Unit detection","Large Language Model","Feature Fusion","LoRA","CASME II","SAMM","Affective Computing"],"falsifier":"Run the same LOSO protocol on CASME II with the backbone and EFP intact but swap the LoRA-tuned LLM for a multi-layer perceptron or linear classifier trained on the identical fused token; if average F1 matches or exceeds 81.4%, the paper's central claim that LLM reasoning drives the improvement collapses.","tokens_in":8393,"feed_emoji":"🎭","tokens_out":6451,"duration_ms":60357,"temperature":0.7,"pith_summary":"This paper sets out to show that a large language model can serve as a reasoning engine for micro-expression Action Unit (AU) detection, a task where subtle, fleeting facial muscle movements make ordinary classifiers struggle. To do this it introduces AU-LLM, which fuses mid-level local-texture features and high-level semantic features from a 3D-CNN backbone into a single visual token and feeds that token to a LoRA-tuned LLM. The authors report that this design beats previous state-of-the-art methods on two standard benchmarks, reaching 81.4% average F1 on CASME II (prior best 78.4%) and 61.9% on SAMM (prior best 57.3%), and that it also generalizes under cross-domain protocols. If true, this would establish LLM-based reasoning as a viable route for fine-grained, low-intensity facial analysis rather than a technique limited to high-level visual understanding.","feed_headline":"LLM reasoning lifts micro-expression AU detection to 81.4% F1","feed_subtitle":"Fusing local and global facial features into one token lets a LoRA-tuned LLM beat prior best on CASME II and SAMM.","key_machinery":"The load-bearing mechanism is the Enhanced Fusion Projector (EFP), an MLP that maps the concatenation of flattened mid-level and high-level visual features to a single information-dense token in the LLM's embedding space. The EFP is what bridges the vision-language semantic gap: a linear projection would discard low-intensity local texture cues, while the MLP's non-linear transformation preserves both local textural detail (from Fmid) and global semantic context (from Fhigh) in one vector. Around it sit the LED temporal-filtering module that amplifies subtle motion, a 3D-CNN backbone with a Squeeze-and-Excitation channel recalibration layer, LoRA fine-tuning of the LLM's attention projections, and the Asymmetric Loss that down-weights negative samples. The final classification is read off the hidden state of the last token, so the entire pipeline is trained end-to-end for multi-label AU prediction.","core_discovery":"The central discovery is that micro-expression AU detection can be reframed as a vision-language reasoning problem, provided the visual representation is distilled correctly. The paper's AU-LLM pipeline extracts temporally filtered spatio-temporal features with a 3D-CNN whose LED module amplifies onset-apex-offset motion patterns, then forms a compact token by concatenating mid-level features (local texture) with high-level features (global semantics) and passing the concatenation through an MLP. This fused token is prepended to a text prompt such as 'Analyze the facial features to classify action units' and processed by a 1.5B-parameter LLM fine-tuned with LoRA; the last token's hidden state feeds a linear AU classifier. With DeepSeek-R1-Distill-Qwen-1.5B as the LLM, the method reports 81.4% average F1 on CASME II and 61.9% on SAMM, and in bidirectional cross-domain tests it outperforms LED-based baselines by 13.4 and 6.7 percentage points respectively. The paper attributes these gains to the EFP's non-linear multi-level fusion, the LLM's reasoning, and the asymmetric loss that counters AU class imbalance.","pith_inferences":["The paper never replaces the LLM with a plain classifier on the same fused token, so the strongest possible reading—that LLM reasoning itself drives the gain—remains unproven; the EFP's fused representation could plausibly be doing most of the work.","A natural extension is to test whether the EFP token alone, without the LLM, already beats the LED-based baselines; that experiment would isolate the contribution of the fusion projector from the language model.","The single-token design may cap the LLM's spatial reasoning; feeding multiple regional tokens (per facial region) might improve AU localization even further, at the cost of more LLM computation.","Because the method's gains appear consistently across three 1.5B LLMs, the result may transfer to other parameter-efficient fine-tuning settings, but whether it scales with larger LLMs is untested and could go either way."],"forward_implications":["LLM-based reasoning is a workable paradigm for low-intensity, fine-grained facial analysis, not just macro-expression recognition, opening micro-expression AU detection to the tools of multimodal LLMs.","Fusing mid-level texture with high-level semantic features beats using either alone in this framework, so future AU detectors should preserve local detail when adapting visual features for language models.","A non-linear fusion projector gives a clear gain over a linear projector (the EFP− variant), implying that the vision-to-language mapping is a major source of accuracy, not a trivial embedding lookup.","Cross-domain results (49.9% on CASME II→SAMM and 52.8% on SAMM→CASME II) predict that LLM-based detectors can transfer between datasets with different subjects, ethnicities, and distributions better than the tested baselines.","The authors' stated future direction—interactive, context-aware micro-expression question answering—is a direct corollary if AU detection with this architecture is robust."],"supporting_citations":[{"why":"Supplies the LED temporal-filtering module and the SSSNet LED baseline that AU-LLM must beat; the backbone builds on it.","marker":"[20]"},{"why":"Provides the parameter-efficient fine-tuning method used to adapt the LLM's attention layers.","marker":"[7]"},{"why":"Provides the multi-label loss function that handles AU class imbalance.","marker":"[15]"},{"why":"Provides the primary within-domain benchmark dataset and its 8-AU annotation protocol.","marker":"[24]"},{"why":"Provides the cross-domain benchmark dataset with 4 shared AUs used for generalization tests.","marker":"[3]"},{"why":"Defines the Action Units that are the detection targets, grounding the task itself.","marker":"[5]"},{"why":"One of the attention-based baselines that AU-LLM compares against and outperforms.","marker":"[11]"}],"fun_headline_variants":["Fused-token LLM hits 81.4% F1 for micro-expression AUs","LLM fusion token lifts AU detection to 81.4% F1","One fused token lets LLM hit 81.4% F1 on facial AUs","LLM reasoning with fused features tops AU benchmark at 81.4% F1","Multi-level fusion lets LLM read micro-expression AUs at 81.4% F1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the LLM's reasoning is what lifts performance rests on an untested assumption: no experiment in the paper replaces the LLM with an ordinary classifier fed the same fused token, so the fused representation alone might explain the gains.","fun_headline_variants_meta":{"raw":{"variants":["Fused-token LLM hits 81.4% F1 for micro-expression AUs","LLM fusion token lifts AU detection to 81.4% F1","One fused token lets LLM hit 81.4% F1 on facial AUs","LLM reasoning with fused features tops AU benchmark at 81.4% F1","Multi-level fusion lets LLM read micro-expression AUs at 81.4% F1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001171,"raw_usage":{"total_tokens":4901,"prompt_tokens":1063,"completion_tokens":3838,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":679,"completion_tokens_details":{"reasoning_tokens":3724}},"tokens_in":679,"tokens_out":3838,"duration_ms":28795,"temperature":1.0,"reasoning_tokens":3724,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:22:55.232654+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same LOSO protocol on CASME II with the backbone and EFP intact but swap the LoRA-tuned LLM for a multi-layer perceptron or linear classifier trained on the identical fused token; if average F1 matches or exceeds 81.4%, the paper's central claim that LLM reasoning drives the improvement collapses.","supporting_citations":[{"cited_title":"In: Image Analysis (2024)","cited_arxiv_id":null,"evidence_quote":"Supplies the LED temporal-filtering module and the SSSNet LED baseline that AU-LLM must beat; the backbone builds on it."},{"cited_title":"In: ICCV (2021)","cited_arxiv_id":null,"evidence_quote":"Provides the multi-label loss function that handles AU class imbalance."},{"cited_title":"In: PLOS ONE (2014) AU-LLM: Enhancing Micro-Expression AU Detection with LLMs 11","cited_arxiv_id":null,"evidence_quote":"Provides the primary within-domain benchmark dataset and its 8-AU annotation protocol."},{"cited_title":"In: IEEE TAFFC (2018)","cited_arxiv_id":null,"evidence_quote":"Provides the cross-domain benchmark dataset with 4 shared AUs used for generalization tests."},{"cited_title":"Consulting Psychologists Press (1978)","cited_arxiv_id":null,"evidence_quote":"Defines the Action Units that are the detection targets, grounding the task itself."},{"cited_title":"Neurocomputing (2021)","cited_arxiv_id":null,"evidence_quote":"One of the attention-based baselines that AU-LLM compares against and outperforms."}],"review_version":1}