{"id":"04e5e110-bacc-4ec4-af88-c0e9b0b0a82b","arxiv_id":"2411.10018","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A computational pipeline aligns movie audio with script text and uses speech emotion models to show that acted emotions in American film track narrative structure, release year, genre, and dialogue function.","lead":"This paper uses speech emotion recognition to measure how actors sound in 2,283 American films, not just what their scripts say. It finds that emotional performances vary with narrative time, release year, genre, and the function of the dialogue phrase.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on the SER model isolating acted emotion, but the deployed contextual model and wav2vec2 features can encode neighboring-utterance context and lexical content; the reported patterns may be script- or context-driven rather than performance-driven.","rationale":"I agree with the reader that measurement validity is the load-bearing assumption; my stress-test narrows it to the specific architectural and representational channels through which non-performance signal can enter. The comparison in Table 1 shows the contextual model is only slightly better than the utterance-level model, so the paper's choice to use it everywhere is consequential and untested. The text-only control I propose would settle whether the acoustic classifier is adding signal beyond the script, and the existing within-phrase-group analysis is the only step in that direction. Because the concern is concrete and addressable with a sensitivity analysis, the reader's CONDITIONAL verdict remains appropriate; I do not see grounds for rejection or for acceptance without such a check. I also note the absence of released code and data, which makes this conditional status harder to resolve, though it is secondary to the construct-validity issue.","tokens_in":9813,"tokens_out":8682,"duration_ms":100853,"concrete_test":"Train a text-only emotion classifier on the MELD training transcripts (same seven labels) and apply it to the ASR transcripts of the movie corpus, then recompute the Sec. 3.1 narrative trajectories, the Sec. 3.2 within-phrase-group diachronic regression, and the Sec. 3.3 genre/dialogue-range rankings from the text-only predictions. If the text-only patterns match the speech-model patterns within bootstrap error, the reported findings are script-level rather than performance-level; if they diverge, the acoustic channel is contributing signal.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim—'narrative structure, diachronic shifts, and genre- and dialogue-based constraints located in spoken performances'—requires the emotion predictions used as outcome variables to be valid measures of acted emotional delivery, not of lexical content or conversational context. That requirement is not currently established. The contextual model (Sec. 2.2.2) predicts each utterance's emotion through a biLSTM over neighboring utterances, so every Sec. 3 case study can be driven by surrounding dialogue rather than by the target utterance's performance; the paper uses this model for all analyses (Sec. 2.2.3). In addition, the wav2vec2 representations fed into the classifier retain phonetic and lexical information, so the model has access to what is said as well as how it is said. The only text-controlled analysis is the within-phrase-group diachronic regression (Sec. 3.2), and it explains only R2=0.048. Low human agreement (Krippendorff alpha=0.334) and 48.8% Movies accuracy do not by themselves falsify aggregate results, but the bootstrap intervals in Figs. 1–3 cover only sampling variability in model outputs, not classifier or label error. The claim may be true, but it is not yet separated from script and context.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a computational pipeline for measuring acted emotion in contemporary American film. The pipeline segments audio into utterances, transcribes and aligns dialogue, trains a wav2vec2-based speech emotion recognition (SER) model on MELD, and evaluates it on MELD and a newly annotated 35-film corpus. Using the model's predictions, the authors run three analyses: (1) trajectories of emotionality and specific emotions over narrative time, (2) diachronic trends in emotionality with a within-phrase-group fixed-effects regression, and (3) a Dirichlet-entropy measure of emotional range across genres and across semantically grouped phrases. The central claim is that these analyses reveal narrative structure, diachronic shifts, and genre- and dialogue-based constraints located in spoken performances.","tokens_in":9986,"tokens_out":3928,"duration_ms":39743,"significance":"Demonstrating that acted emotion in film can be measured reliably from speech and separated from script would be a substantial methodological contribution to computational film and performance studies. The paper has notable strengths: a reproducible pipeline with publicly available components, an in-domain evaluation set with a transparent annotation protocol, a phrase-group control for lexical content, and a candid limitations section. If the measurement-validity issues are resolved, the findings on narrative arcs and genre differences would be of broad interest. As it stands, however, the central claim is contingent on the SER output being a valid measure of acted delivery, and that premise is not yet established.","major_comments":[{"comment":"The contextual SER model predicts each utterance's emotion through a biLSTM over neighboring utterances, and the paper uses its outputs for all analyses in Section 3. Consequently, the reported patterns (e.g., emotionality increasing over narrative time in Fig. 1a, genre ordering in Fig. 3) could be driven by the lexical and conversational content of surrounding dialogue rather than by how the target utterance is performed. The claim that the findings are 'located in spoken performances' requires either using the utterance-level model as the primary outcome or demonstrating that contextual information is not responsible for the observed patterns; for example, by re-running the analyses with the utterance-level model and showing qualitatively identical results.","section":"2.2.2, 2.2.3, 3"},{"comment":"The models obtain 0.488 accuracy and 0.450 weighted F1 on the Movies evaluation set, and the human inter-annotator agreement on the new evaluation set is low (Krippendorff's alpha = 0.334, Fleiss' kappa = 0.333). The 95% bootstrap confidence intervals in Figures 1–3 reflect only resampling of the predicted values across movies or utterances, not the substantial classifier and label noise. This means the statistical precision of the reported trends, such as the ordering of genres in Figure 3, is overestimated. A sensitivity analysis that resamples labels according to the model's confusion matrix, or that varies the classification threshold, would be needed to support the claimed patterns.","section":"2.2.3, Table 1, Figs. 1–3"},{"comment":"The within-phrase-group regression is the key control for separating script from performance, but it explains only R2 = 0.048 (F(1, 21461), p < 0.001). While the coefficient is statistically significant, the tiny effect size leaves ample room for residual confounding from imperfect phrase clustering, acoustic conditions, or other covariates. The text accurately describes the coefficient as slightly negative and significant, but it should also discuss the practical significance of this effect size and report robustness checks such as controlling for film-level random effects or utterance duration.","section":"3.2"},{"comment":"The emotional range measure is defined as the entropy of a Dirichlet fitted to the model's predicted probability vectors. Because the SER model is far from perfectly calibrated and its predicted distributions are known to be noisy (Table 1), this entropy may reflect model uncertainty or label distribution rather than the actor's emotional range. The qualitative examples in Table 3 are suggestive, but no quantitative validation (e.g., correlation with human ratings of range) is provided. Without such validation, the genre and phrase-level conclusions in Section 3.3 should be framed as exploratory.","section":"3.3"}],"minor_comments":[{"comment":"The sentence 'We also train an contextual model' contains a typo; it should read 'We also train a contextual model.'","section":"2.2.2"},{"comment":"The section heading 'Embodied erformance' is a typo and should read 'Embodied Performance.'","section":"4"},{"comment":"The caption says 'Emotionality is higher in older films,' which is consistent with the text, but the text also notes a minimum around 2010; it would help to reconcile the shape of the trajectory (e.g., flat then rising before 1980, declining to 2010, then rising) in both the text and the figure.","section":"3.2 and Figure 2"},{"comment":"The description says 'joyful performances follow a U-shaped curve, with a steep increase towards the end,' but the plotted trajectory appears to decline in the first half and then rise steeply near the end; the narrative description should match the figure's shape more precisely.","section":"3.1 and Figure 1b"},{"comment":"In reference [22] the name is spelled 'Panovsky'; the standard spelling is 'Panofsky' (Erwin Panofsky), both in the reference and in the text of Section 3.2.","section":"References"},{"comment":"The sentence 'We expect the phrases in each group to have similar prior distributions of emotion' is an untested assumption; a short evaluation of phrase-group homogeneity (e.g., measuring within-group variance of predicted emotion probabilities) would strengthen the control.","section":"2.3"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of the conference and addresses a timely topic, but the central measurement-validity concern raised in the stress-test note is legitimate and load-bearing. The authors can address it by re-running the main analyses with the utterance-level model, by propagating classifier uncertainty through the bootstrap, and by bounding the influence of contextual features. I would not reject the manuscript, but the revision needs to seriously engage with these issues rather than treating them as routine limitations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read it. The genuinely new bit is the parallel dataset: acted film speech aligned to screenplay text at scale, plus a measure of emotional range (Dirichlet entropy over the classifier's predicted emotion distributions) that is a reasonable way to talk about performance capacity. The soft bit is the measurement stack. All three case studies—narrative time, diachronic change, genre and phrase constraints—use predictions from a speech emotion classifier with 45–49% accuracy on the in-domain movies set and a human label agreement of only α=0.334. The bootstrap intervals in the figures cover sampling variability in model outputs, not classifier or annotator error.\n\nThe stress-test concern is on target. The contextual model, which is the one actually used for every analysis, is a biLSTM over neighboring utterances, so the narrative-time curves in Sec. 3.1 can reflect conversational context or the script rather than the target utterance's acoustic performance. The wav2vec2 embeddings also carry phonetic and lexical information. The only analysis that controls for text—the within-phrase-group regression in Sec. 3.2—has R²=0.048. So the claim that these patterns live specifically in the spoken performance is not yet separated from script and context.\n\nThat said, this is not a circular argument. The emotion model was trained on MELD, not on the film corpus, and the phrase groups are semantic clusters, not outcome-fitted bins. The problem is construct validity and confounding, and both are fixable. Release the aligned dataset and pipeline, rerun the analyses with the utterance-level model as a sensitivity check, and propagate classifier uncertainty into the intervals. No code or data is currently public, which hurts reproducibility. The paper's own limitations section is unusually candid about the low agreement, the Ekman model, and the director/editor co-authorship of performance.\n\nI'd send this to peer review. The contribution—measuring performed emotion in film as a distinct layer from scripted text—is real and the qualitative findings (anger peaking near the end, joy's U-shape, functional phrases having low range) are plausible and tie to film theory. With the measurement concerns addressed, it would be a solid computational humanities paper. As it stands, it's a competent exploratory study that overstates the confidence of its conclusions.","headline":"New speech–text film corpus and a plausible emotional-range measure, but the classifier's validity is not yet strong enough to support all the case-study claims.","tokens_in":10588,"tokens_out":3174,"would_cite":true,"duration_ms":32396,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Acting performances in American film encode narrative arcs, genre constraints, and historical change in their spoken emotion.","keywords":["acting performance","speech emotion recognition","computational film analysis","variationist sociolinguistics","emotional range","narrative structure","film genre","diachronic analysis"],"falsifier":"Use trained human annotators to label a held-out sample of the same corpus with the same seven emotion categories, then check whether each of the three case-study patterns—rising emotionality over runtime, decline by release year within matched phrase groups, and genre ordering of emotional range—reproduces in the human labels; failure to reproduce would show that the model's predictions, not the performances, are carrying the findings.","tokens_in":9504,"feed_emoji":"🎭","tokens_out":8624,"duration_ms":90578,"temperature":0.7,"pith_summary":"The paper sets out to show that acting performance in popular contemporary American film is a measurable semiotic layer, distinct from the script and analyzable at scale. It aligns spoken utterances with the words being spoken, uses speech emotion recognition to assign each utterance an emotion profile, and then examines how those profiles vary over narrative time, across release years, by genre, and across semantically similar lines. The reported results include rising emotionality over a film's runtime, a U-shaped trajectory for joy with an anger peak near the climax, a mild decline in emotionality across decades that persists even within matched phrase groups, and clear genre- and phrase-based constraints on emotional range. If these findings hold, film scholarship can treat the actor's delivery as a quantitative object rather than a text-derived byproduct.","feed_headline":"Spoken performances in film carry measurable emotion arcs","feed_subtitle":"A speech-based study finds onscreen delivery rises in emotion over narrative time and varies by genre and era.","key_machinery":"The central machinery is a parallel dataset in which every spoken utterance is time-aligned to the script text being spoken, built by speaker segmentation, transcription, and word-level alignment. A contextual speech emotion recognition model, using pretrained speech representations passed through a bidirectional LSTM and trained on acted TV dialogue, outputs a seven-category emotion probability vector for each utterance. Semantically similar lines are clustered into dialogue phrase groups using sentence embeddings and Leiden community detection, which lets the analysis hold the words constant and examine variation in delivery. Emotional range is measured as the Shannon entropy of a Dirichlet distribution fitted to the emotion probability vectors of a set of utterances, so that a set of performances with highly variable emotion profiles has high range and a tightly constrained set has low range. This combination operationalizes a variationist sociolinguistic view in which the scripted line is a linguistic variable and the performance is a choice among emotional variants.","core_discovery":"The paper's central claim is that acted emotion in film speech is structured rather than idiosyncratic. Across 2,283 contemporary American films, the average probability that an utterance is non-neutral rises over narrative time; joy follows a U-shaped arc with a steep final upswing, sadness and anger fall toward the end, and anger peaks around 85 percent of runtime. Earlier films have higher emotionality than later ones, and this diachronic decline survives when the comparison is restricted to the same semantically matched phrase groups, suggesting a shift in performance style rather than only a shift in writing. Genre constrains emotional range, with thrillers, biographies, and mysteries at the low end and family films, musicals, and fantasy at the high end; dialogue phrases that are functional, such as yes/no questions and their answers, have narrow emotional range, while open-ended evaluative phrases such as “You're alive” admit wide emotional latitude. The paper interprets the spoken-performance channel as one that carries meaning in concert with, and sometimes in compensation for, the visual and textual channels of film.","pith_inferences":["Extension: because each performance vector is tied to a script line, the same data could in principle measure actor-level contribution by comparing performances of the same line across different films or remakes; the paper explicitly leaves the division of authorial labor among actor, director, and editor unaddressed.","Extension: the paper's low inter-annotator agreement suggests that a seven-category emotion space may be too coarse; a natural next test is whether continuous or fine-grained emotion labels sharpen the narrative and genre effects, which the paper discusses but does not carry out.","Extension: emotional range as Dirichlet entropy is text-agnostic, so it could be applied to non-film speech such as news, podcasts, or courtroom testimony to test whether functional dialogue is universally low-range across genres of spoken interaction."],"forward_implications":["Narrative arcs in film can be studied from how lines are delivered rather than only from what is written; the rise in emotionality over runtime provides performance-based evidence for climax-resolution structure.","Historical studies of emotion in culture must separate writing from performance: the within-phrase-group decline implies the spoken channel has cooled over recent decades even with written content held fixed.","Genre functions as a prior on emotional delivery, so analyses of acting range or emotional intensity should control for genre; low-range genres like thrillers cannot be compared directly with family films.","The negative diachronic trend and the visual-intensification thesis are compatible: if close-ups increasingly carry expressive nuance, the spoken channel can bear less emotional load, which the paper reads as a performance-side counterpart to that visual shift.","The aligned utterance-to-phrase dataset makes “how they say it” a queryable unit, enabling variationist studies of emotional range for specific lines across speakers, films, or decades."],"supporting_citations":[{"why":"Supplies the acted-dialogue training data for the speech emotion classifier.","marker":"[10]"},{"why":"Provides the layer-weighted wav2vec2 embedding method that the utterance-level model implements.","marker":"[11]"},{"why":"Supplies the 2,283-film corpus of contemporary American film on which all three case studies run.","marker":"[14]"},{"why":"Supplies the Leiden community-detection algorithm used to group semantically similar phrases.","marker":"[13]"},{"why":"Gives the visual-intensification thesis that motivates the diachronic test of rising or falling emotionality.","marker":"[8]"},{"why":"Provides emotion dynamics in movie dialogues whose narrative arc results the paper compares and partly replicates.","marker":"[18]"},{"why":"Documents the decline of emotional expression in Anglophone fiction, the comparison point for the diachronic film finding.","marker":"[20]"},{"why":"Supplies the six-basic-emotions-plus-neutral taxonomy used as the classification scheme.","marker":"[9]"}],"fun_headline_variants":["Film acting's emotion arcs: measured across 2,283 movies","Why movie emotions rise and fall: a data-driven look at acting","Acted emotion isn't random: it follows arcs and genres","The science of movie acting: emotion arcs and genre constraints","Emotion in film performances: structured, not idiosyncratic"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The findings rest on the model's predicted emotion labels being a valid measure of acted emotion in film; the model is correct on only 48.8 percent of the movie evaluation set and two human annotators agreed only weakly with each other, so if the predictions are tracking acoustic or conversational patterns rather than acting choices, the narrative, diachronic, and genre results do not follow.","fun_headline_variants_meta":{"raw":{"variants":["Film acting's emotion arcs: measured across 2,283 movies","Why movie emotions rise and fall: a data-driven look at acting","Acted emotion isn't random: it follows arcs and genres","The science of movie acting: emotion arcs and genre constraints","Emotion in film performances: structured, not idiosyncratic"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000363,"raw_usage":{"total_tokens":1893,"prompt_tokens":816,"completion_tokens":1077,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":991}},"tokens_in":432,"tokens_out":1077,"duration_ms":10228,"temperature":1.0,"reasoning_tokens":991,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:03:41.753660+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use trained human annotators to label a held-out sample of the same corpus with the same seven emotion categories, then check whether each of the three case-study patterns—rising emotionality over runtime, decline by release year within matched phrase groups, and genre ordering of emotional range—reproduces in the human labels; failure to reproduce would show that the model's predictions, not the performances, are carrying the findings.","supporting_citations":[{"cited_title":"Bamman, R","cited_arxiv_id":null,"evidence_quote":"Supplies the 2,283-film corpus of contemporary American film on which all three case studies run."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Leiden community-detection algorithm used to group semantically similar phrases."},{"cited_title":"Bordwell, Intensified Continuity Visual Style in Contemporary American Film, Film Quarterly 55 (2002) 16–28","cited_arxiv_id":null,"evidence_quote":"Gives the visual-intensification thesis that motivates the diachronic test of rising or falling emotionality."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides emotion dynamics in movie dialogues whose narrative arc results the paper compares and partly replicates."},{"cited_title":"Morin, A","cited_arxiv_id":null,"evidence_quote":"Documents the decline of emotional expression in Anglophone fiction, the comparison point for the diachronic film finding."}],"review_version":1}