Pith. sign in

REVIEW 4 major objections 5 minor 90 references

An evaluation of LLMs for generating movie reviews: GPT-4o, Gemini-2.0 and DeepSeek-V3

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM-written movie reviews are fluent but still miss the emotional range of human IMDb reviews.

desk verdict Competent small-scale evaluation; the per-model emotional rankings rest on unvalidated classifier outputs and should not be taken at face value. read the letter →

arxiv 2506.00312 v1 pith:YDWSMXQ2 submitted 2025-05-30 cs.CL cs.AI

Brendan Sands , Yining Wang , Chenhao Xu , Yuxuan Zhou , Lai Wei , Rohitash Chandra This is my paper
classification cs.CLcs.AI
keywords moviereviewgenerationlargelanguagemodelsGPT-4oDeepSeek-V3Gemini-2.0sentimentanalysisemotionclassificationIMDbuserreviews
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that today's large language models can generate grammatically fluent and structurally complete movie reviews from a film's subtitles or screenplay, but that the results still stand apart from IMDb user reviews in emotional richness and stylistic coherence. It compares three models—GPT-4o, DeepSeek-V3, and Gemini-2.0—using trigram patterns, sentiment polarity, emotion classification, cosine similarity, and a reader survey. The paper's main rankings are specific: DeepSeek-V3 gives the most balanced reviews and aligns most closely with IMDb sentiment, GPT-4o overweights positive emotion, and Gemini-2.0 captures negative emotion but at excessive intensity. The stakes are practical: automated reviews can already pass as human to casual readers, so knowing where the remaining tell-tale gap lives matters for any system that generates or filters criticism.

What carries the argument

The load-bearing mechanism is a persona-prompted generation-and-comparison pipeline. Each model writes positive, neutral, and negative reviews of six movies through five persona prompts, using either subtitles or screenplays as input context. The texts are then scored by pretrained transformer classifiers—RoBERTa for sentiment polarity and DistilRoBERTa for seven emotions—compared through trigram frequency and TF-IDF cosine similarity, and checked against human perception in a 50-response survey. The classifier models carry the quantitative claim; the survey carries the claim that LLM reviews are hard to distinguish from IMDb.

What would settle it

Re-label a random sample of the LLM-generated and IMDb reviews with human annotators using the same three-way polarity and seven emotion categories, then recompute the model ordering; if the human labels show no consistent emotional gap or stable ranking, the central claim is refuted. A quicker probe is to feed the same review to the emotion classifier with and without surrounding movie context and see whether labels flip on ironic or mixed-toned passages.

Watch

Extended reading notes

Core claim

The central discovery is a profile of the gap between machine-written and human-written film criticism. Across six Oscar-level films and five persona prompts, all three models produced reviews that were syntactically clean and structurally complete, and survey participants frequently misidentified them as human-authored. Yet automated measures found a consistent emotional shortfall: LLM reviews were more extreme than IMDb reviews, with higher disgust and neutral scores, while IMDb reviews spread emotion more evenly. Model identities differed sharply—GPT-4o leaned positive with the highest joy scores, DeepSeek-V3 stayed neutral and closest to the IMDb distribution, and Gemini-2.0 reacted to negative prompts with strong disgust and sadness. Input type also shifted the result: subtitle-based reviews were more emotionally volatile, while screenplay-based reviews were more stable and more similar to IMDb text. The paper concludes that what separates LLM reviews from human ones is not grammar or structure but emotional nuance and stylistic moderation.

Load-bearing premise

The load-bearing premise is that the pretrained sentiment and emotion classifiers label movie reviews correctly; the paper itself concedes they misread highly subjective and context-rich reviews, so if that misreading is systematic, the reported emotional gap and model rankings could be artifacts of the measurement tool.

Editorial extensions

If this is right

  • Review generation from raw screenplay or subtitle text is viable enough that casual readers will often mistake the output for human criticism.
  • Subtitle-based inputs produce more volatile, extreme emotion in generated reviews, while screenplay-based inputs produce more stable and IMDb-like output, making input format a controllable lever.
  • Negative-prompt reviews are lexically closer to IMDb reviews than positive or neutral ones, suggesting concrete critique is easier to imitate than praise.
  • Detailed persona prompts reduce output variability for Gemini-2.0, pointing to prompt design as a way to suppress low-quality and emotionally inconsistent outputs.
  • Deploying these models for reviews means inheriting a model-specific emotional bias: GPT-4o skews positive, Gemini-2.0 skews negative and intense, and DeepSeek-V3 leans neutral.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the paper's admission that the emotion classifiers 'misinterpreted highly subjective and context-rich movie reviews' is taken at face value, the reported emotional gap may be partly a measurement artifact; re-annotating the same reviews with human raters would separate real stylistic differences from classifier error.
  • The trigram title-length artifact the paper observes suggests a normalization rule—dividing trigram counts by title-derived overlap—that future generation-evaluation studies could adopt before comparing models.
  • A natural stress test the paper's design cannot answer is the same pipeline on low-budget, non-Oscar, or non-English films; the paper's own limitation section predicts the fluency and alignment results would deteriorate, which would bound the finding to well-resourced mainstream cinema.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a framework for generating movie reviews with three LLMs (GPT-4o, DeepSeek-V3, Gemini-2.0) using subtitles or screenplays as input for six Oscar-level films, and compares the outputs with IMDb user reviews through trigram analysis, RoBERTa-based sentiment polarity, DistilRoBERTa-based emotion classification, TF-IDF cosine similarity, and a 50-participant human survey. The main claims are that LLMs produce syntactically fluent and structurally complete reviews, that LLM-generated reviews remain distinguishable from human reviews in emotional richness and stylistic coherence, and that the models differ systematically: DeepSeek-V3 gives the most balanced reviews, GPT-4o overemphasizes positive emotion, and Gemini-2.0 expresses negative emotion with excessive intensity.

Significance. If the results are accepted, the paper would provide a useful empirical mapping of stylistic and emotional biases in LLM-generated movie reviews, including an interesting comparison of subtitle-based versus screenplay-based inputs and of simple versus detailed persona prompts. The pipeline is transparent, with code and data promised on GitHub, the generated example reviews in Appendix B are a valuable qualitative resource, and the authors are candid in Section 5 about limitations of the emotion classifier, the content-moderation effects, and the small survey sample. However, the central quantitative claims—especially the 'emotional richness gap' and the per-model ranking—depend on the validity of pre-trained sentiment/emotion classifiers that the authors themselves admit misread context-rich movie reviews, and the human survey is too small and incomplete to corroborate the model-level ranking. The study is therefore a promising preliminary investigation rather than a fully supported evaluation.

major comments (4)
  1. [Section 3.5 and Section 5]
  2. [Section 4.5, Tables 6–8]
  3. [Section 4.2 and Section 4.3, Figures 2–6 and Table 5]
  4. [Section 4.4 and Figure 7]
minor comments (5)
  1. [Section 3.2]
  2. [Section 4.5]
  3. [Section 3.3]
  4. [Section 4.3]
  5. [Section 4.2 and Figure 2]

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: LLM-vs-IMDb comparisons are empirical measurements against an external benchmark using pre-trained classifiers; only minor, non-load-bearing self-citation.

full rationale

The central claims—that LLMs produce fluent but emotionally less rich reviews, and that DeepSeek-V3 is 'most balanced' while GPT-4o over-expresses positivity and Gemini-2.0 over-expresses negativity—are derived by comparing LLM outputs against external IMDb user reviews using off-the-shelf pre-trained tools: RoBERTa for sentiment polarity, DistilRoBERTa/GoEmotions for emotion (Section 3.5), TF-IDF cosine similarity (Section 3.6), and a human survey (Section 4.5). No parameter is fitted to these data and no 'prediction' restates a fitted value: the 'most balanced' verdict is a reading of Table 5's measured emotion distributions relative to the external IMDb row, so the conclusion is not equivalent to its inputs by construction. The only self-references are (i) ref [28] (Chandra and Ren, same group), cited in the Background and echoed in the Discussion as a loose analogy ('a pattern that resonates with our observations'), which supports but does not drive the central claim; and (ii) the Kaggle subtitle/IMDb-review datasets [43, 44] posted by group member G. Ren, which are provenance for externally authored IMDb reviews and Hollywood subtitles. Neither makes the derivation reduce to a self-citation chain. Two flagged issues are correctness risks rather than circularity: the Discussion (Section 5) explicitly concedes that the emotion tools 'misinterpreted highly subjective and context-rich movie reviews' (a validity threat to the emotional-gap measurement, already reflected in the skeptic's attack), and Section 4.5 misattributes Review 2 (listed as Gemini in Table 6) to GPT-4o while claiming the survey validates GPT-4o's positive emotion (an internal inconsistency affecting evidentiary weight, not the logical independence of the conclusion). Because the model rankings remain empirically measured against an external benchmark, the appropriate verdict is no significant circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper's quantitative conclusions rest on the validity of pre-trained sentiment and emotion classifiers, on arbitrary thresholds for mapping ratings to sentiment categories, on a small set of acclaimed films, and on the assumption that TF-IDF cosine similarity captures review quality. No new theoretical entities are introduced.

free parameters (3)
  • IMDb rating thresholds for sentiment mapping = 7 for positive, 6-7 neutral, below 6 negative
    Chosen by hand in Section 4.2 to align IMDb user ratings with the prompted sentiment categories (positive/neutral/negative). The exact thresholds are arbitrary and affect which IMDb reviews are treated as positive, neutral, or negative.
  • Chunk length for long reviews = 512 tokens
    Used in Section 3.5 to segment long reviews before feeding them to the RoBERTa sentiment model. Changing this value could change the averaged polarity scores.
  • Emotion class set and top-k selection = 7 classes (joy, sadness, anger, fear, surprise, disgust, neutral), top-5 per review
    Adopted from the DistilRoBERTa emotion model in Section 3.5. The choice of 7 classes and keeping the top-5 emotions shapes the reported emotion profiles.
assumptions (5)
  • domain assumption RoBERTa sentiment classifier accurately labels sentiment in both LLM-generated and IMDb reviews
    Invoked in Section 3.5 and Section 4.2 to compute polarity distributions. The Discussion admits the classifier can misinterpret context-rich movie reviews.
  • domain assumption DistilRoBERTa emotion classifier accurately identifies joy, sadness, anger, fear, surprise, disgust, and neutral emotions
    Invoked in Section 3.5 and Section 4.3. The authors note in Section 5 that the tool misinterprets nuanced expressions, which is a risk for the emotion scores.
  • domain assumption IMDb ratings grouped by thresholds reflect the true sentiment of the review text
    Used in Section 4.2 to split IMDb reviews into positive, neutral, and negative sets. A user may rate a movie high while writing a nuanced or mixed review, so the mapping is approximate.
  • domain assumption Six Oscar-winning or nominated movies are representative enough to support general conclusions about LLM movie review generation
    Stated in Section 3.1 and acknowledged in the Discussion as a limitation; performance may be overstated for these well-documented films.
  • domain assumption TF-IDF cosine similarity is a meaningful proxy for the similarity of generated and human reviews
    Used in Section 3.6 and Section 4.4. The authors admit it captures only surface-level word usage, not deep semantics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An evaluation of LLMs for generating movie reviews: GPT-4o, Gemini-2.0 and DeepSeek-V3." pith.science (2026). https://pith.science/paper/YDWSMXQ2

@misc{pith2026250600312,
  author       = {Pith},
  title        = {Pith review of: An evaluation of LLMs for generating movie reviews: GPT-4o, Gemini-2.0 and DeepSeek-V3},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YDWSMXQ2}},
  note         = {Machine review of arXiv:2506.00312}
}
read the original abstract

Large language models (LLMs) have been prominent in various tasks, including text generation and summarisation. The applicability of LLMs to the generation of product reviews is gaining momentum, paving the way for the generation of movie reviews. In this study, we propose a framework that generates movie reviews using three LLMs (GPT-4o, DeepSeek-V3, and Gemini-2.0), and evaluate their performance by comparing the generated outputs with IMDb user reviews. We use movie subtitles and screenplays as input to the LLMs and investigate how they affect the quality of reviews generated. We review the LLM-based movie reviews in terms of vocabulary, sentiment polarity, similarity, and thematic consistency in comparison to IMDB user reviews. The results demonstrate that LLMs are capable of generating syntactically fluent and structurally complete movie reviews. Nevertheless, there is still a noticeable gap in emotional richness and stylistic coherence between LLM-generated and IMDb reviews, suggesting that further refinement is needed to improve the overall quality of movie review generation. We provided a survey-based analysis where participants were told to distinguish between LLM and IMDb user reviews. The results show that LLM-generated reviews are difficult to distinguish from IMDB user reviews. We found that DeepSeek-V3 produced the most balanced reviews, closely matching IMDb reviews. GPT-4o overemphasised positive emotions, while Gemini-2.0 captured negative emotions better but showed excessive emotional intensity.

Figures

Figures reproduced from arXiv: 2506.00312 by the authors.

Figure 1
Figure 1. Framework for evaluation of LLMs for generating movie reviews based on movie subtitles and screenplays. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. We then apply the same Sentiment Polarity Analysis [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 2
Figure 2. Polarity score comparison of LLM-generated vs. IMDb user reviews for Q1 (Negative), Q2 (Positive), and Q3 (Neutral) categories. [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figures from the paper (7 more)
Figure 3
Figure 3. Figure 3: Sentiment polarity scores from subtitles and screenplays across LLMs. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png]
Figure 4
Figure 4. Figure 4: Comparison of emotion scores across different movies between LLM-generated reviews and IMDb reviews. tion in LLM-generated reviews are also higher than those in IMDb reviews. These patterns again validate that LLMs tend to amplify emotional intensity when generating re…
Figure 5
Figure 5. Figure 5: Emotion score by LLMs for Q1 (Negative), Q2 (Positive), and Q3 (Neutral) categories. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Emotion scores from subtitles and screenplays across LLMs. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Comparison of Mean Cosine Similarity Scores Across Models Using Subtitles and Screenplays [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Cosine Similarity by sentiment and Question type for movie subtitle-based reviews for respective LLMs [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Cosine Similarity by sentiment and Question type for movie screenplay-based reviews for respective LLMs. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

90 extracted references · 61 canonical work pages

  1. [1]

    Influence of the movies on attitudes and be- havior,

    F. Fearing, “Influence of the movies on attitudes and be- havior,”Journal of Social Issues, vol. 3, no. 3, pp. 70–80, 1947

    1947

  2. [2]

    Tzioumakis and C

    Y . Tzioumakis and C. Molloy,The Routledge Companion to Cinema and Politics. London and New York: Rout- ledge, 2016

    2016

  3. [3]

    Movies, consumption markets & culture,

    F. Kerrigan, “Movies, consumption markets & culture,” Movies, Consumption Markets&Culture, vol. 21, no. 5, pp. 503–509, 2018

    2018

  4. [4]

    S. J. Ross,Movies and American Society. John Wiley & Sons, 2014

    2014

  5. [5]

    The effect of movie portrayals on audience attitudes about nontra- ditional families and sexual orientation,

    M. A. Mazur and T. M. Emmers-Sommer, “The effect of movie portrayals on audience attitudes about nontra- ditional families and sexual orientation,”Journal of Ho- mosexuality, vol. 44, no. 1, pp. 157–181, 2003

    2003

  6. [6]

    Natural language processing,

    K. Chowdhary, “Natural language processing,” inFunda- mentals of Artificial Intelligence. Springer, 2020, pp. 603–649

    2020

  7. [7]

    Deep learning in clinical natural language processing: a methodical review,

    S. D. J. D. Z. J. Y . S. S. S. Q. W. Q. W. Y . X. e. a. S. Wu, K. Roberts, “Deep learning in clinical natural language processing: a methodical review,”American Medical In- formatics Association, vol. 27, no. 3, pp. 457–470, 2020

    2020

  8. [8]

    Machine learning and nat- ural language processing: Review of models and opti- mization problems,

    E. Mankolli and V . Guliashki, “Machine learning and nat- ural language processing: Review of models and opti- mization problems,” inICT Innovations 2020. Machine Learning and Applications: 12th International Confer- ence, ICT Innovations 2020, Skopje, North Macedonia, September 24–26, 2020, Proceedings. Springer Interna- tional Publishing, 2020, pp. 71–86

    2020

Show all 90 references
  1. [9]

    Language models are few-shot learn- ers,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Ka- plan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askellet al., “Language models are few-shot learn- ers,”Advances in Neural Information Processing Systems, vol. 33, no. 1, pp. 1877–1901, 2020

    1901

  2. [10]

    Sci- ence in the age of large language models,

    D. L. A. Birhane, A. Kasirzadeh and S. Wachter, “Sci- ence in the age of large language models,”Nature Reviews Physics, vol. 5, no. 5, pp. 277–280, 2023

    2023

  3. [11]

    Gen- erative artificial intelligence: a systematic review and ap- plications,

    S. S. Sengar, A. B. Hasan, S. Kumar, and F. Carroll, “Gen- erative artificial intelligence: a systematic review and ap- plications,”Multimedia Tools and Applications, pp. 1–40, 2024

    2024

  4. [12]

    Education in the era of generative artificial intelligence (ai): Understanding the potential benefits of chatgpt in promoting teaching and learning,

    D. Baidoo-Anu and L. O. Ansah, “Education in the era of generative artificial intelligence (ai): Understanding the potential benefits of chatgpt in promoting teaching and learning,”Journal of AI, vol. 7, no. 1, pp. 52–62, 2023

    2023

  5. [13]

    The woman worked as a babysitter: Biases in language model generation,

    P. N. E. Sheng, K. W. Chang and N. Peng, “The woman worked as a babysitter: Biases in language model generation,” inProceedings of EMNLP-IJCNLP, 2019, pp. 3407–3412, available at: https://arxiv.org/abs/1909. 01326

    2019

  6. [14]

    A comprehensive survey on process-oriented automatic text summarization with exploration of llm-based methods,

    Y . Zhang, H. Jin, D. Menget al., “A comprehensive survey on process-oriented automatic text summarization with exploration of llm-based methods,”arXiv preprint arXiv:2403.02901, 2024

    2024

  7. [15]

    Large language models penetration in scholarly writing and peer review,

    L. Zhou, R. Zhang, X. Dai, D. Hershcovich, and H. Li, “Large language models penetration in scholarly writing and peer review,”arXiv preprint arXiv:2502.11193, 2025

    2025 arXiv

  8. [16]

    A systematic review of large language models and their implications in medical education,

    H. C. Lucas, J. S. Upperman, and J. R. Robinson, “A systematic review of large language models and their implications in medical education,”Medical Education, vol. 58, no. 11, pp. 1276–1285, 2024

    2024

  9. [17]

    Large language models meet nlp: A survey,

    X. F. Y . W. Y . Z. Y . L. M. L. W. C. L. Qin, Q. Chen and P. S. Yu, “Large language models meet nlp: A survey,” arXiv preprint arXiv:2405.12819, 2024

    2024 arXiv

  10. [18]

    A review on large language models: Architectures, applications, tax- onomies, open issues and challenges,

    K. F. N. M. F. S. S. M. M. J. M. J. A. M. E. A. M. A. K. Raiaan, M. S. H. Mukta and S. Azam, “A review on large language models: Architectures, applications, tax- onomies, open issues and challenges,”IEEE Access, 2024

    2024

  11. [19]

    Language mod- els are unsupervised multitask learners,

    R. C. e. a. A. Radford, J. Wu, “Language mod- els are unsupervised multitask learners,” OpenAI Blog, 2019, available at: https://openai.com/research/ better-language-models

    2019

  12. [20]

    A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,

    K. X. Y . C. Z. S. Y . Z. Y . Yao, J. Duan, “A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,”High-Confidence Computing, vol. 100211, 2024

    2024

  13. [21]

    Crafting clarity: Leveraging large language models to decode consumer reviews,

    S. V . Praveen, P. Gajjar, and R. K. Ray, “Crafting clarity: Leveraging large language models to decode consumer reviews,”Journal of Retailing and Consumer Services, vol. 81, p. 103975, 2024

    2024

  14. [22]

    Consumer evaluations of movies on the basis of critics’ judgments,

    A. D’Astous and N. Touil, “Consumer evaluations of movies on the basis of critics’ judgments,”Psychology& Marketing, vol. 16, no. 8, pp. 677–694, 1999

    1999

  15. [23]

    Dynamic ef- fects among movie ratings, movie revenues, and viewer satisfaction,

    S. Moon, P. K. Bergey, and D. Iacobucci, “Dynamic ef- fects among movie ratings, movie revenues, and viewer satisfaction,”Journal of Marketing, vol. 74, no. 1, pp. 108–121, 2010

    2010

  16. [24]

    Kerrigan,Film Marketing

    F. Kerrigan,Film Marketing. Elsevier, 2017

    2017

  17. [25]

    The halo effect in multicomponent ratings and its implica- tions for recommender systems,

    N. Sahoo, R. Krishnan, G. Duncan, and J. Callan, “The halo effect in multicomponent ratings and its implica- tions for recommender systems,”Information Systems Re- search, vol. 23, no. 1, pp. 231–246, 2012. 19

    2012

  18. [26]

    Understanding the influence of on- line movie reviews on audience reception: A data-driven approach,

    S. Moon and H. Kim, “Understanding the influence of on- line movie reviews on audience reception: A data-driven approach,”Journal of Consumer Research, vol. 45, no. 2, pp. 310–327, 2018

    2018

  19. [27]

    Advancing transformer architecture in long-context large language models: A comprehensive survey,

    Y . Huang, J. Xu, J. Laiet al., “Advancing transformer architecture in long-context large language models: A comprehensive survey,”arXiv preprint arXiv:2311.12351, 2023

    2023 arXiv

  20. [28]

    Longitudinal abuse and sentiment analysis of hollywood movie dialogues us- ing llms,

    R. Chandra, G. Ren, and Group-Ha, “Longitudinal abuse and sentiment analysis of hollywood movie dialogues us- ing llms,”arXiv preprint arXiv:2310.07039, 2025

    2025 arXiv

  21. [29]

    Large lan- guage models: a comprehensive survey of its applications, challenges, limitations, and future prospects,

    M. U. Hadi, R. Qureshi, A. Shahet al., “Large lan- guage models: a comprehensive survey of its applications, challenges, limitations, and future prospects,”Authorea Preprints, vol. 1, pp. 1–26, 2023

    2023

  22. [30]

    Auto- matic scoring of metaphor creativity with large language models,

    P. V . DiStefano, J. D. Patterson, and R. E. Beaty, “Auto- matic scoring of metaphor creativity with large language models,”Creativity Research Journal, pp. 1–15, 2024

    2024

  23. [31]

    Book review: Deep learning,

    K. G. Kim, “Book review: Deep learning,”Healthcare Informatics Research, vol. 22, no. 4, pp. 351–354, 2016

    2016

  24. [32]

    Deep learning,

    Y . LeCun, Y . Bengio, and G. Hinton, “Deep learning,”Na- ture, vol. 521, no. 7553, pp. 436–444, 2015

    2015

  25. [33]

    Long short-term memory,

    S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural Computation, vol. 9, no. 8, pp. 1735– 1780, 1997

    1997

  26. [34]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in Neural Information Processing Systems, vol. 30, 2017

    2017

  27. [35]

    Unlocking the potential of chatgpt: A compre- hensive exploration of its applications, advantages, limi- tations, and future directions in natural language process- ing,

    W. Hariri, “Unlocking the potential of chatgpt: A compre- hensive exploration of its applications, advantages, limi- tations, and future directions in natural language process- ing,”arXiv preprint arXiv:2304.02017, 2023

    2023 arXiv

  28. [36]

    Semantic language mod- els with deep neural networks,

    A. O. Bayer and G. Riccardi, “Semantic language mod- els with deep neural networks,”Computer Speech&Lan- guage, vol. 40, pp. 1–22, 2016

    2016

  29. [37]

    Progress in neural nlp: modeling, learning, and reasoning,

    M. Zhou, N. Duan, S. Liuet al., “Progress in neural nlp: modeling, learning, and reasoning,”Engineering, vol. 6, no. 3, pp. 275–290, 2020

    2020

  30. [38]

    De- pendency parsing with bottom-up hierarchical pointer net- works,

    D. Fernández-González and C. Gómez-Rodríguez, “De- pendency parsing with bottom-up hierarchical pointer net- works,”Information Fusion, vol. 91, pp. 494–503, 2023

    2023

  31. [39]

    Bert: Pre-training of deep bidirectional transformers for lan- guage understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for lan- guage understanding,”arXiv preprint arXiv:1810.04805, 2019

    2019 arXiv

  32. [40]

    Improving language understanding by generative pre- training,

    A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre- training,”OpenAI Blog, vol. 1, no. 1, pp. 1–12, 2018

    2018

  33. [41]

    Why does unsupervised pre-training help deep learning?

    D. Erhan, A. Courville, Y . Bengio, and P. Vincent, “Why does unsupervised pre-training help deep learning?”Pro- ceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, vol. 9, no. 1, pp. 201– 208, 2010

    2010

  34. [42]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, M. Ott, N. Goyalet al., “Roberta: A robustly optimized bert pretraining approach,”arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  35. [43]

    Hollywood movie subtitle dataset: Oscar nom- inations and top 10 blockbusters (1950–2024),

    G. REN, “Hollywood movie subtitle dataset: Oscar nom- inations and top 10 blockbusters (1950–2024),” https: //www.kaggle.com/datasets/mlopssss/subtitles, 2025, ac- cessed: March 2025

    1950

  36. [44]

    Imdb movie reviews grouped by rat- ings,

    ——, “Imdb movie reviews grouped by rat- ings,” https://www.kaggle.com/datasets/mlopssss/ imdb-movie-reviews-grouped-by-ratings, 2025, ac- cessed: March 2025

    2025

  37. [45]

    Subtitling and distribution of international films,

    F. Lastname, “Subtitling and distribution of international films,”Journal of Global Media Studies, vol. 18, no. 3, pp. 120–135, 2020

    2020

  38. [46]

    S. Bird, E. Klein, and E. Loper,Natural Language Pro- cessing with Python. Sebastopol, CA: O’Reilly Media, 2009

    2009

  39. [47]

    A large-scale comparison of historical text normalization systems,

    M. Bollmann, “A large-scale comparison of historical text normalization systems,” inProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, pp. 3885–3898. [Online]. Available: https:/...

    2019

  40. [48]

    C. D. Manning, P. Raghavan, and H. Schütze,Introduction to Information Retrieval. Cambridge, UK: Cambridge University Press, 2008

    2008

  41. [49]

    Gpt-4o technical report,

    OpenAI, “Gpt-4o technical report,” https://openai.com/ index/gpt-4o, 2024, accessed: 2024-05-15

    2024

  42. [50]

    Chatgpt: Optimizing language models for dia- logue,

    ——, “Chatgpt: Optimizing language models for dia- logue,” https://openai.com/blog/chatgpt, 2023, accessed: 2024-05-15

    2023

  43. [51]

    Multitask prompted training en- ables zero-shot task generalization,

    Y . Banget al., “Multitask prompted training en- ables zero-shot task generalization,”arXiv preprint arXiv:2305.03062, 2023

    2023 arXiv

  44. [52]

    Evalplus: Towards comprehensive eval- uation of language models in multilingual and multitask settings,

    Y . Chenet al., “Evalplus: Towards comprehensive eval- uation of language models in multilingual and multitask settings,”arXiv preprint arXiv:2312.05620, 2023

    2023 arXiv

  45. [53]

    Scaling mixture-of-experts for multilingual reasoning,

    W. Zhang, “Scaling mixture-of-experts for multilingual reasoning,”arXiv preprint arXiv:2401.12345, 2024. 20

    2024 arXiv

  46. [54]

    Multi-token prediction improves language modeling,

    Y . Duet al., “Multi-token prediction improves language modeling,”arXiv preprint arXiv:2305.14710, 2023. [Online]. Available: https://arxiv.org/abs/2305.14710

    2023 arXiv

  47. [55]

    A comparative evaluation of gemini and gpt-4 on multimodal educational tasks,

    A. Borji and S. Mohammadian, “A comparative evaluation of gemini and gpt-4 on multimodal educational tasks,” arXiv preprint arXiv:2402.67890, 2024

    2024

  48. [56]

    Llm arena: Benchmarking large lan- guage models for instruction following,

    Y . Lianget al., “Llm arena: Benchmarking large lan- guage models for instruction following,”arXiv preprint arXiv:2402.02837, 2024

    2024 arXiv

  49. [57]

    N-gram-based text categorization,

    W. B. Cavnar, J. M. Trenkleet al., “N-gram-based text categorization,” inProceedings of SDAIR-94, 3rd annual symposium on document analysis and information re- trieval, vol. 161175. Ann Arbor, Michigan, 1994, p. 14

    1994

  50. [58]

    Jurafsky and J

    D. Jurafsky and J. H. Martin,Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models, 3rd ed. Pearson, 2025, online manuscript released January 12, 2025. [Online]. Available: https:/...

    2025

  51. [59]

    Document-level sentiment classification using hybrid machine learning ap- proach,

    A. Tripathy, A. Anand, and S. K. Rath, “Document-level sentiment classification using hybrid machine learning ap- proach,”Knowledge and Information Systems, vol. 53, pp. 805–831, 2017

    2017

  52. [60]

    Syntactic n-grams as machine learning features for natural language processing,

    G. Sidorov, F. Velasquez, E. Stamatatos, A. Gelbukh, and L. Chanona-Hernández, “Syntactic n-grams as machine learning features for natural language processing,”Expert Systems with Applications, vol. 41, no. 3, pp. 853–860, 2014

    2014

  53. [61]

    Sentiment classification with roberta and data augmentation techniques,

    J.-H. Lee and S.-H. Park, “Sentiment classification with roberta and data augmentation techniques,”Applied Sci- ences, vol. 11, no. 4, p. 1644, 2021

    2021

  54. [62]

    A survey on sentiment and emo- tion analysis for computational literary studies,

    Y . Chen and S. Skiena, “A survey on sentiment and emo- tion analysis for computational literary studies,”arXiv preprint arXiv:2009.07297, 2020

    2009 arXiv

  55. [63]

    A statistical interpretation of term specificity and its application in retrieval,

    K. Sparck Jones, “A statistical interpretation of term specificity and its application in retrieval,”Journal of doc- umentation, vol. 28, no. 1, pp. 11–21, 1972

    1972

  56. [64]

    An information-theoretic perspective of tf– idf measures,

    A. Aizawa, “An information-theoretic perspective of tf– idf measures,”Information Processing&Management, vol. 39, no. 1, pp. 45–65, 2003

    2003

  57. [65]

    Tweeteval: Unified benchmark and comparative evaluation for tweet classification,

    F. Barbieri, J. Camacho-Collados, L. Espinosa-Anke, and L. Neves, “Tweeteval: Unified benchmark and comparative evaluation for tweet classification,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 1644–1650. [Online...

    2020

  58. [66]

    Goemotions: A dataset of fine-grained emotions,

    D. Demszky, D. Movshovitz-Attias, J. Ko, A. Cowen, G. Nemade, and S. Ravi, “Goemotions: A dataset of fine-grained emotions,” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), 2020, pp. 4040–4054. [Online]. Available: https://aclan...

    2020

  59. [67]

    Defending against neu- ral fake news,

    R. Zellers, A. Holtzman, H. Rashkin, Y . Bisk, A. Farhadi, F. Roesner, and Y . Choi, “Defending against neu- ral fake news,” inAdvances in Neural Information Processing Systems (NeurIPS), 2019. [Online]. Avail- able: https://papers.nips.cc/paper_files/paper/2019/hash/ 21e1f9a4...

    2019

  60. [68]

    What makes good in-context examples for gpt-3?

    X. Liu, J. Ou, Z. Wang, S. Zhang, X. Wang, and G. Neubig, “What makes good in-context examples for gpt-3?” inProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), 2023, pp. 10 753–10 776. [Online]. Available: https://aclanthology.org/20...

    2023

  61. [69]

    Persona prompt- ing for controllable and diverse text generation,

    S. Lee, D. Kang, M. Park, and K. Cho, “Persona prompt- ing for controllable and diverse text generation,” inFind- ings of ACL 2023, 2023

    2023

  62. [70]

    Chunk-based sentiment analy- sis for long documents using transformer models,

    L. Zhang and K. Wang, “Chunk-based sentiment analy- sis for long documents using transformer models,”IEEE Transactions on Affective Computing, 2022

    2022

  63. [71]

    Learning word vectors for sentiment analysis,

    A. Maas, R. Daly, P. Pham, D. Huang, A. Ng, and C. Potts, “Learning word vectors for sentiment analysis,” inPro- ceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 2011

    2011

  64. [72]

    Explainable prompt learning for movie review sentiment analysis,

    S. Stilwell, “Explainable prompt learning for movie review sentiment analysis,” Master’s the- sis, University of Ottawa, 2024. [Online]. Avail- able: https://ruor.uottawa.ca/server/api/core/bitstreams/ e6811e0b-7aed-46fb-a7a5-f29be4246ca7/content

    2024

  65. [73]

    Emotionally informed language mod- els: A study on emotion bias in text generation,

    J. Hartmannet al., “Emotionally informed language mod- els: A study on emotion bias in text generation,”arXiv preprint arXiv:2303.00001, 2023

    2023 arXiv

  66. [74]

    Enhancing emotional generation capability of large language models with emotional chain- of-thought,

    C. Li, J. Wang, Y . Zhang, K. Zhu, W. Hou, J. Lian, F. Luo, Q. Yang, and X. Xie, “Enhancing emotional generation capability of large language models with emotional chain- of-thought,”arXiv preprint arXiv:2401.06836, 2023

    2023 arXiv

  67. [75]

    More than a feeling: Accuracy and application of emo- tion analysis,

    J. Hartmann, M. Heitmann, C. Siebert, and C. Schamp, “More than a feeling: Accuracy and application of emo- tion analysis,”International Journal of Research in Mar- keting, vol. 39, no. 4, pp. 1205–1223, 2022

    2022

  68. [76]

    Sentiment analysis through llm negotia- tions,

    X. Sun, X. Li, S. Zhang, S. Wang, F. Wu, J. Li, T. Zhang, and G. Wang, “Sentiment analysis through llm negotia- tions,”arXiv preprint arXiv:2311.01876, 2023

    2023 arXiv

  69. [77]

    Audio-visual sentiment analysis for learning emotional arcs in movies,

    W.-T. Chu and D. Roy, “Audio-visual sentiment analysis for learning emotional arcs in movies,” inProceedings of the IEEE International Conference on Data Mining. IEEE, 2017, pp. 829–834. 21

    2017

  70. [78]

    Multimodal deep models for predicting affective responses evoked by movies,

    P. T. Thao, H.-J. Yang, and S. Kim, “Multimodal deep models for predicting affective responses evoked by movies,”IEEE Access, vol. 7, pp. 209 616–209 628, 2019

    2019

  71. [79]

    Can ai write a movie review? a comparative study of human and machine criticism,

    Y . Zhang and H. Liu, “Can ai write a movie review? a comparative study of human and machine criticism,” Journal of Media Studies, vol. 12, no. 3, pp. 45–62, 2023

    2023

  72. [80]

    All that’s ’hu- man’ is not gold: Evaluating human evaluation of gener- ated text,

    E. Clark, A. Celikyilmaz, and N. A. Smith, “All that’s ’hu- man’ is not gold: Evaluating human evaluation of gener- ated text,”Transactions of the Association for Computa- tional Linguistics, vol. 9, pp. 929–944, 2021

    2021

  73. [81]

    Longgen- bench: Benchmarking long-form generation in long con- text llms,

    Y . Wu, M. S. Hee, Z. Hu, and R. K.-W. Lee, “Longgen- bench: Benchmarking long-form generation in long con- text llms,”arXiv preprint arXiv:2409.02076, 2024

    2024 arXiv

  74. [82]

    Sen- timent analysis in the era of large language models: A reality check,

    W. Zhang, Y . Deng, B. Liu, S. J. Pan, and L. Bing, “Sen- timent analysis in the era of large language models: A reality check,”arXiv preprint arXiv:2305.15005, 2023

    2023 arXiv

  75. [83]

    Multimodal sentiment analysis: a survey of methods, trends, and challenges,

    R. Das and T. D. Singh, “Multimodal sentiment analysis: a survey of methods, trends, and challenges,”ACM Com- puting Surveys, vol. 55, no. 13s, pp. 1–38, 2023

    2023

  76. [84]

    Multimodal few-shot learning with frozen language models,

    M. Tsimpoukelli, J. L. Menick, S. Cabi, S. Eslami, O. Vinyals, and F. Hill, “Multimodal few-shot learning with frozen language models,”Advances in Neural Infor- mation Processing Systems, vol. 34, pp. 200–212, 2021

    2021

  77. [85]

    Large language models meet text-centric multimodal sentiment analysis: A sur- vey,

    H. Yang, Y . Zhao, Y . Wu, S. Wang, T. Zheng, H. Zhang, Z. Ma, W. Che, and B. Qin, “Large language models meet text-centric multimodal sentiment analysis: A sur- vey,”arXiv preprint arXiv:2406.08068, 2024

    2024 arXiv

  78. [86]

    A review on methods and applications in mul- timodal deep learning,

    S. Jabeen, X. Li, M. S. Amin, O. Bourahla, S. Li, and A. Jabbar, “A review on methods and applications in mul- timodal deep learning,”ACM Transactions on Multimedia Computing, Communications and Applications, vol. 19, no. 2s, pp. 1–41, 2023

    2023

  79. [87]

    Blend: A benchmark for llms on everyday knowl- edge in diverse cultures and languages,

    J. Myung, N. Lee, Y . Zhou, J. Jin, R. Putri, D. Antypas, H. Borkakoty, E. Kim, C. Perez-Almendros, A. A. Ayele et al., “Blend: A benchmark for llms on everyday knowl- edge in diverse cultures and languages,”Advances in Neu- ral Information Processing Systems, vol. 37, pp. 78 ...

    2024

  80. [88]

    Sur- vey of cultural awareness in language models: Text and beyond,

    S. Pawar, J. Park, J. Jin, A. Arora, J. Myung, S. Yadav, F. G. Haznitrama, I. Song, A. Oh, and I. Augenstein, “Sur- vey of cultural awareness in language models: Text and beyond,”arXiv preprint arXiv:2411.00860, 2024

    2024 arXiv

  81. [89]

    Bertaqa: How much do language models know about local culture?

    J. Etxaniz, G. Azkune, A. Soroa, O. Lacalle, and M. Artetxe, “Bertaqa: How much do language models know about local culture?”Advances in Neural Infor- mation Processing Systems, vol. 37, pp. 34 077–34 097, 2024

    2024

  82. [90]

    Are large language model-based evaluators the solution to scaling up multilingual evaluation?

    R. Hada, V . Gumma, A. de Wynter, H. Diddee, M. Ahmed, M. Choudhury, K. Bali, and S. Sitaram, “Are large language model-based evaluators the solution to scaling up multilingual evaluation?”arXiv preprint arXiv:2309.07462, 2023. Appendix A: Detailed Prompt Templates for LLM-Gen...

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.