REVIEW 6 cited by
Sentiment Analysis in the Era of Large Language Models: A Reality Check
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Sentiment analysis (SA) has been a long-standing research area in natural language processing. It can offer rich insights into human sentiments and opinions and has thus seen considerable interest from both academia and industry. With the advent of large language models (LLMs) such as ChatGPT, there is a great potential for their employment on SA problems. However, the extent to which existing LLMs can be leveraged for different sentiment analysis tasks remains unclear. This paper aims to provide a comprehensive investigation into the capabilities of LLMs in performing various sentiment analysis tasks, from conventional sentiment classification to aspect-based sentiment analysis and multifaceted analysis of subjective texts. We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets. Our study reveals that while LLMs demonstrate satisfactory performance in simpler tasks, they lag behind in more complex tasks requiring deeper understanding or structured sentiment information. However, LLMs significantly outperform SLMs in few-shot learning settings, suggesting their potential when annotation resources are limited. We also highlight the limitations of current evaluation practices in assessing LLMs' SA abilities and propose a novel benchmark, \textsc{SentiEval}, for a more comprehensive and realistic evaluation. Data and code during our investigations are available at \url{https://github.com/DAMO-NLP-SG/LLM-Sentiment}.
Forward citations
Cited by 6 Pith papers
-
Towards Compute-Optimal Many-Shot In-Context Learning
Hybrid demonstration selection that adds 20 similar examples to a large cached random or k-means set matches or beats similarity-only selection at up to 10x lower estimated inference cost in many-shot ICL.
-
Evaluating Apple Intelligence's Writing Tools for Privacy Against Large Language Model-Based Inference Attacks: Insights from Early Datasets
Apple Intelligence's Friendly and Professional text rewrites can substantially reduce LLM-based emotion inference accuracy in small early datasets.
-
Learning to Select Visual In-Context Demonstrations
A Dueling-DQN agent selects visual in-context demonstrations and outperforms kNN retrieval on objective regression benchmarks but not on subjective preference tasks, per the paper's main table.
-
Small Encoders Can Rival Large Decoders in Detecting Groundedness
Task-specific encoders (e.g., RoBERTa-large) rival large decoders such as Llama-3-8B and GPT-4o on binary groundedness detection, within 5 to 10 accuracy points while requiring one to three orders of magnitude fewer FLOPs.
-
Towards Robust Fact-Checking: A Multi-Agent System with Advanced Evidence Retrieval
A multi-agent LLM pipeline with credibility-filtered full-text web retrieval reports better fact-checking F1 than four baselines on small benchmark subsamples.
-
Hallucination Detection and Mitigation in Scientific Text Simplification using Ensemble Approaches: DS@GT at CLEF 2025 SimpleText
An ensemble of BERT, cosine similarity, NLI, and LLM judgment with a meta-classifier detects distorted simplifications, while an LLM post-editor trades simplicity for faithfulness.
Discussion (0). Sign in to comment.