Learning to Extract Cross-Domain Aspects and Understanding Sentiments Using Large Language Models

Chiranjib Sur; Karukriti Kaushik Ghosh

REVIEW 4 major objections 6 minor 12 references

Learning to Extract Cross-Domain Aspects and Understanding Sentiments Using Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that an LLM-assisted aspect-understanding step allows a single BERT classifier to reach about 92 percent accuracy on SemEval-2015 Task 12 and to transfer between Laptop and Restaurant reviews without domain-specific…

desk verdict Unverifiable cross-domain ABSA numbers and an under-specified method make this a desk-reject, despite a sensible underlying heuristic. read the letter →

arxiv 2501.08974 v1 pith:N5XXGC2B submitted 2025-01-15 cs.CL

Karukriti Kaushik Ghosh , Chiranjib Sur This is my paper · ORCID

classification cs.CL

keywords Aspect-basedsentimentanalysisCross-domainLargelanguagemodelsBERTfine-tuningSemEval-2015Task12AspectextractionclassificationDomainadaptation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that aspect-based sentiment analysis can be made cheaper and more portable by having a large language model provide cross-domain aspect understanding before a single BERT classifier predicts aspect sentiment. The authors' strongest claim is that this pipeline reaches about 92 percent accuracy on the SemEval-2015 Task 12 dataset and that cross-domain performance (88.9 to 90.4 percent) no longer requires fine-tuning BERT on each target domain. A business could therefore train once on one product category and reuse the model on another, avoiding per-domain annotation and expensive fine-tuning.

What carries the argument

The load-bearing machinery is the two-stage pipeline: first, a large language model, aided by external knowledge sources, identifies aspects and bridges domain-specific vocabulary; second, a pre-trained 12-layer BERT is fine-tuned once on a source domain and then applied to the target domain without further fine-tuning. The LLM step is what the authors say removes the need for per-domain fine-tuning, because it supplies the cross-domain understanding that the BERT classifier would otherwise have to learn from labeled target-domain data. The joint aspect-and-sentiment ground truth format of the SemEval-2015 dataset is what the accuracy numbers measure.

What would settle it

Re-run the pipeline on the original, unmodified SemEval-2015 Task 12 test set with the standard evaluation split and compare accuracy with the reported 92.1 percent and 88.9 percent; if the gap is large, the reported gain comes from the unstated dataset modification rather than from the LLM step.

Watch

Extended reading notes

Core claim

The central discovery is that injecting external knowledge through a large language model before classification lets a 12-layer BERT model transfer across domains. On SemEval-2015 Task 12, the authors report 92.1 percent for Laptop reviews when BERT is trained on Laptop and 91.4 percent for Restaurant when trained on Restaurant; when BERT is trained on Laptop and tested on Restaurant, accuracy is 88.9 percent, and training on Restaurant and testing on Laptop gives 90.4 percent. Training on both domains yields 91.1 percent for Laptop and 90.6 percent for Restaurant. These results are compared with a deep memory network baseline of 72.21 percent for Laptop and 80.95 percent for Restaurant. The authors attribute the gain to the LLM bridging the terminology gap between domains, so that the BERT model does not need to be fine-tuned for every domain.

Load-bearing premise

The reported numbers all depend on the assumption that the modified SemEval-2015 dataset still measures the original aspect and sentiment labels, and the paper never describes how the dataset was modified.

Editorial extensions

If this is right

A business could train an ABSA pipeline on one product category and apply it to another without collecting target-domain annotations or fine-tuning BERT.
Cross-domain transfer works in both directions between Laptop and Restaurant, with reported accuracy between 88.9 percent and 90.4 percent depending on direction.
Training on both domains yields more balanced performance, around 91 percent for each domain, so additional data still helps even when transfer is strong.
The reported accuracy exceeds the deep memory network baseline by roughly 10 to 20 percentage points, suggesting the LLM-assisted pipeline is a meaningful improvement over that earlier approach.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves the dataset-modification procedure unstated, so a reader cannot yet tell whether the 92 percent is a property of the method or of the altered evaluation data; this is an inference, not a result the paper establishes.
The same pipeline could likely be tested on other domain pairs, such as electronics to fashion or hospitality, because the authors identify their experiment as a proof of concept with only the two SemEval domains available.
Because only a 12-layer BERT is fine-tuned, the approach suggests that small, cheap classifiers can be reused across domains as long as an LLM front-end provides aspect knowledge; the paper does not compare cost or latency.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a thin engineering write-up whose only concrete claim—92% accuracy via an LLM-assisted BERT pipeline—is unverifiable because the evaluation data was modified without any description and the method itself is barely specified. There is no reproducible protocol, no comparison with any modern LLM-based ABSA work, and no code or data release.

To give credit where it is earned: the idea of using an LLM to bridge vocabulary across domains and then feeding distilled aspects to a lightweight BERT classifier is a reasonable engineering heuristic. If it works, it would indeed make cross-domain ABSA cheaper to deploy. The paper also clearly describes the business motivation and honestly notes the SemEval-2015 dataset has only two domains, reporting both intra- and cross-domain numbers in a single table. The authors seem to understand the practical upside of reducing per-domain fine-tuning.

But the soft spots are load-bearing. The dataset was "modified" but the modification is never described, so the reported numbers cannot be checked against the public benchmark. The LLM prompts, model version, decoding settings, and exact train/test splits are all absent. The only baseline is a 2016 deep memory network, so there is no evidence this is competitive with current LLM or transformer-based ABSA systems. The paper also contradicts itself about fine-tuning: the abstract says "cross-domain does not require finetuning," but Section 5 says "we can fine-tune the model on labeled data once." If the LLM that generated aspects also helped create or relabel the modified dataset, the 92% could be measuring agreement with the authors' own annotations rather than performance on the original task. The mentions of bias masking, data augmentation, and fairness are dropped without any detail. These are not minor gaps; they undermine the central claim.

As written, this is a position summary or a preliminary industry note, not a research paper. A business reader might find the architecture sketch useful, but a researcher cannot build on it because nothing is specified precisely enough to replicate.

I would not send this to peer review as is. It needs a complete rewrite with a real experimental protocol: the unmodified SemEval-2015 split, error bars, and at least a couple of recent LLM-based baselines. If the authors can confirm the numbers on the public test set and release code or prompts, the result might be worth revisiting. Until then, desk reject is appropriate.

Referee Report

4 major / 6 minor

Summary. The paper proposes an aspect-based sentiment analysis (ABSA) framework that uses large language models (LLMs) for aspect extraction and a 12-layer BERT model for sentiment classification, with the goal of enabling cross-domain ABSA without per-domain fine-tuning. The authors claim to achieve 92.1% accuracy on the SemEval-2015 Task 12 dataset for the in-domain Laptop case and 88.9-90.4% accuracy in cross-domain settings, comparing against a Deep Memory Network baseline. The central empirical claim rests on a version of the SemEval-2015 dataset that the authors say they modified, but the modification procedure is never described. The paper also reports no experimental protocol, such as train/test splits, number of runs, hyperparameters, or details of the LLM used. Internal contradictions about whether fine-tuning is required further obscure what was actually evaluated. Overall, the manuscript presents an interesting practical direction but currently fails to provide a verifiable or reproducible evaluation.

Significance. If the claims were fully substantiated, the work would offer a practical cross-domain ABSA pipeline that reduces the need for expensive per-domain annotation and fine-tuning, which would be a useful contribution for applied sentiment analysis. The paper does not ship code, data, or a detailed protocol, and the reported 92.1% accuracy is tied to an undisclosed dataset modification, so the central result cannot be checked against the public SemEval-2015 benchmark. The absence of any error analysis or variance reporting also means the reader cannot assess the reliability of the numbers. The idea of using LLMs to bridge domain terminology is sensible, but as presented the empirical contribution is unverifiable.

major comments (4)

[§1, contribution 2] The paper lists as a contribution: "We modified the existing Aspect Based Sentiment Analysis dataset of SemEval-2015 Task 12," but nowhere in Sections 4 or 5 is the modification procedure described. Since Table 1 reports accuracy on this modified dataset, the reader cannot determine whether the 92.1% figure reflects the official SemEval annotations or a relabeled/reduced version of the data. This is load-bearing: the central claim of the paper is unverifiable without knowing exactly what was changed, why, and whether the modified dataset is available for inspection.
[§5, Table 1] Table 1 reports single accuracy numbers with no number of runs, standard deviations, train/test splits, hyperparameters, or details of the LLM used (the text only says "like llama"). The phrase "LLMs for Aspect" does not specify which model version, how aspects were extracted, whether the LLM saw the test data, or how the BERT model was fine-tuned. Consequently, the experiments are not reproducible and the claimed cross-domain improvements cannot be independently checked.
[§5, first paragraph] The text states "we can fine-tune the model on labeled data once" and later "the performance of the cross-domain aspects improved significantly without the requirement to fine-tune the BERT models," while the abstract and Section 1 claim no fine-tuning is required. This is a direct contradiction about the experimental procedure. It is unclear whether the 92.1% row in Table 1 was obtained with or without fine-tuning, which is essential for interpreting the cross-domain claim.
[§1 and §5] The paper does not rule out a circular evaluation: if the same LLM that generated or extracted aspects was also used to create or relabel the modified dataset, then the reported accuracy could measure agreement with the authors' own annotations rather than performance on the public benchmark. The authors must describe the dataset modification process, state whether the original test annotations were preserved, and ideally release the modified dataset and code so that this concern can be resolved.

minor comments (6)

[Abstract] The abstract contains a grammatical error: "it is possible to that at an effectiveness of 92% accuracy" should be rephrased; also "ASBA" is a typo for "ABSA."
[§3] The description of BERT's architecture is textbook material and does not explain how the proposed framework is implemented; Figure 1 is not referenced in the text and its connection to the method is unclear.
[§4] The methodology mentions "masking some of the critical works during training" and "bias mitigation algorithms," but no such experiments are described or reported in Section 5.
[§5] The text states that "Accuracy and F1-score" are used, but Table 1 only reports accuracy, and the paper does not provide the class distribution that would support the claim that the dataset is well-balanced.
[References] Several citations are mismatched: for example, [8] is cited for customer feedback but is the BERT paper, and [11] is cited for BERT's architecture but is the BART paper; the reference list needs a careful revision.
[§6] The conclusion claims "we have demonstrated" the effectiveness of the approach, but given the missing experimental protocol and dataset description, this overstates what the paper actually shows.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports benchmark accuracies without a derivation chain that reduces to its inputs, and the modified-dataset caveat is a reproducibility concern, not an exhibited circular step.

full rationale

The paper contains no equations and no fitted-parameter-as-prediction mechanism; the central claim is an empirical accuracy comparison (Table 1: 92.1% Laptop and 88.9–91.4% Restaurant for LLM-assisted BERT vs. 82.3/81.5 for plain BERT and 72.21/80.95 for Deep Memory Network [12]). There are no load-bearing self-citations: reference [12] is Tang, Qin, and Liu, not the present authors, and no uniqueness theorem or prior-work ansatz is imported. The only close-to-circular possibility is that the authors "modified the existing Aspect Based Sentiment Analysis dataset of SemEval-2015 Task 12" (Section 1) and then evaluated on that modified benchmark; if the same LLM had generated or relabeled the gold aspects and sentiments, the reported accuracy would measure self-consistency. However, the paper never states that the LLM produced the test annotations, and Section 4 mentions only re-weighting, data augmentation, and bias mitigation as dataset adjustments. Without an exhibited reduction of the evaluation labels to the model's own outputs, this remains an undisclosed-modification/reproducibility issue, not a circularity that can be scored under the rules. The derivation is self-contained in the sense that all reported numbers are experimental outcomes of a stated pipeline on a named benchmark; no prediction is defined in terms of its own fit.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numerical free parameters are declared in the text. The assumptions that carry the central claim are about the validity of the modified dataset and the ability of an unspecified LLM to provide domain knowledge. No new entities are introduced.

assumptions (4)

domain assumption The SemEval-2015 Task 12 dataset is well-balanced, so accuracy is an appropriate evaluation metric.
Section 4 states "Since the dataset is well-balanced, accuracy is used," but no class distribution is given.
ad hoc to paper The authors' modifications to the SemEval-2015 dataset preserve the meaning of annotated aspects and sentiments.
Section 1 contribution 2 says the dataset was modified, but the modification procedure is never described.
domain assumption An LLM such as Llama can supply external domain knowledge that bridges terminology differences between Laptop and Restaurant reviews.
Sections 3 and 5 rely on this bridge to explain cross-domain transfer; no prompt or knowledge source is specified.
domain assumption A 12-layer BERT fine-tuned once can serve both aspect extraction and sentiment classification.
Section 4 describes the BERT architecture as the model for both tasks, with no evidence that one fine-tuned model handles both well.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning to Extract Cross-Domain Aspects and Understanding Sentiments Using Large Language Models." pith.science (2026). https://pith.science/paper/N5XXGC2B

@misc{pith2026250108974,
  author       = {Pith},
  title        = {Pith review of: Learning to Extract Cross-Domain Aspects and Understanding Sentiments Using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N5XXGC2B}},
  note         = {Machine review of arXiv:2501.08974}
}

read the original abstract

Aspect-based sentiment analysis (ASBA) is a refined approach to sentiment analysis that aims to extract and classify sentiments based on specific aspects or features of a product, service, or entity. Unlike traditional sentiment analysis, which assigns a general sentiment score to entire reviews or texts, ABSA focuses on breaking down the text into individual components or aspects (e.g., quality, price, service) and evaluating the sentiment towards each. This allows for a more granular level of understanding of customer opinions, enabling businesses to pinpoint specific areas of strength and improvement. The process involves several key steps, including aspect extraction, sentiment classification, and aspect-level sentiment aggregation for a review paragraph or any other form that the users have provided. ABSA has significant applications in areas such as product reviews, social media monitoring, customer feedback analysis, and market research. By leveraging techniques from natural language processing (NLP) and machine learning, ABSA facilitates the extraction of valuable insights, enabling companies to make data-driven decisions that enhance customer satisfaction and optimize offerings. As ABSA evolves, it holds the potential to greatly improve personalized customer experiences by providing a deeper understanding of sentiment across various product aspects. In this work, we have analyzed the strength of LLMs for a complete cross-domain aspect-based sentiment analysis with the aim of defining the framework for certain products and using it for other similar situations. We argue that it is possible to that at an effectiveness of 92\% accuracy for the Aspect Based Sentiment Analysis dataset of SemEval-2015 Task 12.

Figures

Figures reproduced from arXiv: 2501.08974 by the authors.

**Figure 1.** Diagram illustrating how specific knowledge source or knowledge graph combined with Large Language Models can help in understanding the specifics of a product and the sentiments of the reviews can be extracted with more precision. specific aspects or features related to the subject being discussed, such as product quality, customer service, pricing, or delivery [1]. Each aspect is then evaluated for sentiment, offe… view at source ↗

**Figure 2.** Diagram illustrating the Domain Adaptation Experiment for Aspect-Based Sentiment Analysis (ABSA). It shows the process of training the ABSA model on Domain A (e.g., restaurant reviews) and then adapting it to Domain B (e.g., laptop reviews) without further fine-tuning. The diagram emphasizes the transfer of learned knowledge, with components such as aspect extraction, sentiment classification, and domain-specific ad… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 7 canonical work pages

[1]

T., & Szolovits, P

Jin, D., Jin, Z., Zhou, J. T., & Szolovits, P. (2020, April). Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 05, pp. 8018-8025)

work page 2020
[2]

(2007, June)

Blitzer, J., Dredze, M., & Pereira, F. (2007, June). Biographies, bollywood, boom- boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th annual meeting of the association of computational linguistics (pp. 440- 447)

work page 2007
[3]

Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., & Smith, N. (2020). Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305

arXiv 2020
[4]

Elsayed, G., Krishnan, D., Mobahi, H., Regan, K., & Bengio, S. (2018). Large margin deep networks for classification. Advances in neural information processing systems, 31

work page 2018
[5]

& Lempitsky, V

Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., ... & Lempitsky, V. (2016). Domain-adversarial training of neural networks. Journal of machine learning research, 17(59), 1-35

work page 2016
[6]

Gunel, B., Du, J., Conneau, A., & Stoyanov, V. (2020). Supervised contrastive learn- ing for pre-trained language model fine-tuning. arXiv preprint arXiv:2011.01403

arXiv 2020
[7]

B., & Guestrin, C

Johnson, T. B., & Guestrin, C. (2018). Training deep models faster with robust, approximate importance sampling. Advances in Neural Information Processing Sys- tems, 31

work page 2018
[8]

Kenton, J. D. M. W. C., & Toutanova, L. K. (2019, June). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT (Vol. 1, p. 2)

work page 2019

Show all 12 references

[9]

Khandelwal, U., Fan, A., Jurafsky, D., Zettlemoyer, L., & Lewis, M. (2020). Nearest neighbor machine translation. arXiv preprint arXiv:2010.00710

2020 arXiv
[10]

& Krishnan, D

Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., ... & Krishnan, D. (2020). Supervised contrastive learning. Advances in neural information processing systems, 33, 18661-18673

2020
[11]

Lewis, M. (2019). Bart: Denoising sequence-to-sequence pre-training for nat- ural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461

2019 arXiv
[12]

Tang, D., Qin, B., & Liu, T. (2016). Aspect level sentiment classification with deep memory network. arXiv preprint arXiv:1605.08900

2016 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

T., & Szolovits, P

Jin, D., Jin, Z., Zhou, J. T., & Szolovits, P. (2020, April). Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 05, pp. 8018-8025)

work page 2020

[2] [2]

(2007, June)

Blitzer, J., Dredze, M., & Pereira, F. (2007, June). Biographies, bollywood, boom- boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th annual meeting of the association of computational linguistics (pp. 440- 447)

work page 2007

[3] [3]

Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., & Smith, N. (2020). Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305

arXiv 2020

[4] [4]

Elsayed, G., Krishnan, D., Mobahi, H., Regan, K., & Bengio, S. (2018). Large margin deep networks for classification. Advances in neural information processing systems, 31

work page 2018

[5] [5]

& Lempitsky, V

Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., ... & Lempitsky, V. (2016). Domain-adversarial training of neural networks. Journal of machine learning research, 17(59), 1-35

work page 2016

[6] [6]

Gunel, B., Du, J., Conneau, A., & Stoyanov, V. (2020). Supervised contrastive learn- ing for pre-trained language model fine-tuning. arXiv preprint arXiv:2011.01403

arXiv 2020

[7] [7]

B., & Guestrin, C

Johnson, T. B., & Guestrin, C. (2018). Training deep models faster with robust, approximate importance sampling. Advances in Neural Information Processing Sys- tems, 31

work page 2018

[8] [8]

Kenton, J. D. M. W. C., & Toutanova, L. K. (2019, June). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT (Vol. 1, p. 2)

work page 2019