On SemEval 2025 Task 11 short texts, generated data helped some BERT models, continued pretraining was mixed, and classification head changes barely mattered.
Contextual Sentence Classification: Detecting Sustainability Initiatives in Company Reports
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
We introduce the novel task of detecting sustainability initiatives in company reports. Given a full report, the aim is to automatically identify mentions of practical activities that a company has performed in order to tackle specific societal issues. New methods for identifying continuous sentence spans need to be developed for capturing the multi-sentence structure of individual sustainability initiatives. We release a new dataset of company reports in which the text has been manually annotated with sustainability initiatives. We also evaluate different models for initiative detection, introducing a novel aggregation and evaluation methodology. Our proposed architecture uses sequences of consecutive sentences to account for contextual information when making classification decisions at the individual sentence level.
citation-role summary
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Optimizing Small Transformer-Based Language Models for Multi-Label Sentiment Analysis in Short Texts
On SemEval 2025 Task 11 short texts, generated data helped some BERT models, continued pretraining was mixed, and classification head changes barely mattered.