REVIEW 4 major objections 5 minor 24 references
Extracting Aspects Hierarchies using Rhetorical Structure Theory
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that unsupervised extraction of aspect hierarchies from product reviews, using rhetorical structure theory, reproduces human-coded ConceptNet relations with about 80% coverage on the top 50 aspect pairs.
desk verdict The RST-based aspect hierarchy idea is plausible, but the paper's headline 80% ConceptNet coverage is not yet evidence because the support set is broad and there is no baseline. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Aspect Discourse Tree, built from a Rhetorical Structure Theory discourse tree by replacing each Elementary Discourse Unit with the aspect detected inside it and deleting nucleus-nucleus edges. A breadth-first search over every Aspect Discourse Tree generates directed tuples of the form (nucleus aspect, satellite aspect), and counting these tuples across all reviews produces a ranked list that is merged greedily into a hierarchy: a nucleus already present becomes the parent of the new satellite, and a missing nucleus starts a sub-hierarchy that may later be attached when a tuple links it to the main tree. This machinery converts a discourse-level claim, that a satellite specifies its nucleus, into a graph-level counting operation, and the aggregation over many documents is what lets noisy individual reviews average out.
What would settle it
Compute the same coverage after replacing each extracted nucleus-satellite pair with random aspect pairs drawn from the same reviews and matched in frequency; if the random baseline's coverage approaches 80%, the reported agreement is co-occurrence rather than discourse. A second decisive check is to compute coverage separately for each RST relation type and verify that elaboration-style relations carry the signal while contrast and sequence relations do not.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that discourse subordination tracks aspect specificity: in an RST relation, the nucleus is the more central part and the satellite is subsidiary, and when the two units mention different aspects, the satellite's aspect tends to be a feature or sub-aspect of the nucleus's aspect. Aggregating such pairs across 100,000 Amazon cell-phone reviews, the method builds an aspect tree, with phone at the root and children like case, battery, headset, bluetooth, and price, and with battery in turn as the parent of charge. The evaluation uses coverage $c=|A\cap C|/|A|$, where $A$ is the set of aspect hierarchy tuples generated by the method and $C$ is the set of ConceptNet hierarchical relations reachable in at most three hops. All measured subsets of the top aspect pairs score above 75%, with the top 5 at 100% and the top 50 at 80%.
Load-bearing premise
The method assumes that when one discourse unit is rhetorically subordinate to another, the aspect it mentions is a more specific sub-aspect of the nucleus aspect; if this specificity mapping fails, the extracted trees are just counted co-occurrences.
Editorial extensions
If this is right
- Aspect hierarchies can be produced with no labeled training data, needing only a discourse segmenter and a noun-phrase extractor.
- Sentiment can be attached at any depth of the hierarchy, so a system can report that 'charging' is a positive aspect of battery but also of price, depending on context.
- Because the same aspect can appear as a child under several parents, the representation explicitly models context-dependent meaning rather than forcing each aspect into one node.
- The stable 75–100% coverage across top-5 to top-50 pairs indicates that the discourse signal is not confined to the most frequent and obvious aspect pairs.
Reading between the lines
- I would not read 80% coverage as exact agreement: the ConceptNet reference set is closed under three hops and includes Synonym and RelatedTo, so the metric is a compatibility check, and a stricter one-hop test would likely score lower.
- A direct test of the paper's hypothesis would separate RST relation types: if only elaboration-like relations carry specificity, filtering the tuple list by relation type should preserve coverage while shrinking the set; if all relation types contribute equally, the mechanism is closer to raw co-occurrence.
- The same pipeline should transfer to other review domains, but the strength of the discourse signal may vary with text formality; testing on book or restaurant reviews would show whether the 80% figure is specific to hardware-like product descriptions.
- The authors themselves note that there is no existing aspect hierarchy to compare against, which is why the evaluation uses ConceptNet as a proxy; building a small hand-labeled aspect hierarchy from the same reviews would turn the compatibility metric into a direct accuracy measure.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an unsupervised approach to extracting hierarchical aspect structures from product reviews using Rhetorical Structure Theory (RST). The method segments each review into elementary discourse units, detects noun-phrase aspects, replaces EDUs with the detected aspects, and then builds an Aspect Discourse Tree. The authors hypothesize that the nucleus-satellite relation between discourse units reflects a hierarchical (general-to-specific) relation between the corresponding aspects. All nucleus-satellite aspect pairs are collected across many documents and then assembled into a hierarchy by repeatedly adding satellites as children of their nuclei. The evaluation compares the most frequent aspect pairs against ConceptNet, counting a pair as correct if the two concepts are connected in ConceptNet within a graph distance of three hops, using relations such as PartOf, HasA, IsA, Synonym, and RelatedTo. The paper reports coverage values between 0.75 and 1.00 for the top 5 to 50 aspect pairs, and claims that the method generates aspect hierarchies consistent with human-generated hierarchies.
Significance. If the central claim were established, the paper would make a useful contribution: it offers a simple, unsupervised technique that connects discourse structure with aspect-based sentiment analysis, and the idea of deriving aspect hierarchies from RST relations is novel. The use of a large public Amazon review dataset and the emphasis on an unsupervised pipeline are also strengths. However, the current evaluation is not sufficient to support the claim. The reported coverage figures are computed against a broad and permissive ConceptNet support set, and no baselines or control conditions are provided. As a result, the evidence does not discriminate between the proposed RST-based method and a much simpler aspect co-occurrence model. The central claim is therefore plausible but not yet demonstrated; the paper requires a substantial additional evaluation effort to become convincing.
major comments (4)
- [Sec. 4.2, Eq. (1)] The coverage metric uses a support set C that includes all concept pairs connected in ConceptNet by a path of up to three hops, with relation types including Synonym and RelatedTo, and with no requirement that the direction of the ConceptNet path match the nucleus-to-satellite order in the extracted tuple. This is a permissive oracle: frequently co-occurring product aspects (e.g., phone, battery, charger) are very likely to be within three hops in ConceptNet regardless of whether their discourse relation encodes specificity. Consequently, the reported 80% coverage on the top 50 pairs may simply reflect that the method's frequent pairs are generic semantic associates. To make the claim 'consistent with human-generated hierarchies' substantive, the authors should report (i) a random-pair baseline matched on aspect frequency, (ii) a co-occurrence-based aspect pair extractor as a non-RST baseline, and (iii) the coverage obtained when the ConceptNet path direction is required to align with the nucleus-satellite direction.
- [Sec. 3.3] The hierarchy construction algorithm is described only in prose and is under-specified. It is not clear how the tuple list is ordered after aggregation (e.g., by raw frequency?), how cycles such as (case, phone) and (phone, case) are resolved, what happens when a satellite already has children, or how sub-hierarchies are merged when a later tuple connects two existing nodes. The example in Listing 1 and the textual walk-through do not uniquely define the output hierarchy. Moreover, the evaluation in Section 5 only checks pairwise coverage, never the correctness of the overall hierarchy (e.g., whether the final tree has intuitive root and depth structure). A formal algorithm (pseudocode) and an evaluation of the constructed hierarchy itself, not just its constituent pairs, are needed to substantiate the paper's title claim.
- [Sec. 4] The experimental pipeline is not reproducible as described. The authors state that they used [20] for rhetorical segmentation and Spacy for noun-phrase extraction, but the component that assigns RST relation labels and determines nuclearity - which is essential for generating the nucleus-satellite tuples in Section 3.3 - is not identified anywhere in the paper. Without specifying the discourse parser (or the rule set used to infer nuclei and relations), a reader cannot reproduce the experiments or determine whether the results depend on a particular parser's accuracy. The authors should name the parser (e.g., a specific RST discourse parser and its version), the relation inventory, and how nuclearity was obtained.
- [Sec. 5, Table 2] The coverage results are reported without any statistical context. The values for the top 20 to 50 range from 0.75 to 0.82 with no confidence intervals, no significance test, and no null model. Given the permissive nature of the support set, the null hypothesis 'randomly chosen aspect pairs from the same frequency distribution achieve similar coverage' may well hold. The authors should provide: (i) a frequency-matched random baseline with error bars, (ii) the exact list of the top 50 pairs and whether each is counted due to a direct relation, a multi-hop path, or a Synonym/RelatedTo bridge, and (iii) a test of whether coverage is significantly above the baseline.
minor comments (5)
- [Abstract] The phrase 'proved to be consistently correct' is too strong for an evaluation based on coverage of a subset of pairs; suggest 'is consistent with' or 'agrees with'.
- [Sec. 4.2] Typo: 'ConcepetNet' should be 'ConceptNet'.
- [Sec. 5] Figure 4 is not described in the text; the reader cannot tell what aspects are shown or how the hierarchy was generated from the example.
- [Sec. 1] The introduction does not state the contributions of the paper explicitly; a short list of contributions would help the reader.
- [Sec. 4.2] The sentence 'the distance for antenna and telephone will be 2, because of the relations antenna PartOf receiver and receiver PartOf telephone' is a useful example but the text does not explain how the direction of the path is handled when computing coverage; please clarify.
Circularity Check
No significant circularity: the RST-derived aspect pairs are scored against an independent ConceptNet reference, and the 3-hop expansion, while permissive, does not make the evaluation an identity or a fitted prediction.
full rationale
The paper's central derivation is an unsupervised pipeline: RST discourse trees are parsed, EDUs are replaced by extracted aspects, nucleus-satellite pairs are counted, and the most frequent pairs seed a hierarchy. The evaluation compares these pairs (set A) against ConceptNet tuples (set C), which is an external, independently constructed knowledge graph. There is no equation in which an output is defined in terms of the reference set or in which a fitted parameter is later reported as a prediction. The 3-hop distance threshold in Eq. (1) is a broad inclusion criterion for ConceptNet, not a parameter fitted to the RST output; it may inflate coverage and weaken the benchmark, but it does not make A and C equivalent by construction. The authors' self-citation [1] is used for prior POS-based aspect extraction and as background, not as the load-bearing justification for the hierarchy claim. The main weakness is evidentiary rather than circular: no co-occurrence or random-pair baseline is provided, so the 80% coverage figure is not yet a clean demonstration that RST relations specifically encode aspect specificity. That concern belongs to correctness or experimental design, not to circularity.
Assumptions & free parameters
free parameters (1)
- ConceptNet distance threshold =
3
assumptions (4)
- domain assumption The RST discourse parser (Feng and Hirst [20]) produces reliable discourse trees.
- domain assumption Noun phrases extracted from EDUs are the correct product aspects.
- ad hoc to paper RST nucleus-satellite relations encode hierarchical aspect relationships.
- domain assumption ConceptNet relations within up to three hops are a valid gold standard for aspect hierarchy correctness.
Cite this review
Pith. "Pith review of Extracting Aspects Hierarchies using Rhetorical Structure Theory." pith.science (2026). https://pith.science/paper/7XGWRFLV
@misc{pith2026190901800,
author = {Pith},
title = {Pith review of: Extracting Aspects Hierarchies using Rhetorical Structure Theory},
year = {2026},
howpublished = {\url{https://pith.science/paper/7XGWRFLV}},
note = {Machine review of arXiv:1909.01800}
}
read the original abstract
We propose a novel approach to generate aspect hierarchies that proved to be consistently correct compared with human-generated hierarchies. We present an unsupervised technique using Rhetorical Structure Theory and graph analysis. We evaluated our approach based on 100,000 reviews from Amazon and achieved an astonishing 80% coverage compared with human-generated hierarchies coded in ConceptNet. The method could be easily extended with a sentiment analysis model and used to describe sentiment on different levels of aspect granularity. Hence, besides the flat aspect structure, we can differentiate between aspects and describe if the charging aspect is related to battery or price.
Figures
Reference graph
Works this paper leans on
-
[20]
Two-pass discourse segmentation with pairing and global features
Vanessa Wei Feng and Graeme Hirst. Two-pass discourse segmentation with pairing and global features. CoRR, abs/1407.8215, 2014. 7 Extracting Aspects Hierarchies using Rhetorical Structure Theory A PREPRINT
arXiv 2014
-
[1]
Method for aspect-based sentiment annotation using rhetorical analysis
Łukasz Augustyniak, Krzysztof Rajda, and Tomasz Kajdanowicz. Method for aspect-based sentiment annotation using rhetorical analysis. In Ngoc Thanh Nguyen, Satoshi Tojo, Le Minh Nguyen, and Bogdan Trawi´nski, editors, Intelligent Information and Database Systems, pages 772–781, Cham, 2017. Springer International Publishing
work page 2017
-
[2]
A Bayesian Model for Joint Unsupervised Induction of Sentiment, Aspect and Discourse Representations
Angeliki Lazaridou, Ivan Titov, and Caroline Sporleder. A Bayesian Model for Joint Unsupervised Induction of Sentiment, Aspect and Discourse Representations. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1630–1639, 2013
work page 2013
-
[3]
Discourse indicators for content selection in summarization
Annie Louis, Aravind Joshi, and Ani Nenkova. Discourse indicators for content selection in summarization. In Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue , SIGDIAL ’10, pages 147–156, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics
work page 2010
-
[4]
Using Discourse Structure Improves Ma- chine Translation Evaluation
Francisco Guzm ´an, Shafiq Joty, Lluis M `arquez, and Preslav Nakov. Using Discourse Structure Improves Ma- chine Translation Evaluation. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 687–698, 2014
work page 2014
-
[5]
J.R. Martin. English Text. John Benjamins Publishing Company, Amsterdam, nov 1992
work page 1992
-
[6]
D-STAG: A formalism for discourse analysis based on SDRT and using synchronous TAG
Laurence Danlos. D-STAG: A formalism for discourse analysis based on SDRT and using synchronous TAG. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 5591 LNAI:64–84, 2011
work page 2011
-
[7]
Rhetorical structure theory: Toward a functional theory of text organization
William C Mann and Sandra A Thompson. Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243–281, 1988
work page 1988
Show all 24 references
-
[8]
Discourse markers as signals (or not) of rhetorical relations
Maite Taboada. Discourse markers as signals (or not) of rhetorical relations. Journal of Pragmatics, 38(4):567– 592, 2006
2006
-
[9]
A hierarchical aspect-sentiment model for online reviews
Suin Kim, Jianwen Zhang, Zheng Chen, Alice Oh, and Shixia Liu. A hierarchical aspect-sentiment model for online reviews. In Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence , AAAI’13, pages 526–533. AAAI Press, 2013
2013
-
[10]
Sentiment learning on product reviews via sentiment ontology tree
Wei Wei and Jon Atle Gulla. Sentiment learning on product reviews via sentiment ontology tree. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 404–413, 2010
2010
-
[11]
Aspect Extraction for Opinion Mining with a Deep Convolutional Neural Network
Soujanya Poria, Erik Cambria, and Alexander Gelbukh. Aspect Extraction for Opinion Mining with a Deep Convolutional Neural Network. Knowledge-Based Systems, 108:42–49, sep 2016
2016
-
[12]
Dependency tree-based rules for concept-level aspect-based sentiment analysis
Soujanya Poria, Nir Ofek, Alexander Gelbukh, Amir Hussain, and Lior Rokach. Dependency tree-based rules for concept-level aspect-based sentiment analysis. In Valentina Presutti, Milan Stankovic, Erik Cambria, Iv ´an Cantador, Angelo Di Iorio, Tommaso Di Noia, Christoph Lange, ...
2014
-
[13]
Dlirec: Aspect term extraction and term polarity classification system
Zhiqiang Toh and Wenting Wang. Dlirec: Aspect term extraction and term polarity classification system. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 235–240. Asso- ciation for Computational Linguistics, 2014
2014
-
[14]
Sebastian Ruder, Parsa Ghaffari, and John G. Breslin. A hierarchical model of reviews for aspect-based sentiment analysis. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 999–1005. Association for Computational Linguistics, 2016
2016
-
[15]
Blei, Thomas L
David M. Blei, Thomas L. Griffiths, and Michael I. Jordan. The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies. J. ACM, 57(2):7:1–7:30, February 2010
2010
-
[16]
Modeling topic hierarchies with the recursive chinese restaurant process
Joon Hee Kim, Dongwoo Kim, Suin Kim, and Alice Oh. Modeling topic hierarchies with the recursive chinese restaurant process. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM ’12, pages 783–792, New York, NY , USA, 2012. ACM
2012
-
[17]
Shafiq Joty, Giuseppe Carenini, and Raymond T. Ng. CODRA : A Novel Discriminative Framework for Rhetor- ical Analysis. Computational Linguistics, 41(January):1–50, sep 2015
2015
-
[18]
SemEval- 2015 Task 12: Aspect Based Sentiment Analysis
Maria Pontiki, Dimitrios Galanis, Haris Papageorgiou, Suresh Manandhar and Ion Androutsopoulos. SemEval- 2015 Task 12: Aspect Based Sentiment Analysis. Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Denver, Colorado., pages 486–495, 2015
2015
-
[19]
All maximal independent sets and dynamic dominance for sparse graphs
David Eppstein and David. All maximal independent sets and dynamic dominance for sparse graphs. ACM Transactions on Algorithms, 5(4):1–14, oct 2009
2009
-
[21]
Ups and Downs: : Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering
Ruining He and Julian McAuley. Ups and Downs: : Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering. In Proceedings of the 25th International Conference on World Wide Web - WWW ’16, pages 507–517, New York, New York, USA, 2016. ACM Press
2016
-
[22]
Ng, and Bita Nejat
Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T. Ng, and Bita Nejat. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Do...
2014
-
[23]
Representing general relational knowledge in conceptnet 5
Robert Speer and Catherine Havasi. Representing general relational knowledge in conceptnet 5. In Proceed- ings of the Eighth International Conference on Language Resources and Evaluation (LREC-2012) . European Language Resources Association (ELRA), 2012
2012
-
[24]
Sentiment Aggregation using ConceptNet Ontology
Subhabrata Mukherjee and Sachindra Joshi. Sentiment Aggregation using ConceptNet Ontology. pages 570– 578, 2013. 8
2013
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.