{"id":"28f3ab83-b2d8-43d2-9864-e4b75182cc9a","arxiv_id":"1909.00415","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A bag-of-seed-words teacher distills soft labels into a neural student, and iterative co-training reweights noisy seed words, beating prior weakly supervised aspect detectors by about 14 F1 points on average.","lead":"This paper trains aspect classifiers for product and restaurant reviews using only a small set of seed words per aspect, through a teacher-student setup and iterative co-training. Across 12 datasets, the approach reports consistent F1 improvements over prior weakly supervised methods, with BERT students leading the gains.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Co-training's seed-quality estimates come from the student's own predictions, so the reported iterative gains may reflect self-confirmation rather than correction of noisy seed words.","rationale":"The reader's weakest-assumption analysis identifies the same vulnerability that I consider most load-bearing: Eq. (4) uses the student's own hard predictions to reweight seed words, creating a self-referential loop. I agree that the single-round student-teacher comparison is well isolated from this concern, and the headline 14.1 F1 number is largely a single-round result. However, the paper's title, Algorithm 1, and the third stated contribution are explicitly about iterative co-training, so the soundness of this bootstrap is central to the full claim even if it is not central to the headline number. The paper provides no evidence that the estimated z_j values correspond to true seed-word quality: no oracle comparison, no correlation analysis, and no experiment where the student's predictions are replaced with independent labels. The stopping rule based on student-teacher agreement makes the confirmatory loop especially hard to detect. I therefore keep the reader's CONDITIONAL verdict: the concern is real and testable, but it does not by itself invalidate the paper's strongest empirical claim. A single controlled experiment comparing ISWD's quality estimates with gold-derived oracle estimates would settle whether the concern lands. No changes to the verdict are needed pending that test; the paper should remain CONDITIONAL on this verification.","tokens_in":14687,"tokens_out":15373,"duration_ms":148827,"concrete_test":"Using the labeled validation/test segments, compute the true precision of each seed word (the fraction of segments containing that word whose gold aspect equals the aspect to which the word was assigned). Correlate these true precisions with the ISWD-estimated z_j after rounds 1 and 2. Then run the teacher (Eq. 3) with oracle z_j values computed from the gold labels and compare the final Student-BERT F1 against Student-BERT-ISWD on the same test folds. If the correlation is low, or if oracle weighting outperforms ISWD by more than about 1 F1 point, the co-training gains are not attributable to valid estimation of seed quality; if the correlation is high and oracle weighting gives no material gain, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Algorithm 1, Eq. (4) estimates each seed word's quality z_j by the distribution of the student's hard predictions t_i = argmax_k p_k^i over training segments containing that seed word. The student was itself trained to mimic the teacher via Eq. (2), and the teacher is built entirely from the same seed words. Therefore z_j is not an independent estimate of seed-word quality; it is a function of the model's own beliefs. If the student inherits a systematic error from the teacher (for example, an incorrect seed-word-to-aspect assignment), Eq. (3) will upweight that error, and co-training can converge to a self-consistent but wrong fixed point. The stopping criterion (student-teacher disagreement stops decreasing) is exactly what one would observe at such a fixed point, so it does not establish correctness. The single-round Student-BERT result that drives the 14.1 F1 headline may survive, but the paper's title and its third contribution—iterative co-training to cope with noisy seed words—rest on this bootstrap. The reported co-training gains in Table 5 are modest (about 0.6 to 3.0 F1 for the student) and no oracle comparison or correlation analysis is given, so the mechanism is experimentally unverified. This is the load-bearing weak point for the full proposed method, even though the headline comparison is less dependent on it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies fine-grained aspect detection with no ground-truth labels, using a small set of seed words per aspect. It introduces a teacher model, a bag-of-seed-words classifier that produces soft aspect labels, and a student network (with BOW, word2vec, attention, or BERT encoders) trained to mimic the teacher via a distillation objective. It then proposes iterative co-training (ISWD), where seed-word quality weights are estimated from the student's hard predictions and fed back into a reweighted teacher. The method is evaluated on six OPOSUM product-review domains and six SemEval restaurant-review languages. The paper reports consistent improvements over prior weakly supervised baselines, an average gain of 14.1 F1 points for Student-BERT, and a 62% reduction of the gap to fully supervised models. Ablations, including a no-seed-words student variant and iterative co-training results, are also reported.","tokens_in":14907,"tokens_out":9755,"duration_ms":85439,"significance":"If the results hold, this is a practically useful advance: it demonstrates that a few dozen seed words per aspect can supervise neural aspect detectors that substantially outperform previous weakly supervised methods and close a large portion of the gap to supervised models, across domains and languages. The core single-round comparison is well controlled: Student-ATT and MATE-unweighted share the same seed words, pre-trained embeddings, segment encoder, and classifier, isolating the effect of using the teacher's soft labels as supervision. The consistent direction of improvement across all 12 datasets is a strong point, and the paper is transparent that it does not beat fully supervised models and that non-English restaurant experiments use a single co-training round due to limited training data. The main weaknesses are the unvalidated self-training mechanism for seed-quality estimation and the lack of variance and significance reporting.","major_comments":[{"comment":"The co-training claim in Section 3.3 rests on a self-training bootstrap. Equation (4) estimates each seed word's quality z_j from the student's hard predictions t_i on segments containing that seed word, but the student is itself trained to mimic the teacher (Eq. (2)), and the teacher is defined entirely by the seed words. Thus z_j is a function of the model's own beliefs rather than an independent estimate of seed-word quality; a systematic teacher error (for example, a wrong seed-word-to-aspect assignment) will be upweighted by Eq. (3), and the stopping rule based on student-teacher disagreement cannot distinguish convergence to a correct solution from convergence to a self-consistent fixed point. The modest gains in Table 5 and Figure 4 are therefore not sufficient evidence for the paper's third contribution that iterative co-training 'copes with noisy seed words.' I ask for a concrete validation of the mechanism: correlate the estimated z_j with oracle seed-word precisions computed from held-out labels, compare teachers trained with estimated weights against one trained with oracle weights, and if possible include a control in which the student's hard labels are replaced by an independent noisy-label estimator.","section":"Section 3.3, Algorithm 1, Eq. (4)"},{"comment":"The central quantitative claim is reported without variance or significance information. The text says results are averaged over 5 runs with the configuration that achieves the best validation performance, but only mean micro-F1 is shown in Tables 3 and 4, with no standard deviations, per-run values, or paired tests across datasets. Because the paper claims the method 'consistently outperforms' previous approaches by a large average margin, the absence of error bars or a significance test makes it hard to assess the stability of the improvements. The direction of improvement is uniform across all 12 datasets in the single-round comparison, which is reassuring, but the paper should report standard deviations and, ideally, a paired test (for example, a Wilcoxon signed-rank test over the 12 datasets) for the main comparisons.","section":"Section 4.1 and Section 4.2, Tables 3 and 4"}],"minor_comments":[{"comment":"The sentence 'Student-ATT outperforms MATE-unweighted (by 36.3% in product reviews and by 52.2% in restaurant reviews)' is arithmetically inconsistent with Tables 3 and 4. For restaurant reviews, the relative increase of Student-ATT over MATE-unweighted is (44.9 - 26.1)/26.1 = 72.0%, not 52.2%; 52.2% is the relative increase over MATE (29.5). Please correct the sentence or explicitly state that the restaurant comparison is against MATE.","section":"Section 4.2, paragraph on prior weakly supervised models"},{"comment":"The baseline for the headline 'average absolute increase of 14.1 F1 points' is not stated in the same sentence. Please specify whether the comparison is to MATE, MATE-MT, MATE-unweighted, or the best prior baseline per dataset, and give the per-dataset differences so that the average can be reproduced.","section":"Section 4.2, final paragraph"},{"comment":"The paper states that MATE uses the validation set (with aspect labels) to estimate seed weights, while the proposed method is unsupervised. However, Section 4.1 also selects the student's parameter configuration using the labeled validation set. This is a common practice, but it should be stated explicitly as a limitation of the overall label-free claim.","section":"Section 4.1 and Section 3.3"},{"comment":"The claim that 'the co-training assumption of conditionally independent views is satisfied' for Student-W2V-RSW is not justified: although the student sees only non-seed words and the teacher sees only seed words, the student is trained on the teacher's soft labels, so the two views are not conditionally independent given the label in the sense of Blum and Mitchell (1998). Please rephrase this claim.","section":"Section 4.2 and Figure 3"},{"comment":"The reference 'Srivastava and Sutton, 2071' should be 2017, and the rendering of 'Buciluˇa et al.' includes a stray formatting character. Also, the symbols T_i and S_i in Figure 4 are not defined in the caption or text.","section":"References and Figure 4 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is an extended version of a workshop paper, and the controlled single-round result is a solid contribution that should not be rejected. The main risk is that the title and the third claimed contribution rest on the unvalidated self-training bootstrap in Eq. (4). If the authors can provide the requested oracle validation or reframe the contribution around single-round distillation, the paper will be much stronger. I would not oppose acceptance after such a revision, but the current evidence for the co-training mechanism is insufficient."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core idea is simple and it works: treat each seed word as a noisy annotator, let a bag-of-seed-words teacher soft-label unlabeled segments, and train a neural student on those soft labels. That single-round student-teacher step is the real contribution. The controlled comparison between Student-ATT and MATE-unweighted uses the same seed words, embeddings, and classifier head, so the difference isolates the weak supervision signal. The gains are consistent across six product domains and six languages, and the 14.1-point average improvement over MATE is plausible given the gap between a teacher that actually uses seed words and a model that only initializes aspect embeddings with them.\n\nThe co-training extension is the soft spot, and the stress-test note lands: Eq. (4) estimates seed-word quality from the student's own hard predictions, and that student was trained to mimic the teacher built from those same seeds. So the quality weights are a self-bootstrap, not an independent estimate. The stopping rule (disagreement stops decreasing) would also hold at a self-consistent but wrong fixed point. The table shows modest co-training gains of roughly 0.6 to 3.0 F1, and there is no oracle comparison or correlation analysis to show the weights actually track true seed quality. This is a genuine weakness, but it is not load-bearing for the main result. The single-round student-teacher numbers stand on their own, and the paper would be almost as valuable without the iterative step.\n\nMinor issues: no code or checkpoints, no error bars on the 5-run averages, and validation labels are used for hyperparameter selection, which slightly undercuts the \"no labels\" framing. None of these break the central claim, but they make the precise magnitudes hard to verify from the manuscript alone.\n\nThis paper is for researchers and practitioners in weakly supervised NLP and opinion mining. It deserves a serious referee: the method is simple, the experiments are broad, and the controlled comparison is a good template for the field. I would engage with it in a reading group and would cite it if I were working on label-light aspect detection.","headline":"A clean student-teacher trick for weakly supervised aspect detection that mostly earns its 14-point F1 claim; the co-training bonus is real but self-referential and secondary.","tokens_in":15472,"tokens_out":2264,"would_cite":true,"duration_ms":21681,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A weakly supervised student-teacher method using about 30 seed words per aspect beats prior weakly supervised aspect detectors by 14.1 average F1 points and closes 62 percent of the supervised gap.","keywords":["weakly supervised learning","aspect detection","seed words","student-teacher models","co-training","knowledge distillation","opinion mining","multilingual aspect detection"],"falsifier":"Take a domain from the paper, run the first co-training round, and on the training segments that contain a given seed word compare the student's hard prediction with the true aspect label; if for several seed words the student is wrong on a majority of those segments, then the next teacher round should decrease rather than increase F1. The same design could be tested by adding deliberately misleading seed words (for example, 'great' assigned to every aspect) and checking whether the reported two-to-three-round improvements reverse.","tokens_in":14406,"feed_emoji":"🎯","tokens_out":8492,"duration_ms":68355,"temperature":0.7,"pith_summary":"The paper asks whether a few descriptive keywords (seed words) per aspect can replace the thousands of fine-grained labels normally needed to train aspect detectors for product and restaurant reviews. It argues that earlier weakly supervised methods underuse seed words, and proposes a student-teacher pipeline: a bag-of-seed-words classifier labels unlabeled review segments, then a neural student is trained to imitate those soft labels. The student can exploit non-seed words and pre-trained embeddings, so it generalizes beyond the teacher. An iterative co-training step estimates the quality of each seed word from the student's own predictions, which improves both models. Across six product-review domains and six languages of restaurant reviews, the method reports an average gain of 14.1 F1 points over previous weakly supervised approaches and closes 62 percent of the gap to fully supervised models.","feed_headline":"Few seed words beat weakly supervised baselines by 14.1 F1 points","feed_subtitle":"Student-teacher co-training turns about 30 keywords per aspect into detectors across 12 datasets and six languages.","key_machinery":"The load-bearing mechanism is a two-model loop: a bag-of-seed-words teacher that scores a segment by the counts of seed words it contains, and a student network trained to match the teacher's soft predictions while also seeing the full segment text. A second component is the seed-word quality vector $\\hat{z}_j$ from Equation (4), which reweights seed words according to agreement between the seed word's presence and the student's predicted aspect. This reweighted teacher (Equation 3) turns noisy seed words into graded supervision. The loop is the co-training step that propagates improvements back and forth.","core_discovery":"The central claim is that seed words are best treated not as anchors for initializing topic embeddings but as direct, noisy supervision signals. The teacher is a softmax over per-aspect seed-word counts (Equation 1), producing soft aspect labels for unlabeled segments; the student is any embedding-based classifier trained with the cross-entropy distillation objective (Equation 2). Because the student sees all words, it learns to associate non-seed words with aspects and outperforms the teacher. Iterative co-training then reweights each seed word by its estimated predictive quality (Equation 4), using the student's hard predictions as a proxy for ground truth, and this improved teacher in turn trains a better student. The paper reports consistent gains over earlier weakly supervised methods and a 62 percent reduction of the performance gap to fully supervised models on the 12 datasets.","pith_inferences":["The student-teacher recipe is task-agnostic: any classification problem whose classes can be described by a few keywords, such as document tagging or review usefulness prediction, could use the same teacher and distillation loop.","The quality estimate in Equation (4) is a self-training bootstrap, so its success likely depends on seed words being reasonably distinctive; on domains with heavy lexical overlap between aspects, confidence-weighted or soft student predictions may be more stable.","A natural testable extension is to replace the hard argmax used in seed-quality estimation with teacher-student agreement or prediction confidence, which could further stabilize co-training for low-resource languages.","The reported gap-closing of 62 percent is measured against a fixed label budget; combining this method with a small number of labeled segments might push weakly supervised detectors past the best supervised baselines in data-scarce settings."],"forward_implications":["Detectors for new product domains can be trained from roughly 30 seed words per aspect without collecting segment-level labels.","The same recipe works across languages, since the teacher is language-agnostic and the student can use multilingual pretrained embeddings.","Stronger student backbones directly improve the weakly supervised result: BERT-based students give the best reported F1.","Noisy seed words need not be cleaned by hand; iterative co-training recovers much of their value within two to three rounds.","Weak supervision with seed words becomes a practical alternative when fully supervised models are infeasible, cutting the performance gap to supervised training by 62 percent."],"supporting_citations":[{"why":"Supplies the MATE baseline, the OPOSUM product-review datasets, the 30 seed words per aspect, and the preprocessing/evaluation protocol used throughout.","marker":"Angelidis and Lapata (2018)"},{"why":"Provides the co-training principle that teacher and student can mutually improve by agreeing on unlabeled data.","marker":"Blum and Mitchell (1998)"},{"why":"Gives the distillation objective used to train the student from the teacher's soft predictions.","marker":"(Hinton et al., 2015)"},{"why":"Contributes the MBEM iterative estimation procedure the paper adapts to estimate seed-word quality from student predictions.","marker":"Khetan et al. (2018)"},{"why":"Supplies the BERT contextual embeddings used by the best-performing student configuration.","marker":"Devlin et al. (2019)"},{"why":"Supplies the ABAE unsupervised baseline and the attention-based segment embedding used by Student-ATT.","marker":"He et al. (2017)"},{"why":"Provides the six multilingual restaurant-review datasets and the 12-aspect annotation scheme used for evaluation.","marker":"Pontiki et al. (2016)"},{"why":"Supplies the LDA-Anchors baseline that uses seed words as anchors in topic modeling.","marker":"Lund et al. (2017)"}],"fun_headline_variants":["Seed words as supervision: 14.1 F1 gain over weak baselines","Student-teacher co-training with seed words beats weak baselines","Just 30 keywords per aspect: 14.1 F1 improvement","Co-training seed words boosts aspect detection across 12 datasets","Aspect detection from seed words: 14.1 F1 over weak supervision"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The iterative step assumes that the student's own hard predictions on unlabeled training segments are accurate enough to serve as ground truth when estimating each seed word's quality; if the student is systematically wrong on segments containing a seed word, the reweighted teacher will amplify that error.","fun_headline_variants_meta":{"raw":{"variants":["Seed words as supervision: 14.1 F1 gain over weak baselines","Student-teacher co-training with seed words beats weak baselines","Just 30 keywords per aspect: 14.1 F1 improvement","Co-training seed words boosts aspect detection across 12 datasets","Aspect detection from seed words: 14.1 F1 over weak supervision"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1446,"prompt_tokens":970,"completion_tokens":476,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":381}},"tokens_in":586,"tokens_out":476,"duration_ms":5162,"temperature":1.0,"reasoning_tokens":381,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:53:19.389912+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a domain from the paper, run the first co-training round, and on the training segments that contain a given seed word compare the student's hard prediction with the true aspect label; if for several seed words the student is wrong on a majority of those segments, then the next teacher round should decrease rather than increase F1. The same design could be tested by adding deliberately misleading seed words (for example, 'great' assigned to every aspect) and checking whether the reported two-to-three-round improvements reverse.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MATE baseline, the OPOSUM product-review datasets, the 30 seed words per aspect, and the preprocessing/evaluation protocol used throughout."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the co-training principle that teacher and student can mutually improve by agreeing on unlabeled data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes the MBEM iterative estimation procedure the paper adapts to estimate seed-word quality from student predictions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the BERT contextual embeddings used by the best-performing student configuration."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the six multilingual restaurant-review datasets and the 12-aspect annotation scheme used for evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the LDA-Anchors baseline that uses seed words as anchors in topic modeling."}],"review_version":1}