{"id":"07e402b5-488c-4173-8ccd-b919ffff2f31","arxiv_id":"1908.02579","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Using class labels as Doc2Vec document contexts to fine-tune pretrained word embeddings improves text classification accuracy on four of five tested datasets.","lead":"This paper fine-tunes pre-trained word vectors by treating the text's class label as a context in a Doc2Vec-style model, then uses the refined vectors in a simple LSTM classifier. It reports accuracy gains on four of five Arabic and English text classification tasks and releases code on GitHub.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported gains may stem from in-domain fine-tuning, not from class-conditioned Doc2Vec: no control removes class labels while keeping all other training steps identical.","rationale":"I read the paper as a proposal for class-conditioned fine-tuning. The method is simple and reproducible, and the code release is a plus. However, the experimental setup does not isolate the proposed variable. The improvement could come from additional in-domain training. Because the abstract makes a mechanistic claim, a control is needed. This is not an accusation; it is a standard ablation. The reader's weakest_assumption concerned the plausibility of a single class vector; my concern is that even if the vector is plausible, it is not measured. The proposed test would settle it. The verdict stays CONDITIONAL because the method may still be useful, but the central explanation is currently underdetermined.","tokens_in":9216,"tokens_out":6216,"duration_ms":62246,"concrete_test":"On SST-2 or the Arabic SemEval-2017 dataset, reproduce the fine-tuning with identical hyperparameters (Doc2Vec, 300-dim, 10 epochs, same pretrained vectors) but change only the document tags: (a) one constant tag for all training instances, (b) a unique tag per instance. Train the same LSTM classifier on each resulting embedding set and compare accuracy/F1 on the same held-out test set. If (a) or (b) performs as well as the class-tagged version, the improvement is not attributable to the class label, and the paper's explanation would need revision; if class-tagged clearly wins, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that supplying the class label as Doc2Vec's document tag (Section 4, steps 6–8) makes embeddings more discriminative and thereby improves classification. The experiments only compare 'pretrained' vs. 'pretrained + proposed fine-tuning'. They never compare against fine-tuning on the same target-domain corpus without using the class label, e.g., Doc2Vec with a single shared document ID for all instances, or with unique per-instance document IDs. Because the fine-tuning procedure also trains on the target-domain text for 10 epochs, the observed improvements in Tables 2–5 could be entirely due to standard domain adaptation of generic embeddings to the in-domain vocabulary and style, not to the class context. The mechanism stated in the abstract—class as an additional contextual variable bringing class-distinctive words closer—is therefore not causally identified. The paper's heuristics (observations 1–3) also provide no direct check that class-specific words actually move closer. This is the weakest point of the argument: the experiments are consistent with the proposed explanation but equally consistent with a simpler domain-adaptation explanation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method for fine-tuning pre-trained word embeddings for text classification: it trains a Doc2Vec model on the target task's training data, using the class label of each text instance as the 'document' ID, and then uses the resulting word vectors as input to a simple LSTM classifier. The method is evaluated on three Arabic datasets (emotion detection, SemEval 2017 Arabic sentiment, SemEval 2018 Arabic emotion) and two English datasets (SST-1, SST-2), comparing RNNs initialized with raw pre-trained embeddings (AraVec or Google News) against RNNs initialized with the proposed fine-tuned embeddings. The authors report accuracy or F1 improvements in four of the five tasks and a small degradation on the multilabel Arabic emotion task, which they attribute to the method being less suited to multilabel classification.","tokens_in":9564,"tokens_out":3234,"duration_ms":37655,"significance":"If the central mechanism were established, the method would be a simple and useful recipe for adapting generic embeddings to low-resource classification tasks, and the public code release and inclusion of a negative multilabel result are strengths. However, the experimental design does not currently isolate the proposed class-conditioning mechanism from generic in-domain fine-tuning, and the reported claims of 'significant' improvement are not supported by statistical testing. The paper's significance is therefore conditional on adding the missing control experiments and embedding-space analyses. The honest reporting of the multilabel failure is commendable and provides a falsifiable boundary for the method.","major_comments":[{"comment":"The experiments compare only 'pre-trained embeddings' against 'pre-trained embeddings fine-tuned with the proposed class-as-document approach'. There is no control condition in which Doc2Vec is trained on the same target-domain text for the same number of epochs without using the class label, e.g., with a single shared document ID for all instances or with unique per-instance document IDs. Because the fine-tuning procedure also trains on the target-domain corpus for 10 epochs, the observed improvements in Tables 2-5 could be wholly due to standard domain adaptation of the embeddings to the in-domain vocabulary and style. This is the load-bearing weakness: the experiments are consistent with the proposed explanation but equally consistent with a simpler domain-adaptation explanation. Please add the missing controls for at least the Arabic emotion dataset (Table 2) and SST-2 (Table 4).","section":"Section 4, steps 6-8; Tables 2-5"},{"comment":"All experimental results are reported as single runs without error bars, confidence intervals, or statistical significance tests. The statement in Experiment 1 that 'fine-tuned AraVec significantly improves the results' is therefore not supported, and the cross-system comparisons in Tables 3 and 5 (e.g., claiming that the proposed method outperforms published systems) are not accompanied by any measure of variance. Please report means and standard deviations over multiple random seeds (at least 5) and, where feasible, a paired significance test between the baseline and proposed conditions.","section":"Section 5, Experiments 1-5"},{"comment":"Observations 1-3 assert that exclusive class words become more affected by their class, that shared words are less affected, and that higher-frequency classes dominate; these are central to the proposed mechanism but are never directly verified. The paper provides no intrinsic evaluation showing that class-distinctive words actually move closer to each other in the embedding space after fine-tuning. Please add a direct embedding-space analysis, such as measuring the change in average cosine distance between class-distinctive word pairs (or nearest-neighbor overlap) before and after fine-tuning, for one or two datasets. This would also help distinguish the class-conditioning explanation from the domain-adaptation alternative.","section":"Section 4, Observations 1-3"},{"comment":"Comparisons against previously published systems are not apples-to-apples with respect to the classifier architecture, training data (e.g., the SST-1 training set is augmented with phrases in Experiment 2), or hyperparameters. The claim in Experiment 2 that 'the proposed method outperforms all the systems cited' is a claim about the combination of the proposed embeddings and the authors' RNN, not about the embeddings alone. Please either include baselines using the same RNN architecture with standard fine-tuned embeddings (e.g., the classifier itself fine-tuning the embeddings during training, as in Kim's non-static CNN) or temper the claim accordingly.","section":"Tables 3-5"}],"minor_comments":[{"comment":"In the second paragraph, 'Doc2Vec is a similar approach that was introduces' should read 'was introduced'.","section":"Section 2"},{"comment":"Equation 1 is rendered with garbled symbols and fractions; please typeset it properly and define all variables in the surrounding text.","section":"Equation 1"},{"comment":"The sentence 'The baseline experiment results with AraVec were not as good as the top performer model in task 4 – subtask A (NileTMRG [22]), but the proposed methodology achieved an improvement with a 0.621 F1 average score (in SemEval.' is incomplete and appears to be missing a closing parenthesis and end-of-sentence punctuation.","section":"Section 5, Experiment 4"},{"comment":"Table 5 contains an unfinished parenthetical '(in SemEval.' in the paragraph above the table; please correct the sentence.","section":"Table 5"},{"comment":"Figure 1 is not explicitly referenced in the text and its caption ('CBOW like Doc2Vec') would benefit from a hyphen ('CBOW-like') and a clearer explanation of how the class ID enters the prediction.","section":"Figure 1"},{"comment":"Reference formatting is inconsistent (e.g., [10] lacks proper title formatting, and several references omit venue details); please unify the bibliography style.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short empirical contribution with a plausible but unverified mechanism. The most serious concern is the absence of a control for domain adaptation, which currently prevents the central claim from being accepted. The manuscript also lacks statistical rigor and would benefit from a more careful framing of cross-system comparisons. Given the availability of code and the straightforward nature of the missing experiments, this is fixable within the scope of a revision, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the method is a one-line trick—replace Doc2Vec's paragraph ID with the class label—and the experiments mostly show it helps, but the paper claims a mechanism it doesn't isolate.\n\nThe genuinely new bit is using class labels as the document tag in Doc2Vec. I checked the related work; none of the cited fine-tuning methods do exactly this, and the closest (Labutov and Lipson) re-embeds words via logistic regression. The experiments cover five datasets, two languages, and the authors provide code. The gains are real in the narrow sense: fine-tuned AraVec beats untuned AraVec on three Arabic tasks, and Google News beats itself on SST after fine-tuning. The authors were also honest about the one failure (multilabel), and their explanation that class-level context is too coarse for overlapping labels is plausible.\n\nThe weak spot, which the stress-test note flags, is real: there is no control that fine-tunes on the same target-domain corpus without class labels. So the improvement could be standard domain adaptation—bringing generic embeddings closer to the in-domain vocabulary—not the class-conditioning. The paper never measures whether class-specific words actually move closer. The observations in Section 4 are heuristic, not verified. That missing control is load-bearing because the abstract's \"class as additional contextual variable\" sentence is an explanation, not a demonstrated cause.\n\nThe second weakness is statistical: no error bars, no significance tests, and one set of baselines is from published systems with different architectures (Kim's CNN etc.), so cross-table comparisons are not apples-to-apples. The consistent 2-3 point gains within the same RNN are encouraging, but without variance estimates I can't tell how robust they are.\n\nWho is this for? Anyone looking for a cheap, incremental trick for supervised classification with pretrained embeddings, especially in low-resource settings. It's not a paradigm shift, but it's a usable heuristic that ships code. I would send it to peer review with a request to add a domain-adaptation control and at least a few runs with standard errors. If the control shows the gain is just in-domain fine-tuning, the paper's framing needs to change; if the gain survives, it's a solid workshop-level contribution.","headline":"A simple, plausible trick for supervised embedding fine-tuning that mostly works, but never isolates whether the class label is what helps.","tokens_in":9922,"tokens_out":2459,"would_cite":false,"duration_ms":27812,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Class labels as context sharpen word embeddings for classification","keywords":["word embeddings","fine-tuning","Doc2Vec","class2Vec","text classification","sentiment analysis","Arabic natural language processing","transfer learning"],"falsifier":"Shuffle the class labels used only during fine-tuning, keeping the downstream training labels unchanged; if accuracy still rises about as much as with true labels, the gain is not caused by the class-context mechanism but by in-domain fine-tuning itself.","tokens_in":9018,"feed_emoji":"🏷️","tokens_out":6250,"duration_ms":60270,"temperature":0.7,"pith_summary":"This paper proposes a simple way to adapt pretrained word embeddings to a text classification task: while fine-tuning with Doc2Vec, it replaces each document's identifier with the document's class label, so the class itself acts as an extra context vector in the prediction of each word. The intended effect is to push words that are distinctive of a class closer together in vector space and to separate words like 'good' and 'bad' that otherwise share contexts. On three Arabic datasets and two English sentiment-treebank datasets, the fine-tuned embeddings improved the accuracy of a simple recurrent classifier in most single-label settings, with the largest gain on Arabic sentiment (F1 from 0.535 to 0.621). The one tested multilabel task did not improve, which the authors take as a sign that the method suits exclusive-class tasks rather than multilabel ones. The appeal is that the method requires only the labels already present in the training data, no extra annotation or model changes.","feed_headline":"Class labels as context sharpen word embeddings","feed_subtitle":"Class-aware fine-tuning lifts Arabic and English sentiment and emotion scores by up to several points.","key_machinery":"The machinery is Doc2Vec, an extension of Word2Vec that learns a vector for each document alongside word vectors by using the document vector as an additional context in the word-prediction task. The paper repurposes that document vector as a class vector: all training instances with the same label are treated as one pseudo-document, and the class identifier, not a per-instance identifier, is what gets averaged with the context words. Training starts from pretrained word vectors, randomly initializes vectors for words missing from the pretrained vocabulary, and runs for a small number of epochs (set to 10 here) so the pretrained geometry is adjusted rather than overwritten. The class vector is the mechanism that carries the argument, because it is the only added information beyond the ordinary local word context.","core_discovery":"The central claim is that the class label, when fed into a Doc2Vec training loop as though it were a document identifier, is a useful extra context signal for fine-tuning word embeddings. During fine-tuning, each training text is paired with its class vector, and the model learns to predict words using both the surrounding words and the class vector; after training, only the word vectors are kept. Words appearing exclusively in one class are strongly reshaped by that class, words spread across classes are only weakly affected, and words appearing in several classes are pulled toward the class in which they occur most often. Empirically, this 'class2Vec' fine-tuning raised accuracy on the Arabic emotion dataset from 0.717 with unmodified pretrained embeddings to 0.741, on SST-1 from 0.466 to 0.493, on SST-2 from 0.863 to 0.877, and on the SemEval 2017 Arabic sentiment test from an F1 of 0.535 to 0.621. On the SemEval 2018 multilabel Arabic emotion task, the fine-tuned embeddings scored 0.498 Jaccard versus 0.501 without fine-tuning, the one observed decrease.","pith_inferences":["The method could be tested as a cheap alternative to full fine-tuning of large contextual models in low-resource settings, since it only needs the training labels and a few epochs.","A controlled comparison against fine-tuning with per-document Doc2Vec vectors would separate the contribution of the class signal from the contribution of in-domain fine-tuning itself.","The class-vector idea could be extended to hierarchical labels, multi-task labels, or metadata such as author or time, turning any categorical variable into a context vector.","The small-epoch setting suggests a regularizing interpretation: brief fine-tuning on in-domain data may denoise the pretrained vectors, which is an alternative explanation for part of the gain."],"forward_implications":["Single-label text classifiers can inherit in-domain discrimination from off-the-shelf embeddings with no change to the downstream architecture.","The method is language-neutral: the same recipe improved Arabic and English datasets in the reported experiments.","Words unseen in the training data still keep their pretrained vectors, so the fine-tuned model does not lose coverage on small training sets.","The observed failure on one multilabel task implies that the approach should be expected to help mainly when classes are mutually exclusive.","Because the fine-tuned vectors are conditioned on a specific label set, they become domain- and task-specific rather than general-purpose."],"supporting_citations":[{"why":"Supplies the Doc2Vec document-vector training procedure that the method adapts by substituting class identifiers for document identifiers.","marker":"[5]"},{"why":"Provides the Word2Vec model that Doc2Vec extends and supplies the pretrained English word vectors used as a starting point.","marker":"[3]"},{"why":"Provides the pretrained Arabic embeddings used as the starting point for the Arabic experiments.","marker":"[6]"},{"why":"Provides the Arabic emotion dataset and the complement naive Bayes baseline score that the proposed method must beat.","marker":"[16]"},{"why":"Provides the SST-1 and SST-2 sentiment treebank datasets with their train, dev, test splits and phrase-level training material.","marker":"[18]"},{"why":"Supplies the convolutional neural network classifiers used as published comparison points on SST-1 and SST-2.","marker":"[19]"},{"why":"Provides the SemEval 2017 Arabic sentiment task data and the top-participant scores used for comparison.","marker":"[22]"},{"why":"Provides the SemEval 2018 multilabel emotion task data and top-participant scores used for comparison.","marker":"[23]"}],"fun_headline_variants":["Class labels as extra context boost word embeddings","Class-aware fine-tuning lifts text classification","Add class context to word vectors for accuracy gains","Fine-tune embeddings with class info for better scores","Class-driven embedding fine-tuning improves classification"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that a single shared class vector can represent the context of every training text in that class, so Doc2Vec learns a coherent class-specific signal rather than fitting noise from individual examples.","fun_headline_variants_meta":{"raw":{"variants":["Class labels as extra context boost word embeddings","Class-aware fine-tuning lifts text classification","Add class context to word vectors for accuracy gains","Fine-tune embeddings with class info for better scores","Class-driven embedding fine-tuning improves classification"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000185,"raw_usage":{"total_tokens":1303,"prompt_tokens":905,"completion_tokens":398,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":331}},"tokens_in":521,"tokens_out":398,"duration_ms":5093,"temperature":1.0,"reasoning_tokens":331,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:38:57.088732+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Shuffle the class labels used only during fine-tuning, keeping the downstream training labels unchanged; if accuracy still rises about as much as with true labels, the gain is not caused by the class-context mechanism but by in-domain fine-tuning itself.","supporting_citations":[{"cited_title":"Distributed Representations of Sentences and Documents","cited_arxiv_id":null,"evidence_quote":"Supplies the Doc2Vec document-vector training procedure that the method adapts by substituting class identifiers for document identifiers."},{"cited_title":"AraVec: A set of Arabic Word Embedding Models for use in Arabic NLP","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained Arabic embeddings used as the starting point for the Arabic experiments."},{"cited_title":"Emotional Tone Detection in Arabic Tweets","cited_arxiv_id":null,"evidence_quote":"Provides the Arabic emotion dataset and the complement naive Bayes baseline score that the proposed method must beat."},{"cited_title":"Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank","cited_arxiv_id":null,"evidence_quote":"Provides the SST-1 and SST-2 sentiment treebank datasets with their train, dev, test splits and phrase-level training material."},{"cited_title":"Convolutional Neural Networks for Sentence Classification","cited_arxiv_id":null,"evidence_quote":"Supplies the convolutional neural network classifiers used as published comparison points on SST-1 and SST-2."},{"cited_title":"NileTMRG at SemEval-2017 Task 4: Arabic Sentiment Analysis","cited_arxiv_id":null,"evidence_quote":"Provides the SemEval 2017 Arabic sentiment task data and the top-participant scores used for comparison."},{"cited_title":"SemEval-2018 Task 1: Affect in Tweets","cited_arxiv_id":null,"evidence_quote":"Provides the SemEval 2018 multilabel emotion task data and top-participant scores used for comparison."}],"review_version":1}