{"id":"b147af58-1d04-4083-a0fd-946e7fa21685","arxiv_id":"2508.06360","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Adding predicted aggression labels to prompts improves LLM cyberbullying detection F1 on a single dataset, but the claim is confounded by prompt-format changes and missing statistics.","lead":"This paper tests whether adding a predicted aggression label to the input prompt helps a language model detect cyberbullying in social media posts. Across three Gemma models and one test set, the enriched prompt boosts macro-F1 over standard fine-tuning, but the result lacks error bars and proper controls.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EPP vs. LoRA comparison confounds aggression-label content with prompt-format change; no neutral-prefix control, so causal attribution is unsupported.","rationale":"Reader's weakest_assumption aligns with this concern: the causal interpretation requires that the aggression label content, not the prompt template, drives the gain. The paper's Section 4.4 explicitly states the aggression results are identical for LoRA and EPP because no retraining was performed, confirming that the only intervention is the prompt prefix. Without a neutral-prefix ablation, the 'aggression-informed context' claim is not identified. This is the single most load-bearing issue because even if the aggression labels are perfect, the experiment would not rule out a prompt-format effect; and if the labels are noisy, the claim fails on both grounds. The proposed control directly tests the mechanism. No other concern (e.g., absence of error bars) is as central to the causal claim.","tokens_in":9194,"tokens_out":4363,"duration_ms":48595,"concrete_test":"On the D6 test set, run a neutral-prefix control identical to EPP except that the aggression token is replaced by a fixed placeholder (e.g., 'This post was predicted as [unavailable]. Based on this, classify the following content for cyberbullying.'), using the same LoRA cyberbullying adapter and inference settings as in §3.3/§4.4. If the macro-F1 of this control equals the reported EPP values (0.99, 0.99, 0.86) within 0.01, the improvement over standard LoRA is due to prompt format, not aggression content. As a secondary check, compute the aggression adapter's macro-F1 on D6 test posts to assess whether the labels carry signal.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract; §4.4) is that the Enriched Prompt Pipeline's gain over standard LoRA shows 'aggression-informed context significantly boosts cyberbullying detection.' The comparison in Table 2 is between a LoRA adapter queried with a plain prompt and the same type of adapter queried with a prompt that prepends a sentence containing a predicted aggression label (§3.3, Steps 1–3). These conditions differ in two ways: the added sentence and the label value. There is no control condition in which the added sentence contains a neutral, non-aggression token (or a shuffled label) of similar length. Therefore, the observed F1 increase (0.84→0.99, 0.93→0.99, 0.84→0.86) could be due entirely to the prompt-format change rather than to the semantic content of the aggression label. This is not a remote possibility: the cyberbullying LoRA adapter was never trained with the enriched prompt template, so the test-time input distribution is shifted. Furthermore, the paper never reports accuracy of the aggression adapter on D6 test posts; if the predicted labels are near-random on this out-of-domain data, the 'enrichment' is effectively a constant/uninformative prefix. Both omissions undermine the specific causal claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates whether aggression detection can serve as an auxiliary signal for cyberbullying detection in instruction-tuned LLMs. It evaluates zero-shot, few-shot, LoRA-based supervised fine-tuning, joint multi-task learning, and a proposed Enriched Prompt Pipeline (EPP) in which predicted aggression labels are embedded into cyberbullying-classification prompts. Experiments use five aggression datasets (D1–D5) and one cyberbullying dataset (D6) with Gemma-2-2B, Gemma-2-9B, and Gemma-3-4B. Table 2 reports EPP macro-F1 scores of 0.99, 0.99, and 0.86 versus 0.84, 0.93, and 0.84 for standard LoRA on cyberbullying detection. The authors conclude that aggression-informed context significantly boosts detection.","tokens_in":9525,"tokens_out":5058,"duration_ms":59479,"significance":"If validated, the proposed EPP is simple, parameter-efficient, and practically appealing for low-resource moderation settings. The paper uses publicly available benchmarks and covers three model scales, which are strengths. However, the central causal claim is not currently established: the EPP comparison confounds prompt-format change with the semantic content of the predicted aggression label, the quality of the predicted labels on the target set is never reported, and the results lack statistical grounding. The contribution is therefore promising but preliminary.","major_comments":[{"comment":"The central claim that 'aggression-informed context significantly boosts cyberbullying detection' is confounded. EPP differs from LoRA in two ways: it prepends the sentence 'This post was predicted as [...]' and it includes a predicted label value. There is no control condition with a neutral or fixed prefix (e.g., 'This post was predicted as [not-applicable].') or with a shuffled/random label. Since the cyberbullying LoRA adapter was not trained on enriched prompts, the test-time input distribution is shifted. The reported F1 gains (0.84→0.99, 0.93→0.99, 0.84→0.86) could therefore be due to the prompt-format change alone. The paper must include a neutral-prefix ablation to support its causal interpretation.","section":"§3.3, §4.4, Table 2"},{"comment":"Step 1 passes D6 test posts through a LoRA aggression model, but no accuracy, F1, or predicted-label distribution is reported for that model on D6. If the predicted aggression labels are near-random on this out-of-domain data, the 'enrichment' is effectively an uninformative or constant prefix. The authors should report the aggression model's performance on D6 and ideally compare EPP with oracle/ground-truth aggression labels to verify that the semantic signal, not the template, drives the improvement.","section":"§3.3, Step 1; §4.4"},{"comment":"All results are single-point estimates without standard deviations, confidence intervals, or significance tests. In particular, the Gemma-3-4B improvement (0.84 vs. 0.86) is only two macro-F1 points and may be within run-to-run noise given the fixed hyperparameters (e.g., lr=1e-4, r=8, 1 epoch). The 'consistently outperforms' claim requires repeated runs or at least a randomization test.","section":"§3.4, Table 2"},{"comment":"The five aggression datasets D1–D5 are collapsed into a single 'Aggression Detection' column in Table 2, but the methodology never specifies whether the aggression LoRA was trained on pooled D1–D5, on a selected subset, or separately per dataset with averaged results. This matters because the datasets differ in language (English, Hindi-English, Bengali), source, and annotation conventions; an unexplained aggregation could bias the predicted labels used in EPP and make reproduction impossible. The exact training protocol must be described.","section":"§3.1, Table 1, Table 2"},{"comment":"The generalization claim rests on a single cyberbullying dataset (D6). No second cyberbullying corpus, cross-platform evaluation, or domain-shift experiment is provided. EPP's advantage may be dataset-specific, so the paper should either temper the generalization language or add an external validation set.","section":"§4.4, Table 2"}],"minor_comments":[{"comment":"The prompt template is described only schematically ('This post was predicted as [...]'). Please provide the exact full prompt, including the original post and any instruction formatting, to enable reproducibility.","section":"§3.3, Step 2"},{"comment":"Footnote 1 ('Transformer Heads Library') lacks a URL or citation. Also, the model repository for Gemma-3-4B is named as 'unsloth/gemma-3-4b-it-unsloth-bnb-4bit'; please specify the quantization procedure and whether it was applied to all models.","section":"§3.4"},{"comment":"The MTL description is vague: 'joint loss (sum) from different heads to both LoRA adapters' and training epochs 'between 3 and 6' are underspecified. Exact loss weighting, epoch counts, and head architecture should be stated.","section":"§3.2"},{"comment":"There is no code or data availability statement. Even a pointer to a repository and the exact Kaggle version/DOI for D6 would improve reproducibility. Also, the paper has minor typographical and formatting inconsistencies (e.g., 'gemma' capitalization, no in-text reference to Figure 1).","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is salvageable if the authors add a neutral-prefix control, report aggression-label quality on D6, provide repeated-run statistics, and clarify how D1–D5 were aggregated. If those experiments cannot be added, the central causal claim should be removed and the paper reframed as a descriptive study. The current evidence is insufficient for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper is a decent empirical exploration of a reasonable idea, but the headline claim is not supported by the evidence as presented. The confound the stress-test flags is real and load-bearing.\n\nWhat's new: the specific pipeline—predict an aggression label with one LoRA adapter, inject it into a prompt, then classify cyberbullying with a second LoRA adapter—is not in the cited literature. The paper also does something I appreciate: it lays out zero-shot, few-shot, LoRA SFT, and MTL baselines side by side for three Gemma models, and it is honest that MTL is inconsistent. The datasets are public, the setup is simple, and the pipeline is reproducible.\n\nThe soft spot is the comparison in Table 2. EPP differs from standard LoRA in two ways at once: the prompt gets an extra sentence, and that sentence contains a predicted aggression label. There is no control condition where the extra sentence is neutral or the label is shuffled. So the macro-F1 gains (0.84→0.99 for Gemma-2-2B, for example) could come entirely from the format change. The paper never reports the aggression adapter's accuracy on the cyberbullying test posts, so we don't know whether the labels are even informative in this domain. No error bars, no significance tests, and only one cyberbullying dataset. That makes the abstract's 'significantly boosts' too strong.\n\nI don't think this is a fatal flaw in the idea—the approach is plausible and maybe works—but the causal claim is not established. A reviewer should ask for a neutral-prefix ablation and aggression-label quality on the target test set before this can be accepted. Given that the paper is transparent and the problem is practical, I'd send it to review rather than desk-reject it. I just wouldn't accept it as is.","headline":"A plausible idea with a confounded comparison: the EPP gain over LoRA is not attributable to the aggression label until a neutral-prefix control and label-quality check are run.","tokens_in":9949,"tokens_out":2707,"would_cite":false,"duration_ms":31557,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Enriching cyberbullying prompts with predicted aggression labels consistently beats standard LoRA fine-tuning, raising macro-F1 to 0.99 on two of three language models.","keywords":["cyberbullying detection","aggression detection","prompt enrichment","LoRA fine-tuning","multi-task learning","large language models","contextual augmentation","social media safety"],"falsifier":"Run the same EPP pipeline with the aggression label replaced by a neutral or randomly assigned label (for example, always 'Not-Aggressive' or a permuted label) and compare macro-F1 on the cyberbullying test set. If performance stays near 0.99, the gain is not from the aggression signal. As a second check, measure the aggression model's accuracy on the cyberbullying test posts; near-chance accuracy would show the labels cannot be carrying the semantic information the paper attributes to them.","tokens_in":9143,"feed_emoji":"🛡️","tokens_out":6901,"duration_ms":64160,"temperature":0.7,"pith_summary":"This paper tries to establish that adding a predicted aggression label to the prompt improves cyberbullying detection across three instruction-tuned language models, compared with fine-tuning the same models on the cyberbullying data alone. It tests zero-shot, few-shot, LoRA fine-tuning, joint multi-task learning, and an enriched prompt pipeline (EPP) that prepends the predicted aggression category—overtly aggressive, covertly aggressive, or not aggressive—to each post before classification. On macro-F1 for cyberbullying, EPP reaches 0.99 for two models and 0.86 for a third, beating the corresponding LoRA baselines of 0.84, 0.93, and 0.84. The paper's point is that cheap prompt-level context can substitute for extra training signal in a low-resource safety task, though the results are presented as preliminary.","feed_headline":"Aggression labels in prompts lift cyberbullying F1 to 0.99","feed_subtitle":"Enriched prompting beats plain LoRA on three language models, with the biggest gain on the smallest model.","key_machinery":"The central mechanism is the Enriched Prompt Pipeline (EPP), a two-stage inference procedure: a LoRA-tuned aggression model first labels each post as Overtly Aggressive, Covertly Aggressive, or Not-Aggressive; that label is inserted into a fixed prompt template that states the predicted aggression and asks for cyberbullying classification; then a second LoRA-tuned cyberbullying classifier reads the enriched prompt. This converts single-task text classification into a contextual reasoning problem, with aggression serving as a guiding signal, and it does so without changing model weights or adding training time beyond the original LoRA adapters.","core_discovery":"The paper's central discovery is that aggression-informed contextual prompting—embedding a predicted overt/covert/not-aggressive label into the cyberbullying classification prompt—consistently improves macro-F1 over standard LoRA fine-tuning on the same models. In the reported numbers, the enriched prompt pipeline raises Gemma-2-2B from 0.84 to 0.99, Gemma-2-9B from 0.93 to 0.99, and Gemma-3-4B from 0.84 to 0.86. The authors interpret this as evidence that auxiliary aggression signals, injected at inference time, help models recognize subtle and indirect bullying that they miss when classifying raw text alone. They also report that multi-task learning gives inconsistent results across models","pith_inferences":["A neutral-prefix control—replacing the aggression label with a fixed or random token—would test whether the F1 gain comes from the label's semantics or simply from a longer prompt template.","The aggression labeler is trained on five datasets spanning English, code-mixed Hindi-English, and Bengali transliteration, but the paper does not measure cross-domain transfer, so whether EPP generalizes beyond the Kaggle test set remains open.","Errors in the aggression predictor propagate into the prompt: a covertly aggressive post mislabeled 'Not-Aggressive' may suppress the cyberbullying classifier, so the pipeline is only as good as the auxiliary model on the target domain.","With F1 at 0.99, the Kaggle benchmark is near saturation; testing on a more imbalanced or out-of-domain set would reveal whether the enrichment effect persists at lower label quality."],"forward_implications":["EPP raises cyberbullying macro-F1 to 0.99 on two of three models and 0.86 on the third, all above their LoRA baselines, without retraining the cyberbullying classifier at inference time.","The largest gain appears on the smallest model, suggesting prompt enrichment may be most useful when model capacity or annotated data is limited.","Because the aggression scores are identical between LoRA and EPP, the improvement is attributed entirely to the prompt context rather than to any change in the model weights.","Joint multi-task learning produced inconsistent results across models, so the paper argues that sequential prompt enrichment is a more reliable way to transfer aggression knowledge than joint training.","The same two-stage enrichment idea could be applied to other safety-critical text classification tasks where a coarse auxiliary label is cheap to predict."],"supporting_citations":[{"why":"Supplies the TRAC-1 aggression identification benchmark used to train the aggression labeler.","marker":"Kumar et al., 2018"},{"why":"Supplies the TRAC-2 aggression evaluation data, another training source for the aggression labeler.","marker":"Kumar et al., 2020"},{"why":"Provides the aggression and misogyny detection multi-task data used as one of the five aggression training sets.","marker":"Samghabadi et al., 2020"},{"why":"Provides code-mixed aggression and offensive language data that broadens the aggression labeler's coverage.","marker":"Nafis et al., 2023"},{"why":"Provides the political aggression dataset (D1) used to train the aggression predictor.","marker":"Rawat et al., 2023"},{"why":"Supplies the multi-class cyberbullying dataset (D6) on which EPP versus LoRA macro-F1 is measured.","marker":"Fati et al., 2025"},{"why":"Supports the premise that aggression-labeled context helps disambiguate harmful from non-harmful content.","marker":"Qian et al., 2019"},{"why":"Establishes the aggression spectrum from overt to subtle that motivates using aggression categories as prompts.","marker":"Salminen et al., 2020a"}],"fun_headline_variants":["Aggression labels in prompts beat LoRA for cyberbullying","Prompting with aggression hits 0.99 F1 on cyberbullying","Aggression-informed prompts boost LLM cyberbullying detection","Auxiliary aggression signals improve cyberbullying detection in LLMs"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The paper assumes that the predicted aggression labels placed into the prompts are accurate enough to be informative on the cyberbullying test posts, and that the F1 gains come from the label content rather than from the prompt template alone; it reports no test-set accuracy for the aggression predictor and no control prompt with a neutral or random label.","fun_headline_variants_meta":{"raw":{"variants":["Aggression labels in prompts beat LoRA for cyberbullying","Prompting with aggression hits 0.99 F1 on cyberbullying","Aggression-informed prompts boost LLM cyberbullying detection","Auxiliary aggression signals improve cyberbullying detection in LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000235,"raw_usage":{"total_tokens":1321,"prompt_tokens":712,"completion_tokens":609,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":456,"completion_tokens_details":{"reasoning_tokens":535}},"tokens_in":456,"tokens_out":609,"duration_ms":7328,"temperature":1.0,"reasoning_tokens":535,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T22:45:02.321364+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same EPP pipeline with the aggression label replaced by a neutral or randomly assigned label (for example, always 'Not-Aggressive' or a permuted label) and compare macro-F1 on the cyberbullying test set. If performance stays near 0.99, the gain is not from the aggression signal. As a second check, measure the aggression model's accuracy on the cyberbullying test posts; near-chance accuracy would show the labels cannot be carrying the semantic information the paper attributes to them.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the TRAC-1 aggression identification benchmark used to train the aggression labeler."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the TRAC-2 aggression evaluation data, another training source for the aggression labeler."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the aggression and misogyny detection multi-task data used as one of the five aggression training sets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the political aggression dataset (D1) used to train the aggression predictor."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the multi-class cyberbullying dataset (D6) on which EPP versus LoRA macro-F1 is measured."}],"review_version":1}