{"id":"217ce69e-0987-4f51-977e-63833cf9b6f1","arxiv_id":"2510.05864","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"LLMs detect harmful sentences in long inputs best at around 25% prevalence and when placed early, and worse when sparse, late, or implicit.","lead":"This paper tests how well three large language models can pick out harmful sentences buried inside long, mixed prompts, varying prompt length, density, position, and how explicit the harm is. It finds that models detect harm best at moderate density and earlier positions, but struggle to pinpoint sparse, late, or implicit harmful sentences.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline patterns rest on incoherent synthetic prompts; the paper's own Limitation concedes this may make the task easier, so transfer to coherent long inputs is unverified.","rationale":"The reader's weakest_assumption correctly identifies the synthetic no-coherence prompt as the load-bearing premise. My reading agrees: all four headline findings share this premise, and the paper's own Limitations section explicitly flags the coherence gap without providing evidence on its direction or magnitude. The paper's internal controls are otherwise strong — three datasets, three models, 128 seeds, and a clear extraction task — which is why the concern is about external validity, not internal consistency. A targeted coherent-document experiment would settle the question. A controlled stress test can still be valuable, but the Discussion's 'actionable insights' for real systems require that the patterns persist when harmful sentences appear inside coherent discourse. Since the reader already returned CONDITIONAL, my critique does not move the verdict; it reinforces the condition.","tokens_in":22903,"tokens_out":6720,"duration_ms":47985,"concrete_test":"Construct a coherent-document variant for at least one dataset (e.g., JigsawToxic original Wikipedia talk-page comments, with surrounding thread context preserved). Insert the same controlled harmful sentences at the same ratios (0.05, 0.10, 0.25, 0.50), lengths (1500, 6000 tokens), and regions (beginning, middle, end, all), while keeping natural discourse structure. Run the identical index-extraction prompt on LLaMA-3-8B-Instruct. Compare macro-F1 and harmful recall to Figures 1–3: if the r=0.25 peak and the position/dilution orderings do not replicate within bootstrap 95% CIs, the central claims are specific to incoherent prompts rather than general properties of long-context sensitivity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claims — non-monotonic prevalence peaking near r=0.25, dilution, beginning advantage, and explicit-over-implicit advantage — are all measured on prompts built by randomly concatenating independent sentences (§3). The load-bearing premise is that this controlled stress test captures how LLMs prioritize harmful content in realistic long inputs such as threads, documents, or RAG contexts. The paper's own Limitation states: 'this produces prompts that lack natural coherence... it may make the task easier for models, since they do not need to track harmful content in a flowing narrative.' That admission directly targets external validity: if discourse structure, topic coherence, or narrative context changes how models allocate attention or use semantic priors, the measured curves may not transfer. The Discussion nonetheless draws 'direct implications' for moderation, RAG, and multi-turn safety (§6). Without a coherent-document check, the headline findings are claims about a synthetic stress test, not about real long inputs.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a controlled stress-test evaluation of how instruction-tuned LLMs handle harmful sentences embedded in long inputs. The authors construct synthetic prompts by concatenating independent harmful and neutral sentences, varying four factors: prompt length (600–6,000 tokens in the main text, up to 30k in the appendix), harmful-to-neutral ratio (0.05–0.5), harm type (explicit/implicit/both), and placement (beginning/middle/end/all). They evaluate LLaMA-3-8B-Instruct, Qwen2.5-7B-Instruct, and Mistral-7B-Instruct-v0.3 on three datasets (IHC, OffensEval, JigsawToxic), asking the model to output the indices of harmful sentences. The main reported findings are: (i) non-monotonic prevalence sensitivity, with macro-F1 peaking around r=0.25; (ii) a dilution effect, with recall and precision degrading as more neutral sentences are added; (iii) a position effect, with beginning-placed harmful sentences detected most reliably; and (iv) higher sensitivity to explicit than implicit harm. The paper frames this as a complementary diagnostic to coherent-document evaluation and draws implications for moderation, RAG, and multi-turn safety.","tokens_in":23127,"tokens_out":5585,"duration_ms":53099,"significance":"If the findings hold, the paper contributes a systematic, factorial evaluation of long-context safety that goes beyond binary compliance scores, and the promised public framework could be reusable by safety teams. The design has genuine strengths: three models, three datasets, independent manipulation of several factors, and k=128 repetitions per setting. The reported patterns are consistent across models and datasets, which lends some robustness. However, the current evidence is weakened by the absence of any uncertainty quantification, the possibility that the headline prevalence peak is at least partly a macro-F1 artifact under class imbalance, and the entirely synthetic input construction, which the authors themselves concede may make the task easier. The paper is a useful stress-test study, but its broader claims about real long inputs are not yet supported.","major_comments":[{"comment":"The central non-monotonic prevalence claim—macro-F1 peaking at r≈0.25—is presented without any measure of variance. Although k=128 repetitions are run, §4.5 reports only pooled micro metrics, so the reader cannot tell whether the differences between r=0.1, 0.25, and 0.5 are meaningful or within sampling noise. Some curves in Fig. 1 appear to differ by only a few F1 points. The authors should provide per-run confidence intervals (e.g., bootstrap over the 128 runs), or a significance test, before asserting that the pattern is systematic.","section":"§4.3, §4.5, Fig. 1"},{"comment":"The prevalence peak may be a metric artifact. The authors themselves note that harmful precision and recall continue to improve as r increases, and the macro-F1 decline at high r comes from the non-harmful class. Under macro-F1, any classifier whose predicted prevalence roughly tracks the true ratio (as the PPV plots show) will tend to have lower macro-F1 at extreme r simply because one class dominates. To support the substantive claim of a 'moderate prevalence sweet spot' in §6, the paper should show that the non-monotonicity is not trivially produced by a prevalence-matching baseline, and should report a class-balance-robust metric such as Matthews correlation coefficient or balanced accuracy.","section":"§5.2, Fig. 1, §6"},{"comment":"The prompts are random concatenations of independent sentences, and the Limitations section concedes that 'it may make the task easier for models, since they do not need to track harmful content in a flowing narrative.' The Discussion (§6) nonetheless draws 'direct implications' for RAG pipelines, content moderation, and multi-turn conversations—settings that involve coherent or discourse-structured inputs. Without at least a small coherent-document validation, the external validity of the central findings for these real-world applications remains unverified. The authors should either add such a condition or explicitly restrict the paper's claims to the synthetic stress-test regime.","section":"§3, Limitations"},{"comment":"The sentence-level baseline is not a controlled comparison with the long-context setting. The sentence-level prompt uses a binary yes/no format with a 4-shot prompt, while the long-context prompt asks for index extraction with a 1-shot prompt. Thus, claims such as 'multi-sentence prompting leads the model to approximate the underlying harm ratio more closely' (comparing PPV curves) confound context length with task format and instruction differences. This limits the interpretation of sentence-level versus long-context comparisons throughout §5.2.","section":"§5.1, §5.2, Tables 1/3/4"}],"minor_comments":[{"comment":"The abstract states input lengths of 600–30,000 tokens, while the main experiments use 600–6,000 tokens (with 15k and 30k only in the appendix). Clarify in the abstract that the 30k results are appendix-only.","section":"Abstract and §4.3"},{"comment":"The x-axis label 'JigsawT oxic' contains a typo, and the dataset order in Fig. 3 differs from other figures; harmonize for readability.","section":"Fig. 3"},{"comment":"The text uses informal phrasing such as 'You can find their information on dilution analysis in Appendix E' and 'the same patterns are occurring'. Please revise to formal scientific style.","section":"Appendix D–G"},{"comment":"The acronym PPV is defined as 'Predicted Prevalence Value', which may be confused with positive predictive value. Consider renaming to 'Predicted Prevalence' or explicitly stating the intended meaning to avoid ambiguity.","section":"§4.5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical stress-test study, but the gap between the synthetic-prompt design and the broad claims in the abstract/Discussion is a concern. The non-monotonic prevalence claim in particular would benefit from a robustness check against a trivial baseline; if it turns out to be a macro-F1 artifact, the paper's main novelty would be substantially reduced. I would encourage the editor to ask for the statistical clarifications and a reframing or a coherent-text sanity check before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a genuinely useful empirical paper, and the main findings are probably right. The new thing is that it goes beyond compliance rates: models are asked to return the indices of harmful sentences in long multi-sentence prompts, and the paper systematically varies prevalence, dilution, position, and explicitness. Prior long-context safety work mostly reports binary compliance, so this is a real step forward. The dilution experiment is especially well designed — fixing the number of harmful sentences while growing the total isolates dilution cleanly. The PPV result, that models can roughly estimate how much harmful content is present but fail to say which sentences are harmful, is interesting and well supported.\n\nThe design is careful: three datasets, three models, 128 repeats per setting, fixed seeds, and a sentence-level baseline that nicely exposes the over-prediction bias of isolated classification. I believe the core patterns — moderate prevalence peak, degradation with length, beginning advantage, explicit over implicit — would survive additional scrutiny.\n\nNow the soft spots, in proportion. First, the paper reports no variance, confidence intervals, or significance tests despite having 128 repeats per cell. Some of the differences being interpreted, like the macro-F1 peak at 0.25 versus 0.5, may not be large, and without intervals you cannot tell whether they are stable effect sizes or noise. This is a moderate issue, not fatal, but easy to fix. Second, the prompts are random concatenations of independent sentences, so they lack discourse structure. The authors acknowledge this in the Limitations and even concede that it may make the task easier. That is the right honest move, but the Discussion still claims \"direct implications\" for RAG, moderation, and multi-turn safety. That overreaches. The results describe a controlled stress test; transfer to coherent documents is untested. A single sanity check on coherent long documents would have largely addressed this. Third, the abstract says context length 600–30,000 tokens, but the main experiments are 600–6,000, with 15k/30k only in the appendix for two models. Minor inconsistency. Fourth, the evaluation code is promised but not yet available, which matters for a benchmark-style contribution.\n\nThe citation pattern is fine; self-citations are prior detection work, not inputs to the results. The paper is coherent on its own terms and the authors are honest about the main limitations.\n\nWho should read it: anyone working on long-context safety, content moderation pipelines, or RAG safety evaluation. It deserves a serious referee. I would send it out and ask for variance reporting, a toned-down Discussion, and ideally one coherent-document experiment. The core work is solid enough to build on.","headline":"A careful controlled stress-test mapping how LLMs extract harmful sentences from long synthetic inputs; the patterns are plausible but need error bars and a coherent-document check before they generalize.","tokens_in":23549,"tokens_out":2217,"would_cite":true,"duration_ms":22999,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Instruction-tuned LLMs detect harmful sentences in long inputs best when about a quarter of the prompt is harmful; detection degrades as context lengthens, harm moves later in the input, or harm becomes implicit.","keywords":["harmful content detection","long-context LLMs","sensitivity analysis","prevalence effect","dilution effect","position bias","implicit vs explicit harm","instruction-tuned models"],"falsifier":"Run the same localization protocol on coherent long documents — forum threads, reports, or multi-turn conversations — where harmful sentences appear naturally; if recall no longer drops with length, or if the performance peak moves away from roughly 25 percent harmful prevalence, the controlled stress-test pattern does not describe real inputs. A simpler check is to compare random-shuffle prompts to the same sentences reordered into a narrative with topic flow and anaphora.","tokens_in":22827,"feed_emoji":"🛡️","tokens_out":3084,"duration_ms":28948,"temperature":0.7,"pith_summary":"The paper tries to establish that instruction-tuned LLMs, when asked to list harmful sentences embedded in long prompts, follow consistent, predictable patterns: their best precision-recall balance occurs when harmful content makes up about a quarter of the input; recall and precision drop as the prompt grows and harmful sentences are diluted; harmful sentences near the beginning are detected more reliably than those in the middle or end; and explicit harm is identified more reliably than implicit harm. These patterns hold across toxic, offensive, and hate content and across three open-weights instruction-tuned models. A sympathetic reader would care because content moderation, retrieval-augmented generation, and long-document filtering increasingly rely on LLMs to spot harmful spans inside extended mixed content, where sentence-level evaluations do not transfer directly.","feed_headline":"LLM harm detection peaks at 25 percent prevalence","feed_subtitle":"Longer inputs, later positions, and implicit wording all weaken models' ability to pinpoint harmful sentences.","key_machinery":"The central instrument is a controlled stress-test prompt: sentences are sampled from existing datasets, numbered, randomly concatenated, and appended to an instruction asking the model to return the indices of harmful sentences. By varying four factors independently — total token length, harmful sentence ratio, harm type (explicit/implicit), and position region — the design isolates each factor's effect while holding others constant. The core metrics are macro-F1, predicted prevalence value (the fraction of sentences the model labels harmful), and harmful-class precision and recall; the contrast between predicted prevalence and F1 is what exposes the calibration-versus-localization gap.","core_discovery":"Using prompts built by randomly concatenating harmful and neutral sentences into numbered lists, the authors show that LLM sensitivity to harmful content is non-monotonic in prevalence: performance rises as the harmful fraction increases toward roughly 0.25 and then stagnates or declines as the fraction reaches 0.5, because the model over-predicts harm and loses accuracy on benign sentences. With the number of harmful sentences fixed, adding more neutral sentences lowers both recall and precision. Harmful sentences placed in the first third of the prompt are detected better than those in later regions, and explicit harm outperforms implicit harm on every dataset. The authors interpret this a","pith_inferences":["If the pattern transfers to coherent documents, safety teams could add a prevalence-calibration probe: ask the model how many harmful sentences exist before asking which ones, and use the discrepancy to flag low-confidence localization.","The position bias suggests a primacy effect for safety judgments; a testable extension is whether an explicit 'scan the whole input' instruction or attention-promoting formatting shifts detection toward later positions.","The random-concatenation setup removes discourse cues, so a natural next experiment is to graft harmful sentences into coherent narratives and check whether the quarter-prevalence optimum and the position bias survive.","Because predicted prevalence tracks true ratio while recall lags, a cheap two-pass strategy could be built: use the prevalence estimate to decide when to run a more thorough second pass over the input."],"forward_implications":["Content moderation systems that rely on long-context LLMs will miss harmful sentences more often when those sentences are rare or buried among many benign ones.","In retrieval-augmented generation, harmful passages placed late in the retrieved context are more likely to slip through, so ranking or reordering that surfaces early or central content may reduce positional bias.","Models can often estimate how much harmful content exists even when they cannot name which sentences are harmful, a signal that could be used for triage before deeper inspection.","Explicit harm can be caught with reasonable reliability, but implicit hate and offense require either longer attention windows or complementary detectors.","The reported patterns are consistent across three model families and three harm categories, suggesting a common instruction-following behavior rather than a dataset-specific artifact."],"fun_headline_variants":["LLM harm detection peaks at 25% prevalence, not 50%","Long inputs reduce LLM harm detection accuracy","Explicit harm outshines implicit for LLM detection","Earlier harmful sentences get more LLM attention","LLM harm sensitivity non-monotonic with prevalence"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The results rest on prompts made by randomly shuffling independent sentences; if coherent, topically connected real-world inputs change how models weight harmful content, these prevalence, dilution, and position effects may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["LLM harm detection peaks at 25% prevalence, not 50%","Long inputs reduce LLM harm detection accuracy","Explicit harm outshines implicit for LLM detection","Earlier harmful sentences get more LLM attention","LLM harm sensitivity non-monotonic with prevalence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001144,"raw_usage":{"total_tokens":4576,"prompt_tokens":727,"completion_tokens":3849,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":471,"completion_tokens_details":{"reasoning_tokens":3771}},"tokens_in":471,"tokens_out":3849,"duration_ms":22901,"temperature":1.0,"reasoning_tokens":3771,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T11:13:16.875736+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same localization protocol on coherent long documents — forum threads, reports, or multi-turn conversations — where harmful sentences appear naturally; if recall no longer drops with length, or if the performance peak moves away from roughly 25 percent harmful prevalence, the controlled stress-test pattern does not describe real inputs. A simpler check is to compare random-shuffle prompts to the same sentences reordered into a narrative with topic flow and anaphora.","supporting_citations":[],"review_version":1}