{"id":"fe5d2706-192d-42a2-af22-41f0fc40d378","arxiv_id":"2508.17490","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Zero-shot long-document classification can keep accuracy while cutting context by half using TF-IDF sentence ranking.","lead":"This paper tests a simple trick for classifying long documents: rank the sentences by importance and feed only the top half to a language model. The authors report similar accuracy to using the full document, but with up to 35% lower inference cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TF-IDF top-50% sentence selection may not preserve label-relevant evidence outside news; need direct evidence that ranking correlates with classifier signal.","rationale":"The reader correctly identified that TF-IDF's sufficiency for label-relevant sentence selection is the load-bearing assumption. I agree with the reader's weakest_assumption. Since this is an abstract-only review, there is not enough information to reject or accept; the verdict UNVERDICTED is appropriate. The concrete test described would settle whether the central claim holds on the original dataset and whether it generalizes. If the test shows TF-IDF closely tracks the oracle, the claim survives; otherwise the claim needs qualification to specific genres or requires a more selective ranking method.","tokens_in":633,"tokens_out":2026,"duration_ms":27690,"concrete_test":"On MahaNews, compute for each document the zero-shot classifier's accuracy and confidence under three conditions: (1) full context, (2) top-50% TF-IDF sentences, (3) bottom-50% TF-IDF sentences. Additionally compute an oracle subset of the 50% of sentences that individually maximize classification confidence. If the oracle subset substantially outperforms the TF-IDF subset (e.g., >2–3 points accuracy), then TF-IDF is not capturing label-relevant ordering, and the 'comparable' result is likely a property of news redundancy. Then repeat the top-50% TF-IDF experiment on a second long-document benchmark with a different evidence distribution (e.g., legal contracts or scientific papers); if accuracy drops significantly, the claim does not generalize.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that retaining the top 50% of TF-IDF-ranked sentences preserves full-document classification accuracy. This assumes that within-document term frequency ranks sentences by their contribution to the label-relevant information that the zero-shot classifier needs. The abstract provides no evidence for this assumption: no analysis of where the discriminating evidence actually resides, no comparison with oracle sentence selection, and no evaluation outside MahaNews. For news articles, topical information is often concentrated in the lead, making top-heavy TF-IDF selection effective; but in other genres (legal, biomedical, scientific), labels may depend on rare or scattered sentences that TF-IDF would rank low. Dropping the bottom half could then remove exactly the sentences that decide the label. The reported 35% inference speedup is only meaningful if the accuracy maintenance is robust, which is not established beyond a single dataset.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a zero-shot long-document classification approach that ranks sentences by TF-IDF within each document, retains a top fraction (50% in the headline result), and feeds only those sentences to a short-text transformer model, avoiding architectural changes. The evaluation uses the MahaNews dataset of long Marathi news articles across three context-reduction strategies. The abstract reports that retaining the top 50% of ranked sentences yields accuracy comparable to full-document inference while cutting inference time by up to 35%.","tokens_in":852,"tokens_out":1893,"duration_ms":24075,"significance":"If established, the result would be a simple and attractive efficiency result: an unsupervised, model-agnostic sentence-ranking preprocessing step that reduces context without fine-tuning or architecture modification. The claimed inference-speedup and maintained accuracy are practically relevant for deploying short-text classifiers on long documents in low-resource settings. The paper's strengths are its zero-shot framing, the absence of fitted parameters in the ranking itself, and the concrete efficiency claim. However, the evidence in the abstract is not sufficient to establish the central claim: 'comparable' is unquantified, only one dataset is used, and no analysis links TF-IDF ranks to classification-relevant information.","major_comments":[{"comment":"The central claim, 'retaining only the top 50% ranked sentences maintains performance comparable to full-document inference,' is not supported by any quantitative comparison. No accuracy numbers, error bars, significance tests, or equivalence margins are reported. A point estimate of 'comparable' is not sufficient to rule out a meaningful drop, especially for zero-shot settings where variance across classes and documents is typically high. The paper must report exact accuracies, standard deviations over repeated runs or folds, and a statistical comparison with the full-document baseline.","section":"Abstract"},{"comment":"The evaluation is limited to a single dataset, MahaNews, comprised of long Marathi news articles. News articles commonly concentrate topical information in the lead paragraphs, which likely inflates the effectiveness of any top-ranking sentence selector. The claim that this is a generally effective technique for zero-shot long-document classification requires evidence from domains where label-relevant information is more distributed, such as legal opinions, scientific papers, or biomedical records. Without such evidence, the result may be a genre-specific artifact.","section":"Abstract"},{"comment":"The abstract asserts that TF-IDF-based sentence ranking selects 'the most informative sentences,' but no evidence is provided that TF-IDF scores correlate with the information the zero-shot classifier relies on. A direct test would compare the proposed top-50% selection against an oracle that retains the sentences with the highest contribution to the classifier's decision, or an analysis showing that the bottom 50% contains negligible discriminative signal. This is load-bearing because the entire method depends on TF-IDF ranking preserving label-relevant evidence.","section":"Abstract"},{"comment":"The claimed 'inference time reduced by up to 35%' is a single point estimate without specification of the measurement protocol. It is unclear whether this includes sentence tokenization, ranking computation, or only transformer forward-pass time; whether it reflects a particular batch size, GPU, or sequence-length configuration; and what the variability is across documents. These details matter because the TF-IDF ranking itself adds overhead that could offset the forward-pass savings in many deployment scenarios.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract does not name the transformer model used (e.g., mBERT, XLM-R, or a Marathi-specific model), the input-length truncation policy, or the source of the short-text training data ('headlines'). Adding these details is necessary for reproducibility.","section":"Abstract"},{"comment":"The term 'zero-shot' is used but not defined. Clarify whether the classifier has never seen long documents at training time, and whether the evaluation set labels come from the same label space as the headline-trained model.","section":"Abstract"},{"comment":"The comparison across 'three context reduction strategies' is announced but not described. Naming the strategies and stating the baseline (full-document inference) would help the reader interpret the 50% result.","section":"Abstract"},{"comment":"Minor wording: 'comparable' is a vague term; suggest reporting a predefined equivalence bound or a confidence interval for the accuracy difference.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"This is an abstract-only review, so the assessment rests on what the abstract explicitly claims. The central idea is plausible and worth pursuing, but the current evidence is too thin to support the claimed generality. The authors should be asked to provide full experimental details, multiple datasets, statistical comparisons, and a direct check of whether TF-IDF ranks align with classifier-relevant evidence. If the full paper already contains these elements, the revision may be straightforward; if not, the claims need to be substantially softened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a simple, practical efficiency idea—use TF-IDF sentence ranking to cut the input context by half and keep zero-shot classification accuracy about the same. The new part is not the ranking itself (that's old) but the specific combination with zero-shot transformer inference and the MahaNews Marathi dataset. If the full paper actually shows the 35% speedup and 'comparable' accuracy with proper baselines and splits, that's a useful result for practitioners who need cheap long-document classification in low-resource languages.\n\nWhat the paper does well: the approach is refreshingly architecture-agnostic. It adapts short-text models to long documents without retraining, which matters for low-resource settings where you don't have labeled data or compute. The choice of TF-IDF is defensible—it's unsupervised, label-free, and cheap. The abstract is honest about the scope: it says 'comparable' and 'up to 35%,' though those terms need unpacking.\n\nThe soft spots are mostly about what the abstract doesn't say. 'Comparable' is doing a lot of work—no error bars, no statistical tests, no description of baselines or data splits. The 35% inference reduction is a point estimate on one dataset. The stress-test worry is legitimate: TF-IDF within a document ranks sentences by term salience, but that doesn't guarantee those sentences carry the label-relevant signal. For news, the lead often does, so top-50% selection works; for legal or biomedical texts, the decisive sentence might be rare and scattered. The paper needs at least one out-of-domain check or an oracle-style comparison to show the ranking correlates with what the classifier actually uses. Without that, the claim is only about MahaNews-news, not about long-document classification in general.\n\nThat said, the central claim is plausible and the method is not circular—TF-IDF is unsupervised and the only tuned parameter is the 50% cutoff, which is a simple heuristic rather than a fitted model. The absence of a full text in the review means I can't verify baselines or whether the split is clean, but those are fixable with a careful empirical section.\n\nWho is this for? People working on efficient inference, zero-shot transfer, or Marathi NLP. It's not a breakthrough, but it's a legitimate empirical contribution that could save someone from a more complicated approach. I'd send it to referees—but only with a clear request to see the full experimental details and ideally a second dataset. If the paper delivers that, it deserves publication at a workshop or applied venue.\n\nRecommendation: engage with it seriously in peer review. The idea is simple but useful, and the main risks are empirical rigor, not conceptual flaws.","headline":"A practical, modest efficiency result for zero-shot long-document classification on Marathi news; the abstract leaves key experimental details unverified, but the idea is sound enough to referee.","tokens_in":1295,"tokens_out":1072,"would_cite":false,"duration_ms":15170,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TF-IDF sentence ranking lets zero-shot models handle long documents","keywords":["zero-shot classification","long document classification","sentence ranking","TF-IDF","context reduction","Marathi news","efficient inference","transformer models"],"falsifier":"Take a text whose decisive sentence is buried among many high-TF-IDF fillers (for example, a synthetic document with one informative sentence and fifty generic but topic-word-heavy filler sentences), run the top-50% ranking, and check whether the classifier still gets the label; if it frequently drops the decisive sentence, the claim that TF-IDF ranking preserves label-relevant information is falsified outside the news distribution.","tokens_in":596,"feed_emoji":"📰","tokens_out":3403,"duration_ms":39075,"temperature":0.7,"pith_summary":"This paper tries to show that a long document can be classified by a short-text transformer without retraining or architectural changes: rank the document's sentences by TF-IDF score, keep half, and send only those to the zero-shot classifier. On long Marathi news articles, the authors report that this 50% context reduction keeps classification accuracy comparable to full-document inference while cutting inference time by up to 35%. The point of the claim is practical: if true, it turns an input-length bottleneck into a preprocessing step and lets headline-trained models work on long-form documents. The method is deliberately simple, which is the claim—no learned sentence selector, no model modification, just a cheap ranking step.","feed_headline":"Ranked sentence cut to 50% holds long-document accuracy","feed_subtitle":"On Marathi long news, half the ranked sentences preserves accuracy and trims inference time by up to 35%.","key_machinery":"The load-bearing object is a TF-IDF-based sentence ranker. Each sentence in the document is scored by term-frequency/inverse-document-frequency style statistics, the sentences are ordered by that score, and the top fraction is kept as the classifier's input. What it does is turn an arbitrarily long document into a fixed-size, high-precision context for a zero-shot classifier, and the paper's quantitative claim—50% of ranked sentences, comparable accuracy, up to 35% faster inference—is the measured output of this mechanism.","core_discovery":"The central claim is that sentence ranking by TF-IDF is a sufficient context-reduction mechanism for zero-shot long-document classification. Using the MahaNews dataset of long Marathi news articles, the authors compare three context-reduction strategies and report that retaining only the top 50% of ranked sentences yields performance comparable to full-document inference, while reducing inference time by up to 35%. This is meant to show that a model trained on short texts, such as headlines, can be adapted to long-form documents by selecting the most informative sentences. The result is framed as architecture-agnostic: no change to the transformer itself, only to the input it receives.","pith_inferences":["A natural extension is to use the same ranking as a cheap pre-filter for retrieval or reading tasks, where a stronger model only sees the top-ranked sentences.","The 50% threshold is likely tuned by news-genre redundancy: Marathi news articles typically front-load the story, so top-ranked sentences carry most label evidence; in genres with diffused evidence the threshold may need to be lower.","Since ranking is by sentence count, not token count, the actual compute saving depends on sentence lengths; 'top 50% of sentences' is not necessarily 'top 50% of tokens'.","The method could serve as a zero-shot benchmark for low-resource languages: no labeled data, no sentence-selection training, and any pretrained multilingual short-text model can be applied."],"forward_implications":["Zero-shot long-document classification becomes a two-stage pipeline: cheap sentence ranking, then standard short-text inference.","Headline-trained models can be pointed at full news articles without any fine-tuning or positional-encoding tricks.","Cutting input context by half lowers compute and latency, making long-document classification more practical at scale.","The same model can process documents longer than its training input, as long as sentence ranking selects the relevant portion."],"supporting_citations":[],"fun_headline_variants":["Top 50% ranked sentences keep long-doc accuracy","Sentence ranking halves input, trims inference up to 35%","For long docs, rank sentences and cut to half","Zero-shot long-doc: top half sentences suffice","Efficient long-doc via sentence ranking: half enough"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"TF-IDF scores computed within each document rank sentences by the label-relevant evidence the zero-shot classifier actually uses; if the deciding clues are scattered across many low-ranked sentences, dropping half the text could remove exactly what the classifier needs.","fun_headline_variants_meta":{"raw":{"variants":["Top 50% ranked sentences keep long-doc accuracy","Sentence ranking halves input, trims inference up to 35%","For long docs, rank sentences and cut to half","Zero-shot long-doc: top half sentences suffice","Efficient long-doc via sentence ranking: half enough"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000559,"raw_usage":{"total_tokens":2458,"prompt_tokens":669,"completion_tokens":1789,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":413,"completion_tokens_details":{"reasoning_tokens":1722}},"tokens_in":413,"tokens_out":1789,"duration_ms":14417,"temperature":1.0,"reasoning_tokens":1722,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T16:50:41.907722+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a text whose decisive sentence is buried among many high-TF-IDF fillers (for example, a synthetic document with one informative sentence and fifty generic but topic-word-heavy filler sentences), run the top-50% ranking, and check whether the classifier still gets the label; if it frequently drops the decisive sentence, the claim that TF-IDF ranking preserves label-relevant information is falsified outside the news distribution.","supporting_citations":[],"review_version":1}