{"id":"3620158a-2176-4608-a674-0db790ac2eb2","arxiv_id":"2505.11177","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A pipeline combining Tesseract, Gemini, and Google Translate claims 88% classification accuracy and moderate BLEU/ROUGE scores, but provides no reproducible evaluation.","lead":"This paper stitches together existing tools (Tesseract OCR, Gemini, Google Translate, BART) into a multilingual document pipeline for Indic languages. It reports evaluation numbers, but gives no datasets, no code, and no details on how the numbers were produced.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation numbers in Section 5 are unverifiable: no datasets, splits, metric details, or post-OCR correction definition are provided.","rationale":"I read the paper as a short system description that makes concrete quantitative claims about OCR, classification, summarization, and translation performance. The load-bearing requirement is that these numbers come from actual measurements on described data under a described protocol. Section 5 fails to provide datasets, splits, reference summaries/translations, raters, or a definition of the post-OCR correction; Section 4.2 lists only preprocessing steps and no correction module; Figure 4 references are broken. There is no code, appendix, or reproducibility statement. The reader's verdict of REJECT is appropriate because the core evidence is absent rather than merely weak. My concern is identical to the reader's weakest assumption: the evaluation results are unverifiable as stated. I do not base the rejection on novelty overlap or self-citation alone, though the reference [1] to a GitHub repository without disclosure is a secondary transparency issue. Agreement: the reader identified the same load-bearing concern about unverifiable evaluations, so I mark agree. The verdict should remain REJECT (UNCHANGED), with CONDITIONAL acceptance as an alternative only if the authors supply the missing evaluation details and release data/code for verification.","tokens_in":4934,"tokens_out":1342,"duration_ms":10935,"concrete_test":"Request a complete evaluation appendix from the authors specifying for each metric: (1) dataset names, sizes, and train/test splits; (2) the exact reference texts used for ROUGE/BLEU and the metric implementation (e.g., sacreBLEU signature); (3) the CER/WER computation details, including whether OCR output was evaluated against clean ground truth and which languages were included; (4) a precise description of the post-OCR correction module and its ablation result; and (5) human evaluation instructions, number of raters, and raw scores. If any of these cannot be supplied, the Section 5 numbers are unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the integrated pipeline achieves reported quantitative performance (88% classifier accuracy, 12.7% CER, 18.4% WER, post-OCR WER improvement of 3.4 points, ROUGE-1 0.41–0.56, BLEU 18.7–32.1, human ratings). These numbers are asserted in Section 5 without any description of the evaluation datasets, their sizes, language distributions, reference construction, or the protocol used to compute CER/WER and summary/translation metrics. The claimed post-OCR correction is never defined, even though Table 1 and Section 5 mention it as a result. Figure 4a references are broken (Figure ??), so no visual evidence is provided. If the datasets or protocol do not exist or the metrics were computed differently, the headline performance claim collapses. The paper frames itself as a working pipeline, but the quantitative claims are the only evidence that the pipeline works; without that evidence, the central claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes an end-to-end multilingual document-processing pipeline that combines Tesseract OCR, text preprocessing, abstractive summarization via Cohere or Gemini, Google Translate, sentiment analysis, topic classification, and date extraction, all exposed through a Gradio interface. It reports classification accuracy comparisons (Table 1), OCR error rates (CER 12.7%, WER 18.4%), summarization quality (ROUGE-1 0.41–0.56, ROUGE-L 0.39, human rating 3.7/5), and translation quality (BLEU 18.7–32.1, human adequacy 72%, fluency 68%), and claims the pipeline is suitable for low-resource Indian languages. The manuscript provides no evaluation datasets, dataset sizes, language pair lists, reference construction details, or metric computation protocols, and several figure references are broken.","tokens_in":5188,"tokens_out":5154,"duration_ms":49133,"significance":"If the reported performance were supported by a reproducible evaluation, the pipeline could be a useful integration of existing tools for Indic-language document access, particularly for users needing summarized and translated content from images. The paper's strength is the modular assembly of readily available APIs and models, and the design choices are easy to follow. However, it introduces no new method, provides no code or evaluation artifacts, and its quantitative claims are currently unverifiable because the evaluation is not described. The significance of the contribution therefore depends entirely on evidence that is absent from the manuscript.","major_comments":[{"comment":"All quantitative results are asserted without any description of evaluation datasets, sizes, language distributions, or metric computation protocols. The classification accuracies in Table 1, the CER/WER averages, the ROUGE/BLEU scores, and the human ratings are presented as bare numbers; the reader cannot verify whether the error rates were computed on OCR output, whether reference summaries and translations were human-authored, how many documents were used, or which language pairs were tested. Because these numbers are the only evidence that the pipeline works, the central claim is unsupported.","section":"Section 5"},{"comment":"The 'post-OCR correction' that improves average WER by 3.4 percentage points is never defined. The paper does not state whether this refers to a learned model, a rule-based normalizer, or a component of the pipeline, nor does it explain how the correction was applied to the OCR output. Without this definition, the reported OCR error rates cannot be interpreted or reproduced.","section":"Section 5 (OCR Performance Impact)"},{"comment":"The description of the classification component is internally inconsistent: Section 3 says SVM and Random Forest are used, the Introduction mentions zero-shot facebook/bart-large-mnli, and Section 5 evaluates Logistic Regression, SVM, Random Forest, and XGBoost and selects Logistic Regression. The reader cannot determine which classifier is actually in the pipeline, so the '88% accuracy' claim may not correspond to the system being described.","section":"Sections 3–5"},{"comment":"The human evaluation numbers (3.7/5 for summaries, 72% adequacy and 68% fluency for translations) are reported without any details on the number of raters, the language pairs involved, the annotation instructions, or inter-annotator agreement. As with the automatic metrics, these numbers are not sufficient to support the claimed reliability of the pipeline.","section":"Section 5 (Translation Performance)"},{"comment":"Figure references are broken: the text cites 'Figure ??' twice for the BBC Sport dataset and the model accuracy comparison, and the figures are not connected to the metric claims. The visual evidence is therefore not accessible, which further weakens the presentation of the results.","section":"Section 5 / Figures"}],"minor_comments":[{"comment":"The abstract says sentiment analysis is performed with a custom-trained TensorFlow model, but no training or evaluation for that model is described anywhere, and the pipeline stages in Section 4 do not mention sentiment analysis as a module.","section":"Abstract and Section 4"},{"comment":"The 'Limitation' column entries are not objective limitations and some are misleading (e.g., 'assumes linear relationships' is not inherently a limitation for all datasets). The accuracy values are also reported as integers without confidence intervals, standard deviations, or dataset sizes.","section":"Table 1"},{"comment":"Removing stop words and punctuation before feeding text to an abstractive summarization API may remove information that is useful for summarization, and the paper does not justify this aggressive preprocessing for the summarization stage.","section":"Section 4.2"},{"comment":"Reference [1] is an unpublished GitHub repository with a placeholder access date, and references [17] and [19] both refer to the same underlying artifact (FLORES-200/NLLB), which should be cited cleanly.","section":"References"},{"comment":"The conclusion repeats the reported numbers without synthesizing the results, discussing limitations of the pipeline, or providing concrete directions for improvement; it also overstates the reliability of the system given the missing evaluation details.","section":"Section 6"}],"recommendation":"reject","confidential_remarks":"The manuscript, in its current form, is not yet a research paper: the empirical core of the work is a list of unverifiable metrics. I recommend rejection. If the authors can supply a complete evaluation protocol with defined datasets, sizes, reference construction, and the missing definition of the post-OCR correction, a resubmission might be considered. It may also be worth verifying that the cited GitHub repository (reference [1]) actually contains the claimed pipeline and that the reported numbers can be reproduced from it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a student-project write-up for a pipeline that chains Tesseract, Gemini/Cohere, Google Translate, and BART. That integration is fine for a prototype, but the paper presents no new model, dataset, or method. The only thing that could make it a research contribution, the evaluation in Section 5, is unverifiable.\n\nWhat's genuinely decent: the problem is real — scanned documents in Indian languages are hard to access, and turning them into summarized, translated text with free tools is useful. The pipeline stages are sensible, and using off-the-shelf components is the right call for a demo.\n\nThe soft spots are not minor. Every number in Section 5 comes without context. There is no description of the datasets, their sizes, language splits, reference summaries, or how CER/WER/ROUGE/BLEU were computed. The 'post-OCR correction' that supposedly improves WER by 3.4 points is never defined. Figure references in Section 5 are broken (Figure ?? appears twice), so there is no visual evidence. Human ratings (3.7/5 summary, 72% adequacy, 68% fluency) are presented with no protocol. The conclusion then claims 'reliability across the range of Indic scripts tested' — but we don't know what was tested.\n\nAlso worth noting: reference [1] is a GitHub repo that looks like the authors' own ('JC230903'), cited without disclosure. That's not fatal by itself, but combined with the phantom evaluation, it gives the impression of padding.\n\nThe literature survey is a paragraph of vague statements, and the references, while numerous, are not actually connected to the claims. But the core issue is the missing evaluation.\n\nIs this paper worth a serious referee? No. As a standalone preprint, it doesn't meet the bar: there is no way to check any result. I'd tell the authors to release the code, data, and a clear protocol if they want credit for the integration. For now, it's a demo description, not a research paper.\n\nRecommendation: desk reject. Not because the pipeline is useless, but because the evidence doesn't exist.","headline":"An off-the-shelf OCR-to-NLP demo whose evaluation numbers are asserted, not demonstrated; no new research contribution, but a legitimate prototype concept.","tokens_in":5616,"tokens_out":3687,"would_cite":false,"duration_ms":34114,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A five-stage OCR-to-translation pipeline aims to make scanned Indian-language documents accessible, and reports stagewise metrics including 88% classification accuracy.","keywords":["low-resource languages","OCR pipeline","text summarization","machine translation","Indic scripts","document processing","zero-shot classification","multilingual NLP"],"falsifier":"Run the same five-stage pipeline on a public Indic OCR benchmark such as IIIT-ILOCR, measure character error rate and word error rate before and after the correction step, then measure translation BLEU on the FLORES Indic language pairs; if the correction step produces no word error rate change or BLEU degrades, the central integration claim fails.","tokens_in":4699,"feed_emoji":"📄","tokens_out":5068,"duration_ms":49823,"temperature":0.7,"pith_summary":"The paper argues that a modular pipeline of off-the-shelf components—OCR, text preprocessing, abstractive summarization, machine translation, and an interactive output interface—can turn image-based documents in low-resource Indian languages into concise, translated summaries. It reports concrete measurements: average character error rate of 12.7%, word error rate of 18.4% (improved by about 3.4 points after correction), ROUGE-1 scores of 0.41 to 0.56, BLEU scores of 18.7 to 32.1, and 88% accuracy for a logistic-regression topic classifier. The sympathetic reading is that existing free and API-based tools, when carefully chained, are enough to serve languages and scripts that dedicated NLP systems often ignore.","feed_headline":"OCR pipeline turns scanned Indic documents into translated summaries","feed_subtitle":"Reported stagewise scores: 88% classifier accuracy, 12.7% character error, BLEU from 18.7 to 32.1.","key_machinery":"The load-bearing object is the pipeline itself: image input, OCR via Tesseract, text preprocessing, abstractive summarization through Cohere or Gemini, machine translation through Google Translate or Gemini, and output through a Gradio interface, with auxiliary sentiment, topic classification, and date extraction modules. The mechanistic claim is that each off-the-shelf stage preserves enough information for the next stage, such that OCR errors do not cascade beyond usability, and that the zero-shot BART classifier adds useful document context without task-specific training.","core_discovery":"The central claim is that an end-to-end pipeline built from publicly available OCR, a large-language-model API for abstractive summarization, a translation API, and a BART-based zero-shot topic classifier can process image documents in Indic scripts into summarized, translated text with usable quality. The paper presents stagewise numbers as evidence, including 12.7% average character error rate, 18.4% word error rate, ROUGE-1 of 0.41 to 0.56, BLEU of 18.7 to 32.1, human adequacy at 72%, fluency at 68%, and a summary quality rating of 3.7 out of 5. It further claims that the pipeline's modular architecture and its chosen logistic-regression classifier, at 88% accuracy, make the system suitable for resource-constrained multilingual settings.","pith_inferences":["Beyond the paper, the reported metrics are not tied to any named public dataset in the text, so the strongest testable inference is that the numbers should reproduce on a standard Indic OCR benchmark; if they do, the integration claim is robust.","The pipeline's reliance on commercial APIs leaves open the question of reproducibility and long-term cost; a local-model variant using open-weight summarization and translation models would be the natural extension.","The paper's own observation that OCR quality gates machine translation suggests a concrete experiment the paper does not run: deliberately degrade OCR output in controlled increments and measure the resulting BLEU drop for a fixed language pair.","If the 3.4-point WER improvement is genuine, it should translate into a measurable BLEU gain; the paper does not report that, so verifying that link would strengthen the pipeline's causal story."],"forward_implications":["Scanned documents in Hindi, Tamil, Bengali, and similar languages can be summarized and translated without custom model training, using a combination of free OCR and commercial APIs.","The reported 3.4-point word error rate improvement from post-OCR correction suggests that lightweight correction steps materially benefit downstream summarization and translation.","A logistic-regression classifier at 88% accuracy, with lower computational cost than alternatives, indicates that simple models remain competitive for document classification in low-resource settings.","The BLEU range from 18.7 to 32.1 implies that translation quality varies sharply with language-pair resource availability, so low-resource pairs remain the clear bottleneck.","If the pipeline holds together, it provides a template for other low-resource language communities to build document-access tools from existing APIs rather than waiting for dedicated models."],"supporting_citations":[{"why":"Serves as the companion implementation and the concrete artifact the paper describes and evaluates.","marker":"[1]"},{"why":"Supplies the benchmark context for OCR on Indian scripts that the pipeline's OCR stage builds upon.","marker":"[2]"},{"why":"Introduces BART, the pretrained model underlying the zero-shot topic classifier used in the pipeline.","marker":"[12]"},{"why":"Provides the human-centered low-resource machine translation background that frames the BLEU expectations.","marker":"[19]"},{"why":"Defines BLEU, the metric used to score the translation stage's output.","marker":"[36]"}],"fun_headline_variants":["OCR-LLM pipeline for scanned Indic docs hits BLEU 32.1","Image docs in Hindi, Tamil get OCR summaries and translation","OCR pipeline: 88% accuracy, BLEU 18.7-32.1 for Indic scripts","Scanned Hindi/Tamil text: OCR to summary, BLEU up to 32.1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's central claim rests on the assumption that the reported evaluation numbers come from actual measurements made on defined documents under the described protocol, and that the post-OCR correction step that improves word error rate by about 3.4 points really exists and was applied.","fun_headline_variants_meta":{"raw":{"variants":["OCR-LLM pipeline for scanned Indic docs hits BLEU 32.1","Image docs in Hindi, Tamil get OCR summaries and translation","OCR pipeline: 88% accuracy, BLEU 18.7-32.1 for Indic scripts","Scanned Hindi/Tamil text: OCR to summary, BLEU up to 32.1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000551,"raw_usage":{"total_tokens":2576,"prompt_tokens":842,"completion_tokens":1734,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":1641}},"tokens_in":458,"tokens_out":1734,"duration_ms":12429,"temperature":1.0,"reasoning_tokens":1641,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:55:21.491654+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same five-stage pipeline on a public Indic OCR benchmark such as IIIT-ILOCR, measure character error rate and word error rate before and after the correction step, then measure translation BLEU on the FLORES Indic language pairs; if the correction step produces no word error rate change or BLEU degrades, the central integration claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Serves as the companion implementation and the concrete artifact the paper describes and evaluates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the benchmark context for OCR on Indian scripts that the pipeline's OCR stage builds upon."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces BART, the pretrained model underlying the zero-shot topic classifier used in the pipeline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines BLEU, the metric used to score the translation stage's output."}],"review_version":1}