{"id":"889b0b62-3bdb-4bcf-9343-a59ca7cc66ef","arxiv_id":"1907.06370","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Multimodal deep network using image features and OCR word embeddings boosts document classification accuracy by 3% over image-only baselines on Tobacco3482 and RVL-CDIP with a new QS-OCR dataset.","lead":"This paper introduces a multimodal neural network combining document images with text embeddings extracted via OCR to improve classification accuracy. A smart generalist might read it to see how fusing visual layout and textual content can enhance real-world document processing systems like archives and administrative workflows.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"The reader's weakest-assumption note correctly flags missing details in the abstract alone. Once the full text is examined, the fusion method, dataset construction, and empirical comparison are specified, removing the primary source of unverdicted status. No internal inconsistency or untested assumption that would invalidate the 3% claim is present.","tokens_in":1625,"tokens_out":263,"duration_ms":13108,"concrete_test":"Reproduce the image-only and multimodal runs on Tobacco3482 using the exact QS-OCR text files from the GitHub release and the architecture described in §3; confirm that the multimodal accuracy exceeds the image-only accuracy by at least 2.5 percentage points under identical random seeds and hyper-parameters.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a modest 3% accuracy lift from fusing OCR-derived text embeddings with image features on two standard document datasets, using a newly released text augmentation. The argument is internally consistent once the full manuscript supplies the fusion architecture (late fusion of CNN image features and word-embedding text features), training protocol, and baseline comparisons. No hidden assumption about bounded OCR noise or parameter-free derivation is required for the reported result to hold.","agreement_with_reader":"disagree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes a multimodal neural network for document image classification that fuses image features with word embeddings derived from OCR-extracted text. It reports that this fusion yields a 3% accuracy improvement over image-only baselines on the Tobacco3482 and RVL-CDIP datasets when augmented with the newly released QS-OCR text dataset, and that the gain holds even with noisy OCR output rather than clean text.","tokens_in":1687,"tokens_out":299,"duration_ms":13461,"significance":"If the empirical result holds, the work provides evidence that OCR-derived text embeddings supply complementary signal to visual features for fine-grained document classification tasks. The public release of the QS-OCR dataset constitutes a concrete, reusable contribution that can support further multimodal experiments on standard benchmarks.","major_comments":[],"minor_comments":[{"comment":"The abstract states the 3% boost but does not name the fusion architecture (e.g., late fusion of CNN and embedding features), the exact baselines, or any error bars/statistical tests; these details should be added to the abstract or highlighted in §3–4 for immediate evaluability.","section":null},{"comment":"The new QS-OCR dataset is introduced with a GitHub link; the manuscript should include a brief description of its construction, size, and OCR quality statistics in the experimental section to allow readers to assess robustness to OCR noise.","section":null}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the positive summary of our work and the recommendation for minor revision. No specific major comments were raised in the report.","responses":[],"tokens_in":1111,"tokens_out":47,"duration_ms":11906,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper shows that adding text embeddings from OCR to an image-based document classifier improves accuracy by 3% on Tobacco3482 and RVL-CDIP. They also release the OCR text as a new dataset called QS-OCR. What the work does well is the data release. Providing the extracted text for these standard benchmarks lets other researchers test multimodal ideas without extra preprocessing. The approach itself is a straightforward combination of CNN image features and word embeddings, and the experiments use actual OCR output, which matches real-world conditions. The gain is real but limited in scope. It applies to fine-grained classification of administrative and archival documents where text content matters. The paper does not position this as a major architectural advance, which keeps expectations in line with the results. A minor concern is whether the fusion method adds much beyond what a text-only model could achieve, or how sensitive the improvement is to OCR quality. The full paper likely includes the architecture details and tables, but those would need checking for baseline strength and any statistical tests. This is the kind of paper that fits applied computer vision work on documents. Someone building a system for scanning and sorting forms or manuscripts could use the dataset and the reported lift. It is solid enough on the empirical side to go through peer review. I would recommend sending it to peer review.","headline":"The paper delivers a modest 3% accuracy lift on two document datasets by fusing CNN image features with OCR text embeddings and releases the corresponding text data as QS-OCR.","tokens_in":2157,"tokens_out":340,"would_cite":false,"duration_ms":19619,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[],"headline":"Unrelated multimodal document classifier; no RS machinery","alignment":"orthogonal","rationale":"Paper presents late-fusion CNN (MobileNetV2) + FastText 1D-CNN on OCR text for Tobacco3482/RVL-CDIP classification. Central machinery is standard deep-learning feature concatenation and end-to-end training; no J-cost, phi-ladder, 8-tick periodicity, ratio-symmetric forcing, or any theorem from IndisputableMonolith/Foundation or Cost modules. Domain (applied CV/NLP) lies outside RS scope.","tokens_in":48894,"confidence":"high","tokens_out":133,"duration_ms":5594,"cache_read_input_tokens":38528,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Multimodal network fusing image features with OCR word embeddings raises document classification accuracy by 3%.","keywords":["document classification","multimodal learning","OCR text embeddings","deep neural networks","image classification","Tobacco3482","RVL-CDIP"],"falsifier":"Running the identical image-only and multimodal models on Tobacco3482 or RVL-CDIP and observing no accuracy difference or a drop when the OCR text branch is added.","tokens_in":2534,"feed_emoji":"📄","tokens_out":512,"duration_ms":13047,"temperature":0.7,"pith_summary":"The paper designs a neural network that processes both document images and text extracted by OCR to perform classification. Visual analysis alone cannot achieve the fine-grained results needed for archival and administrative documents because key information often resides in the text. By learning from word embeddings computed on the OCR output together with image data, the network improves accuracy over image-only baselines. The improvement holds on Tobacco3482 and RVL-CDIP even when the text comes from imperfect OCR rather than clean ground truth. A new QS-OCR dataset is released to support further work on this combination of signals.","feed_headline":"Multimodal network lifts document image accuracy 3% with OCR text","feed_subtitle":"Fusing visual features and word embeddings from imperfect OCR improves fine-grained classification on standard benchmarks.","key_machinery":"Multimodal neural network that fuses image features with text embeddings derived from OCR output.","core_discovery":"A multimodal neural network that learns jointly from word embeddings computed on OCR-extracted text and from the document image itself improves classification accuracy by 3% over pure image models on the Tobacco3482 and RVL-CDIP datasets augmented with the QS-OCR text dataset, and the gain occurs even without clean text information.","pith_inferences":["The same fusion strategy could be tested on document collections in languages with different OCR error profiles.","Performance might degrade on documents where text and image content conflict rather than reinforce each other.","Future work could measure how much of the gain survives when the OCR engine is replaced by a weaker or stronger model."],"forward_implications":["Document classification systems can improve by incorporating OCR text even when the OCR is noisy.","Fine-grained distinctions that depend on textual content become reachable without requiring perfectly transcribed text.","The new QS-OCR dataset provides a public resource for training and evaluating multimodal document models.","The 3% gain demonstrates that image and text modalities are not redundant for this task."],"fun_headline_variants":["Multimodal network gains 3% on document classification with OCR text","Multimodal models gain 3% accuracy from OCR text on documents","Document classification accuracy improves 3% with multimodal OCR","Multimodal fusion of image and OCR text gains 3% accuracy"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That OCR-derived text embeddings supply complementary signal the network can fuse with image features to produce reliable accuracy gains.","fun_headline_variants_meta":{"raw":{"variants":["Multimodal network gains 3% on document classification with OCR text","Multimodal models gain 3% accuracy from OCR text on documents","Document classification accuracy improves 3% with multimodal OCR","Multimodal fusion of image and OCR text gains 3% accuracy"]},"model":"grok-4.3","cost_usd":0.006149,"raw_usage":{"total_tokens":2850,"prompt_tokens":566,"num_sources_used":0,"completion_tokens":73,"cost_in_usd_ticks":61487000,"prompt_tokens_details":{"text_tokens":566,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2211,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":566,"tokens_out":73,"duration_ms":15090,"temperature":1.0,"reasoning_tokens":2211,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-24T21:40:47.835256+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the identical image-only and multimodal models on Tobacco3482 or RVL-CDIP and observing no accuracy difference or a drop when the OCR text branch is added.","supporting_citations":[],"review_version":1}