{"id":"337a6a4a-be3c-43f4-92df-0719af0c3f70","arxiv_id":"2607.11562","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A document-oriented ViT family pretrained with text generation plus pixel reconstruction on 113M images transfers across recognition, detection, parsing, and understanding, setting open-source SOTA on MDPBench with a tiny frozen encoder.","lead":"MonkeyOCRv2 is a document-native vision encoder pretrained on 113M multilingual document images with joint text generation and pixel reconstruction. Kept frozen with a small LLM, a 0.7B parser beats larger open-source systems on multilingual document parsing while also lifting five classical OCR-style tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The headline MDPBench SOTA is not cleanly attributable to the encoder under the paper's own system-level caveat.","rationale":"The reader's weakest assumption (automatic multi-expert/LLM labels correlating with evaluation) is real but secondary: official train splits only, held-out tests, and consistent gains on external detection/tampering/segmentation benchmarks reduce the chance that label noise alone explains the whole story. The more load-bearing issue for the packaged strongest claim is causal attribution of the MDPBench SOTA. The paper itself routes encoder credit to controlled Tab. 8 and flags Tab. 7 as system-level; applying that standard to Tab. 6 leaves the flagship +2.8 result under-supported as pure visual-foundation evidence. That does not overturn the multi-task transfer story or justify REJECT; it keeps CONDITIONAL with the condition sharpened: either ship the matched-encoder parsing ablation (or code that makes it trivial) or reframe the parsing claim as system-level rather than encoder-foundation SOTA. Agreement with the reader is partial because label quality and same-lineage risk are related, but the decisive missing control is the frozen-encoder swap inside the parsing pipeline.","tokens_in":40310,"tokens_out":742,"duration_ms":7071,"concrete_test":"Retrain the Sec. 4.6 two-stage parsing recipe with the encoder frozen, swapping only Ev for OpenVision-B (or RADIOv2.5-B) at matched visual-token budget and the same Qwen3-0.6B + MLP; evaluate on MDPBench under the official protocol. If the OpenVision/RADIO parser is within ~1 point of MonkeyOCRv2-B-Parsing's 83.3, the headline +2.8/11× claim is not encoder-attributable; if the gap remains ≥2 points, the concern is largely defused.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim packages backbone-substitution gains on five tasks with a 0.7B frozen-encoder parser that is +2.8 over 3B dots.mocr on MDPBench (Tab. 6) and beats CLIP/DINO/SAM under matched VLM training (Tab. 8). The five-task and Tab. 8 evidence is relatively clean. The load-bearing soft spot is the parsing headline: Sec. 4.6 and Tab. 7 explicitly state that OmniDocBench is a system-level comparison (different data, post-training, layout modules, pipelines) and that encoder-level attribution should use Tab. 8, not Tab. 7. MDPBench is still reported as open-source SOTA for MonkeyOCRv2-Parsing, yet the same architecture couples a frozen encoder to an autoregressive layout predictor plus per-element re-crop recognition and assembly (Sec. 4.6). No matched ablation freezes a strong alternative encoder (e.g., OpenVision-B, RADIOv2.5-B, or a SigLIP2/Keye-style backbone) inside that exact parsing pipeline and training recipe. Without that, the +2.8 and 11×-smaller-encoder framing can credit pipeline design and same-lineage MDPBench construction (authors share the MDPBench line) as much as document-native pretraining. The reconstruction ablations (Fig. 5, Tab. 8/9) support the objective but do not close the parser attribution gap.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"MonkeyOCRv2 proposes a document-native visual encoder pretrained on MonkeyDoc v2 (113M images, 17 languages) with a joint objective of image-to-text generation and pixel-level reconstruction (Eqs. 1–11). The encoder is evaluated as a backbone substitution on five document analysis tasks (text recognition, formula recognition, text detection, tampering detection, overlapping text segmentation) and, frozen, as the vision tower of lightweight VLMs for document parsing and understanding. The paper reports consistent gains from encoder replacement (Tabs. 2–5, Fig. 4), open-source SOTA on MDPBench for a 0.7B frozen-encoder parser (+2.8 over 3B dots.mocr with a much smaller ViT; Tab. 6), and superior document-understanding scores versus CLIP/DINO/SAM/OpenVision under matched LLM, data, and training (Tab. 8). Reconstruction is further supported by scrambled-text and CHAOS-Bench analyses (Sec. 5.1–5.2).","tokens_in":40734,"tokens_out":1453,"duration_ms":19351,"significance":"If the transfer results hold under the stated controls, the work is a substantial contribution to document AI: it argues, with multi-task evidence, that character-level document pretraining can serve as a foundation rather than a domain adaptation of natural-image encoders. Strengths include a large multilingual corpus, a dual-objective recipe with explicit reconstruction ablations (MSE and structure-aware variants in Tab. 8; scrambled-text gap narrowing in Fig. 5), controlled frozen-encoder VLM comparisons on eight benchmarks (Tab. 8), and honest system-level caveats for OmniDocBench (Sec. 4.6, Tab. 7). The five-task backbone-swap protocol is particularly useful for the community. Code and data release is promised, which would further raise impact.","major_comments":[{"comment":"Sec. 4.6 and Tab. 6 present MonkeyOCRv2-Parsing as open-source SOTA on MDPBench (+2.8 over dots.mocr, ~11× smaller vision encoder). The same section correctly notes that OmniDocBench (Tab. 7) is system-level and that encoder attribution should use Tab. 8. That caveat is not applied to MDPBench: the parser couples a frozen encoder to autoregressive layout prediction, per-element re-crop recognition, and assembly, with no matched experiment that freezes a strong alternative encoder (e.g., OpenVision-B, RADIOv2.5-B, SigLIP 2) inside the identical parsing pipeline, data, and training recipe. Without that control, the headline +2.8 and size framing can credit pipeline design and training as much as document-native pretraining. Please either add the matched encoder swap in the parsing stack or reframe Tab. 6 as a system result and rest the encoder claim primarily on Tabs. 2–5 and Tab. 8.","section":null},{"comment":"Sec. 3.1 (Expert Model Labeling; Data Filtering) relies on multi-expert OCR agreement and LLM layout/reading-order filters for large-scale supervision. The residual error structure of those automatic labels is not quantified against human gold or against the evaluation stacks (including MDPBench, which shares research-lineage with prior MonkeyOCR work). If label failure modes correlate with the pretraining objective or with same-lineage benchmarks, measured gains partly reflect label–model correlation. A short audit—e.g., human agreement rates on a stratified sample, or performance when pretraining only on fully public human-annotated subsets—would strengthen the claim that gains are representation quality rather than supervision artifacts.","section":null},{"comment":"Tab. 8 is the cleanest encoder-level comparison, but input configurations differ substantially (App. D: CLIP 196 tokens vs SAM 4096 vs MonkeyOCRv2 ~1082). The paper states each encoder uses its native setting, which is reasonable, yet token budget and resolution are known confounders for document VQA. A sensitivity check that equalizes approximate visual-token count (or reports a fixed-token budget ablation for the top baselines) would make the 13.2-point gap over OpenVision-B more attributable to pretraining rather than resolution policy.","section":null}],"minor_comments":[{"comment":"Abstract and Fig. 2(a) lead with the MDPBench SOTA and 11× smaller encoder; after addressing the major comment on attribution, align abstract wording with the revised claim so abstract and Sec. 4.6 do not over-promise encoder-only causality.","section":null},{"comment":"Eq. (11) sets λ=1.0 with α, β, T, τ fixed without tuning (Sec. 3.2). A brief sensitivity note (even one-dimensional in λ) would help readers assess robustness of the dual-objective balance.","section":null},{"comment":"Table 1 lists MonkeyDoc v2 as 113M multi-type documents / 17 languages; App. A shows strong English/Chinese skew. Mentioning this imbalance earlier (not only in Limitations) would set expectations for low-resource scripts.","section":null},{"comment":"Fig. 4 uses bar charts without numeric tables in the main text; adding exact F-measures in a small table or appendix would aid citation and reproducibility.","section":null},{"comment":"Typo/consistency: abstract says “previous best 3B dots.mocr” while related work and Tab. 6 use “dots.mocr”; unify naming. Also “UniMERNet-T to outperform the 325M UniMERNet-B” is clear in Tab. 3 but the abstract’s “enabling the 110M” phrasing could state ExpRate/CDM explicitly.","section":null},{"comment":"Sec. 5.1 correctly treats the semantic–scrambled gap as an operational proxy; consider moving one sentence of that caveat into the figure caption of Fig. 5 so casual readers do not over-interpret the gap as a pure hallucination metric.","section":null}],"recommendation":"major_revision","confidential_remarks":"The five-task swaps and Tab. 8 are publishable evidence for a document foundation encoder; the main risk is marketing the MDPBench number as encoder SOTA when the authors’ own OmniDocBench discussion shows they understand system-level confounding. MDPBench co-authorship lineage is disclosed via citations but still warrants editorial attention if the venue prioritizes independent benchmarks. I would accept after the parsing claim is either controlled or reframed—not reject. Fit for a top CV/ML venue is good if the revision is tight."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful core here is a from-scratch document encoder family (S/B/AS) trained on a 113M, 17-language corpus with joint image-to-text generation and pixel reconstruction, then shown to transfer as a backbone swap on five classical tasks and as a frozen vision tower under matched VLM training.\n\nWhat is new is the scale and multilingual breadth of MonkeyDoc v2 relative to DiT/Donut/Pix2Struct/oCLIP/DiG, plus the dual objective evaluated as a standalone backbone rather than only inside an end-to-end VLM. The five-task swaps are the cleanest evidence: CRNN and PARSeq recognition, UniMERNet formula, DBNet/PSENet/DPText-DETR detection, FFDN tampering, Mask2Former/MOTS overlapping segmentation all improve under the paper’s protocols, and the 110M formula model beats the 325M baseline. Tab. 8 is also clean: same Qwen3-1.7B, same data and schedule, frozen encoders only; MonkeyOCRv2 beats CLIP, SigLIP2, DINO, SAM, OpenVision, RADIO, oCLIP, and DiT. Reconstruction ablations (MSE then structure-aware) and the scrambled-text / CHAOS-Bench analyses support the claim that reconstruction keeps stroke-level evidence when language priors are weak. Citations and math look ordinary and solid; free parameters are the usual loss weights and LR, not hidden knobs.\n\nSoft spots, in proportion. The stress-test is right on the parsing headline: Tab. 6’s +2.8 over 3B dots.mocr with an 11× smaller ViT is system-level (autoregressive layout + re-crop recognition + assembly, frozen encoder + 0.6B LLM). The paper itself says OmniDocBench is not encoder-attributable and points to Tab. 8 for that; there is no matched ablation that freezes OpenVision/RADIO/SigLIP2 inside the same parsing pipeline. MDPBench shares authorship lineage, so treat the open-source SOTA claim as useful but not pure encoder proof. Automatic multi-expert labeling plus LLM filters is a load-bearing premise that is not independently audited. Code and data are promised, not shipped yet. None of that undoes the backbone-swap or Tab. 8 results.\n\nThis is for people building OCR stacks, document VLMs, or enterprise digitization who need a reusable document vision tower. It deserves a serious referee. I would engage, cite the encoder-swap and controlled VQA results, and discount the MDPBench framing until a matched pipeline ablation appears.","headline":"Solid document-native encoder with clean multi-task transfer; the MDPBench SOTA headline is real but not cleanly encoder-attributable.","tokens_in":41394,"tokens_out":630,"would_cite":true,"duration_ms":7375,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A document-native vision encoder, trained with text generation plus pixel reconstruction, transfers across OCR, parsing, and understanding better than natural-image foundations.","keywords":["document AI","visual pretraining","OCR","document parsing","document understanding","image reconstruction","multilingual documents","vision foundation models"],"falsifier":"Under fully matched data, optimization, and decoding, freeze MonkeyOCRv2 and a strong natural-image encoder of similar size, train the same lightweight document parser, and check whether the document-native encoder still wins on photographed non-Latin pages of MDPBench and on scrambled-text recognition at low resolution; a clear loss would falsify the claim that the dual objective, not scale or task setup, is doing the work.","tokens_in":41221,"feed_emoji":"📄","tokens_out":927,"duration_ms":13334,"temperature":0.7,"pith_summary":"Mainstream vision encoders learn object and scene semantics from natural photos, so they miss the stroke-level detail that document images depend on. This paper argues that a dedicated visual foundation for documents is possible: pretrain an encoder on a large multilingual document corpus so that it both reads text and reconstructs pixels. The generation objective ties features to words; the reconstruction objective keeps character strokes, glyphs, and layout that pure language supervision would throw away. Swapping that encoder into existing systems improves text recognition, formula recognition, detection, tampering localization, and overlapping-text segmentation. Kept frozen and paired with a small language model, it also drives competitive multilingual document parsing and stronger document understanding than CLIP-, DINO-, or SAM-style backbones under matched training. The practical claim is that document intelligence can rest on its own visual pretraining rather than borrowed natural-image features.","feed_headline":"Tiny document encoder beats larger open parsers by 2.8 points","feed_subtitle":"Joint text generation and pixel reconstruction make a vision backbone built for strokes, not scenes.","key_machinery":"Dual-objective document pretraining on MonkeyDoc v2 (113M images, 17 languages): image-to-text generation aligns visual tokens with textual content, while pixel-level reconstruction (MSE, optionally edge- and distance-aware) forces the encoder to retain strokes, glyphs, and layout that text supervision alone can discard.","core_discovery":"Document-oriented pretraining that jointly optimizes image-to-text generation and pixel-level reconstruction produces transferable character-level visual representations: as a backbone swap it improves five document analysis tasks, and as a frozen encoder with a lightweight language model it yields a 0.7B parser that reaches open-source state of the art on multilingual MDPBench while also outperforming CLIP, DINO, and SAM counterparts on eight document-understanding benchmarks under identical training.","pith_inferences":["The same dual recipe may help other dense-symbol domains (sheet music, circuit diagrams, engineering drawings) where global semantic encoders discard local marks.","If reconstruction is what narrows the semantic–scrambled gap, progressive post-training of the language head alone may not close remaining gaps on saturated parsing benches without stronger visual evidence.","Balancing MonkeyDoc-style supervision toward low-resource and historical scripts would be a direct test of whether the method generalizes beyond high-resource languages that dominate the current mix."],"forward_implications":["Document systems can replace ImageNet, CLIP, DINO, or SAM backbones with a compact document encoder and expect gains without rewriting the rest of the pipeline.","A frozen ~0.1B document vision encoder plus a ~0.6B language model is enough for competitive multilingual parsing, so large general VLMs are not required for that task class.","Pixel reconstruction should reduce reliance on language priors when text is scrambled, low-resolution, or deliberately conflicted with linguistic expectations.","Future document foundation work can treat text strokes and layout as first-class visual targets rather than side effects of semantic alignment."],"fun_headline_variants":["0.7B frozen document encoder beats 3B open parsers by 2.8 points","Joint text gen and pixel rebuild yields transferable document vision","MonkeyOCRv2 backbone swap lifts five core document analysis tasks","Frozen visual-text encoder tops CLIP DINO SAM on eight doc benchmarks","Character-level pretraining for docs sets open SOTA on multilingual MDPBench"],"cache_read_input_tokens":32896,"weakest_assumption_plain":"The large training labels from multi-expert agreement and automatic layout filters are clean enough that measured gains reflect better visual features rather than shared errors between those labels and the evaluation stack.","fun_headline_variants_meta":{"raw":{"variants":["0.7B frozen document encoder beats 3B open parsers by 2.8 points","Joint text gen and pixel rebuild yields transferable document vision","MonkeyOCRv2 backbone swap lifts five core document analysis tasks","Frozen visual-text encoder tops CLIP DINO SAM on eight doc benchmarks","Character-level pretraining for docs sets open SOTA on multilingual MDPBench"]},"model":"grok-4.5","effort":"low","cost_usd":0.004328,"raw_usage":{"total_tokens":1374,"prompt_tokens":883,"num_sources_used":0,"completion_tokens":79,"cost_in_usd_ticks":43280000,"prompt_tokens_details":{"text_tokens":883,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":412,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":883,"tokens_out":79,"duration_ms":3963,"temperature":1.0,"reasoning_tokens":412,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T04:44:35.070340+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Under fully matched data, optimization, and decoding, freeze MonkeyOCRv2 and a strong natural-image encoder of similar size, train the same lightweight document parser, and check whether the document-native encoder still wins on photographed non-Latin pages of MDPBench and on scrambled-text recognition at low resolution; a clear loss would falsify the claim that the dual objective, not scale or task setup, is doing the work.","supporting_citations":[],"review_version":1}