{"id":"afe7556e-ae08-4734-a539-dcc525a13817","arxiv_id":"1908.09231","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A Mask R-CNN and attention-based text spotter handles curved text by masking RoI features instead of rectifying them, and uses OCR-engine labels as extra training data to set state-of-the-art results on ICDAR15 and Total-Text.","lead":"This paper presents an end-to-end neural network that detects and reads text of arbitrary shape in a single pass, using Mask R-CNN for detection and an attention decoder for recognition. It reports large gains over prior systems on curved-text benchmarks, though part of the gain comes from extra training data generated by a proprietary OCR engine.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline SOTA margins are confounded by 1M proprietary OCR-labeled images; public-only ICDAR15 numbers and label-noise/mask details are missing, so the 4.6%/16% gains are not yet cleanly attributable to the architecture.","rationale":"The central claim is conditional on the partially labeled data being an honest, transferable training signal. The paper shows a positive ablation, which is real evidence, and the public-only Total-Text row supports the architecture. But the headline comparisons are not clean: the strongest numbers use 1M proprietary labels and 30k private manual labels, and the exact handling of mask-free labels is omitted. This does not invalidate the core method; it means the SOTA margins should not be accepted as cleanly attributable without a public-data-only control on ICDAR15 and a noise/mask analysis. This is the same weak spot the reader identified, though I weight the undisclosed-data confound more heavily than the label-noise question per se; hence 'partial' agreement. No rejection is warranted because the ablations and public-only Total-Text result provide independent support for the architecture.","tokens_in":14140,"tokens_out":7307,"duration_ms":80366,"concrete_test":"Run the full pipeline with the same architecture and only public fully-labeled images (SynthText, ICDAR15, COCO-Text, ICDAR-MLT, Total-Text) and report official E2E F-scores on both benchmarks; then add 1M unlabeled images with teacher labels from an independent open-source OCR engine (or from Cloud Vision with a hand-transcribed 1k-image subset to measure word/character accuracy) and see whether the 4.6%/16% margins persist. If public-only margins collapse or gains disappear with an independent teacher, the headline claims should be revised to credit the private data, not the method.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central SOTA claim rests on models trained with two non-public components: 30k manually labeled web images and 1M images labeled by the Google Cloud Vision API (PhotoOCR, whose authors include this paper's authors). Table 3 shows the partial-label branch supplies a large share of the gain (ICDAR15 AP E2E: 53.1 with RoI masking alone to 58.7 with partial labels, then 59.0 full), and the public-only comparison is reported only for Total-Text (63.9 E2E), not for ICDAR15. Section 3.5 says only that low-confidence teacher outputs are filtered; it does not quantify teacher label accuracy, does not describe how oriented-rectangle machine labels are converted into instance masks for the RoI-masking step that Section 3.3 makes essential, and does not check overlap between the 1M web images and test sets. If teacher errors are biased or if most of the gain is scale of private data, the headline margins are not evidence for the architecture. The architecture itself has some independent support (ablations, public Total-Text), so the concern is attribution/reproducibility, not an internal contradiction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an end-to-end text spotting model based on Mask R-CNN as the detector and a seq2seq attention decoder as the recognizer. Instead of rectifying detected text regions before recognition, the model applies a proposed RoI masking step that multiplies cropped features by the predicted instance segmentation mask, allowing the attention decoder to attend to arbitrarily shaped text. To address the shortage of fully annotated training images, the authors add partially labeled data produced by the Google Cloud Vision API OCR engine on one million web images, training only the recognizer branch on such samples. The model is evaluated on ICDAR15 and Total-Text, where it reports state-of-the-art end-to-end F-scores (4.6% and 16.7% relative improvements over prior work, respectively). Ablations indicate consistent gains from RoI masking and from partial labels.","tokens_in":14418,"tokens_out":5289,"duration_ms":51961,"significance":"If the reported results hold, the paper makes a useful contribution to scene text spotting by showing that feature rectification can be avoided: RoI masking plus an attention decoder directly handles curved and rotated text. The internal ablations support the value of RoI masking and partially labeled data, and the public-data-only Inception-ResNet model still improves on previous Total-Text results, giving some independent evidence for the architecture. The main caveat is attribution: the headline benchmark claims rest on 30k private web images and one million machine-labeled images, so the reported margins may overstate the architectural contribution. Still, the paper is an important datapoint for the community, provided the data-dependence is clarified through additional experiments.","major_comments":[{"comment":"The headline ICDAR15 result (4.6% improvement over FOTS MS in Table 1) is reported for models trained with 30k private web images and one million machine-labeled images (Section 3.6), while the public-data-only comparison is given only for Total-Text in Table 2. Without a comparable public-only ICDAR15 number, the reported gain cannot be separated from the effect of additional training data. Please report the 'Inc-Res public' configuration on ICDAR15 under the same end-to-end metrics used in Table 1.","section":"Section 4.1, Tables 1 and 3"},{"comment":"The partially labeled branch updates only the recognizer according to Eq. (5), yet the recognizer receives features produced by the RoI masking operation of Section 3.3, which requires per-instance segmentation masks. The machine labels are described as oriented rectangles in Section 3.6, and the paper does not specify how these rectangles are converted to masks for the RoI masking step. Please clarify whether masks are used for machine-labeled regions, and if so, how they are synthesized from oriented rectangles.","section":"Section 3.5 and Eq. (5)"},{"comment":"The only quality control described for teacher labels is a vague 'increase the confidence threshold,' with no threshold value, no measurement of teacher accuracy, and no analysis of error modes. Given that Table 3 attributes large gains to partial labels (e.g., Inc-Res E2E AP from 53.1 to 58.7 when adding PD), the reported improvements could be an artifact of the teacher OCR engine's label distribution rather than a property of the architecture. Please provide a label-noise analysis, a sensitivity study over confidence thresholds, and a comparison against recognizer training on human labels for a held-out subset.","section":"Section 3.5"},{"comment":"The paper collects 30k manually labeled web images and one million automatically labeled web images but does not check whether any of these images overlap with the ICDAR15 or Total-Text test sets. Even a small number of duplicated or near-duplicate test images could inflate the benchmark numbers. Please report an overlap check (e.g., exact or near-duplicate image search) or explain why leakage is implausible.","section":"Section 3.6"}],"minor_comments":[{"comment":"The phrase 'due to the short of fully annotated images' should be 'due to the shortage of fully annotated images.'","section":"Section 1"},{"comment":"The reference to 'the conv4 12 layer' should be written as 'the conv4_12 layer' (or 'the 12th convolutional block of stage conv4') for clarity.","section":"Section 3.1"},{"comment":"There is a missing space in 'label smoothing set to0.9'; it should read 'set to 0.9.'","section":"Section 3.5"},{"comment":"The citation '(Heet al.)' should read '(He et al.)' with a space after 'He.'","section":"Section 4.1"},{"comment":"The description of RoI masking says features are resized 'so that the shorter dimension is equal to 14 while maintaining the overall aspect ratio,' but the paper does not state the exact output tensor dimensions or how the mask is resampled; please give these details for reproducibility.","section":"Section 3.3"},{"comment":"The sentence 'In the bottom right image, prediction errors are shown in blue, some predictions are skipped for better visualization. All the skipped predictions are correctly predicted by our method' is confusing; please reword to explain why correct predictions are skipped.","section":"Figure 4 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper relies on a proprietary OCR engine (Google Cloud Vision API, cited as PhotoOCR [4], whose authors include two of the current authors) and on 30k manually labeled web images that are not released. This does not constitute grounds for rejection, but the editor should weigh the reproducibility and potential conflict-of-interest concerns when deciding whether to solicit a revised version. The authors should be asked to report public-data-only results on ICDAR15 and to provide the label-quality and overlap analyses described in the major comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Xi,\n\nYou should know two things. The architecture work is real: RoI masking is a simple, well-ablated idea that lets an attention decoder operate on unrectified arbitrary-shape features, and the public-only Total-Text model still beats prior art by a large margin. But the headline state-of-the-art margins on ICDAR15 and Total-Text are not cleanly attributable, because the full model is trained on 30k private web images plus one million images automatically labeled by the authors' own PhotoOCR engine, and ICDAR15 results for the public-only model are not reported. The stress-test note is right; this is an attribution and reproducibility problem, not an internal contradiction.\n\nWhat is genuinely new: the RoI masking step. Instead of rectifying text features to a horizontal strip, the model multiplies cropped features by the instance segmentation mask and lets attention follow the text path. The ablations in Table 3 support it consistently: +1.1 AP with ResNet-50 on the E2E baseline, +2.2 with Inc-Res, and +3.3 with ResNet-50 when partial data is used. The partially labeled data idea is also useful, and the paper honestly shows its effect. The public-only Inception-ResNet on Total-Text gets 63.9 E2E F-score versus 54.0 for TextNet and 52.9 for Mask TextSpotter; that is solid evidence the method itself works without the proprietary ingredients.\n\nSoft spots, in proportion. The missing public-only ICDAR15 number is the main one; Table 3 reports AP only for the full private pipeline. Without it, the 4.6% E2E F-score gain over FOTS MS is confounded by data scale and teacher quality. The mask handling for machine-labeled data is also under-specified: Section 3.5 says only low-confidence outputs are filtered, but the OCR engine provides oriented rectangles, not instance masks, and the paper does not say how masks are synthesized for the RoI masking step. That is a genuine gap for reproducibility. The label noise question is real but secondary, since the ablations show partial data helps even with whatever noise exists. None of this invalidates the core technique, and the paper is not hiding its data choices—they list everything—but the clean headline margin is not yet established.\n\nFor a reader: this is a solid industrial paper. The technique is worth knowing and the ablations are decent. It deserves a serious referee who should ask for the public-only ICDAR15 run and details on mask synthesis. I would take it, with requested revisions. I would not cite the SOTA numbers as-is, but the RoI masking idea is citable once the details are pinned down.","headline":"RoI masking is a genuine, well-ablated idea and the public-only Total-Text result stands on its own, but the headline SOTA margins are confounded by 1M proprietary OCR labels and missing public-only ICDAR15 numbers.","tokens_in":14918,"tokens_out":2397,"would_cite":false,"duration_ms":23546,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"End-to-end OCR reads curved and straight text in a single forward pass, without rectification.","keywords":["end-to-end text spotting","scene text recognition","curved text","arbitrary shape text","Mask R-CNN","attention decoder","instance segmentation","partially labeled data"],"falsifier":"A controlled experiment that trains the same architecture on the same fully labeled data with and without the machine-label branch, but replaces the teacher's transcriptions on a random subset of the million images with human corrections; if end-to-end performance on benchmarks does not drop when teacher labels are replaced by corrected labels, the partially labeled data effect is robust. Conversely, if performance collapses when the teacher is swapped for a different OCR engine, the results depend on the specific teacher rather than the architecture.","tokens_in":13962,"feed_emoji":"🔠","tokens_out":4699,"duration_ms":46433,"temperature":0.7,"pith_summary":"This paper claims that an end-to-end trainable network can detect and recognize text of arbitrary shape, including curved text, in a single forward pass. The key move is to remove the feature rectification step used by previous end-to-end models, letting an attention decoder read directly from masked instance features. The authors report that the model surpasses previous state-of-the-art end-to-end recognition by 4.6% on the straight-text ICDAR15 benchmark and by more than 16% on the curved-text Total-Text benchmark. They also show that automatically labeled data from an existing multi-step OCR engine can be used as partially labeled training data, improving both detection and recognition. If true, this makes end-to-end text spotting practical for irregular text and simplifies the pipeline compared to cascaded OCR systems.","feed_headline":"End-to-end OCR reads curvy text, beating prior best by 16%","feed_subtitle":"Masking text regions lets an attention decoder follow any shape, lifting Total-Text end-to-end F-score by 16.7%.","key_machinery":"The central mechanism is RoI masking: for each detected text region, features cropped from the image-level feature map are multiplied by the corresponding instance segmentation mask, suppressing neighboring text and background so the attention decoder sees only the target text instance. The recognizer is a Bahdanau-style seq2seq attention decoder that generates one character per step, with attention weights that shift along the text path. The training machinery also includes a partially labeled data branch: on machine-annotated images only the recognizer is trained, using outputs from an existing multi-step OCR engine as ground truth.","core_discovery":"The central claim is that feature rectification is not required for end-to-end text recognition: an attention decoder can learn to follow arbitrary text paths when it is given cropped features multiplied by the predicted instance segmentation mask. The paper demonstrates this by building a model that uses Mask R-CNN for detection (predicting boxes and masks) and a sequence-to-sequence attention decoder for recognition, trained jointly on fully labeled data plus large-scale partially labeled machine-annotated data. The reported results show large gains over prior end-to-end systems on both straight and curved benchmarks, with the curved-text margin being especially large.","pith_inferences":["RoI masking may generalize to other tasks that attach a recognition head to an instance segmentation detector, such as reading labels on objects or reading text on irregular surfaces.","The success of unrectified attention decoding suggests that prior end-to-end models were held back mainly by rectification; adding a rectification module to this model would likely hurt performance, which would further support the claim.","The partially labeled data approach could be turned into a self-improving loop, where the model's own predictions regenerate labels for new images, though teacher bias would need to be monitored.","The reported failure on upside-down text suggests that explicit orientation augmentation or a rotation-aware decoder might be needed to handle rare reading directions."],"forward_implications":["End-to-end text spotting architectures can be simplified by removing rectification modules while gaining the ability to read curved and arbitrarily shaped text.","Training the recognizer provides a learning signal that improves the shared feature extractor, which can boost detection performance even when detection is not directly trained on that branch.","Large-scale automatically labeled data from existing OCR engines can be used to train end-to-end models, reducing the need for fully annotated scene text images.","The same single-pass model works on both straight and curved text, suggesting a unified approach for diverse scene text reading applications.","Attention weight visualization gives a direct way to see and debug where the model is reading, which can help identify failure modes."],"supporting_citations":[{"why":"Provides the Mask R-CNN detector architecture that produces the bounding boxes and instance segmentation masks used by the model.","marker":"[20]"},{"why":"Supplies the Bahdanau-style attention mechanism used by the sequence-to-sequence recognizer.","marker":"[3]"},{"why":"Describes the existing multi-step OCR engine whose outputs are used as partially labeled training data.","marker":"[4]"},{"why":"Introduces the Total-Text dataset, the curved-text benchmark used for evaluation.","marker":"[11]"},{"why":"Defines the ICDAR15 benchmark and its end-to-end evaluation protocol for straight text.","marker":"[30]"},{"why":"The previous state-of-the-art end-to-end model based on Mask R-CNN, which the paper compares against on both benchmarks.","marker":"[42]"},{"why":"A strong prior end-to-end system for oriented straight text; the paper's results on ICDAR15 are compared to it.","marker":"[37]"},{"why":"A prior end-to-end network that handles irregular text with rectification; the paper compares against it on Total-Text.","marker":"[54]"}],"fun_headline_variants":["No rectification needed: attention reads curvy text, +16% on Total-Text","End-to-end spotting that follows any shape, no warping, +16% curved","Mask and attend: OCR that reads arbitrary shapes, beats prior by 16%","Skip rectification, use attention: curved text OCR jumps 16%","End-to-end text spotting: masks + attention, no rectification, +16%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported improvements from partially labeled data assume that the automatic OCR engine's outputs are accurate enough to serve as training ground truth for the recognizer, and that text regions labeled only with oriented rectangles, without instance masks, still provide a valid training signal through the RoI masking pipeline; if the teacher's errors are systematic and align with benchmark content, the gains could overstate the architecture's contribution.","fun_headline_variants_meta":{"raw":{"variants":["No rectification needed: attention reads curvy text, +16% on Total-Text","End-to-end spotting that follows any shape, no warping, +16% curved","Mask and attend: OCR that reads arbitrary shapes, beats prior by 16%","Skip rectification, use attention: curved text OCR jumps 16%","End-to-end text spotting: masks + attention, no rectification, +16%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000327,"raw_usage":{"total_tokens":1765,"prompt_tokens":816,"completion_tokens":949,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":841}},"tokens_in":432,"tokens_out":949,"duration_ms":7213,"temperature":1.0,"reasoning_tokens":841,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:18:03.343377+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled experiment that trains the same architecture on the same fully labeled data with and without the machine-label branch, but replaces the teacher's transcriptions on a random subset of the million images with human corrections; if end-to-end performance on benchmarks does not drop when teacher labels are replaced by corrected labels, the partially labeled data effect is robust. Conversely, if performance collapses when the teacher is swapped for a different OCR engine, the results depend on the specific teacher rather than the architecture.","supporting_citations":[{"cited_title":"Photoocr: Reading text in uncontrolled con- ditions","cited_arxiv_id":null,"evidence_quote":"Describes the existing multi-step OCR engine whose outputs are used as partially labeled training data."},{"cited_title":"Total-text: A com- prehensive dataset for scene text detection and recognition","cited_arxiv_id":null,"evidence_quote":"Introduces the Total-Text dataset, the curved-text benchmark used for evaluation."},{"cited_title":"Icdar 2015 competition on robust reading","cited_arxiv_id":null,"evidence_quote":"Defines the ICDAR15 benchmark and its end-to-end evaluation protocol for straight text."},{"cited_title":"Mask textspotter: An end-to-end trainable neu- ral network for spotting text with arbitrary shapes","cited_arxiv_id":null,"evidence_quote":"The previous state-of-the-art end-to-end model based on Mask R-CNN, which the paper compares against on both benchmarks."},{"cited_title":"Fots: Fast oriented text spotting with a uni- fied network","cited_arxiv_id":null,"evidence_quote":"A strong prior end-to-end system for oriented straight text; the paper's results on ICDAR15 are compared to it."}],"review_version":1}