{"id":"50ed54da-139b-4084-a6d1-a81945d384e0","arxiv_id":"2411.10261","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A scene text retrieval system that embeds queries and detected text lines into one space and uses dynamic programming to rank and localize partial text matches.","lead":"This paper introduces a method for retrieving full text lines and partial text patches from scene images using a query word. The method learns a shared embedding for words and image text, then uses a dynamic programming search to locate partial matches, including non-contiguous ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The RankMIL loss in Eq. 7 is justified by an untested 'always contains a more similar patch' premise, and BCA's own admitted equal-spatial-character assumption can make that premise fail for variable-width or curved text.","rationale":"The reader's conditional verdict already rests on the same load-bearing assumption: RankMIL's training signal depends on the claim that every positive bag contains a patch more similar to the query than the full text-line, while BCA's equal-spatial-character assumption is admitted to be weak. I agree that this is the most important point to test, because it is the structural novelty of the paper's partial-patch training. The paper deserves credit for its consistent ablations, the DPMA speed/accuracy comparison, and the released code and dataset, which make the proposed test feasible. A secondary concern is that the margin hyperparameter m is tuned on the ReCTS evaluation dataset (Section 4.3.3) and then used for all reported results, which could inflate the headline numbers, but this is a methodology issue rather than a potential break in the training argument. The proposed synthetic test directly measures whether the RankMIL premise holds and whether the Δs>0 filter is responsible for the reported gains, so it would settle whether the concern actually lands. Because the reader already marked the verdict conditional, my read does not change it.","tokens_in":21867,"tokens_out":9841,"duration_ms":110508,"concrete_test":"Run the released code on a synthetic split with per-character ground-truth boxes (e.g., SynthText-like renderings with both uniform-width and variable-width fonts). For every positive training bag, record (i) whether BCA's windows contain the true character span of the query and (ii) whether the trained model satisfies Δs>0 for the bag's selected patch. Then retrain a variant restricted to bags satisfying (i) and another variant that optimizes all positive bags regardless of Δs, comparing PPR mAP on the real benchmarks. If the Δs>0 filter is essential to reproduce the reported results, the equal-width premise is load-bearing; if verified bags alone give the same results, the premise is not the bottleneck.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim--simultaneous text-line and partial-patch retrieval without extra annotations--requires the RankMIL loss (Eq. 7) to supply a sound training signal. That loss fires only when the BCA-generated bag label contains the query and when Δs = sim_f(f_q, f_pθ) − sim_f(f_q, f_pl) > 0; otherwise the sample is discarded. Section 3.4 justifies this with the assertion that the bag 'always contains a patch that is more similar' to the query than the full text-line. But the bag labels are generated by BCA (Section 3.3.1) under the equal-spatial-character assumption that Section 3.3.2 itself calls 'relatively strong and hardly holds' for arbitrary-shaped text. With variable-width characters (e.g., 'W' versus 'i') or curved text, the equal-width window can miss the true character span, so the selected patch can be a false positive; when its similarity still exceeds the line score, Eq. 7 will reinforce that false association rather than filter it. When its similarity does not exceed the line score, Eq. 7 abandons the sample, removing positive supervision from exactly the hard cases where the reported gains are largest. No experiment reports how often the Δs>0 condition actually holds on positive bags, and the Table 4 ablation cannot separate RankMIL's contribution from DPMA's inference-time search. The empirical gains are consistent with the claim, but the 'always' premise that makes RankMIL principled is unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces partial scene text retrieval (PPR), a task in which a model must retrieve both full text-line instances and continuous/non-continuous partial patches of scene text, given a query string. The proposed PSTR framework trains a shared cross-modal embedding with a text-line retrieval loss (CMSL) and a ranking-based multiple instance learning loss (RankMIL), where pseudo bag labels are generated by a sliding-window Bag Constructing Algorithm (BCA) over ground-truth text-line annotations. At inference, a Dynamic Partial Match Algorithm (DPMA) searches for an optimal subsequence of local features from each detected text-line proposal, avoiding explicit bag construction. The authors report large mAP improvements over prior retrieval and spotting baselines on English (STR, CTR, ArT) and Chinese (CSVTRv2, ReCTS, LSVT) benchmarks, and release code and datasets.","tokens_in":22194,"tokens_out":5948,"duration_ms":62074,"significance":"If the results are reproducible, the paper makes a useful contribution: it is the first to formalize partial scene text retrieval, and its DPMA provides a practical way to search partial patches without dense enumeration. The method works across Latin and non-Latin scripts, and the released code and CSVTRv2 dataset are valuable assets. The ablations are broadly consistent, and the gains over strong spotters such as SwinTextSpotter are substantial. However, the central training mechanism, RankMIL, rests on an unverified premise about bag contents, and several experimental choices (hyperparameter selection, lack of error bars) currently weaken the confidence in the numerical claims.","major_comments":[{"comment":"The RankMIL loss is justified by the assertion in Section 3.4 that a positive bag 'always contains a patch that is more similar to the given query text than the text-line instance.' This premise is not established and is in tension with the paper's own statement in Section 3.3.2 that BCA's equal-spatial-character assumption 'is relatively strong and hardly holds,' especially for non-horizontal text. For variable-width characters (e.g., 'W' versus 'i') or curved text, the BCA window can miss the true character span, so the selected patch may be a false positive. When such a patch still has similarity exceeding the line score, Eq. (7) actively reinforces a wrong association; when its similarity does not exceed the line score, the sample is discarded, removing positive supervision exactly on the hard cases where the reported gains are largest. The paper does not report how often the condition Delta_s > 0 actually holds on positive bags, so the training signal's reliability is unknown. I request a direct measurement of this condition before and during training on each dataset (with per-dataset statistics), an analysis of failure cases due to BCA misalignment, and an ablation that shows RankMIL's gain is not an artifact of discarding hard positives. In addition, since BCA labels only contain contiguous substrings, the authors should explain how non-continuous partial patches receive any RankMIL supervision; Table 6 shows large NCPP gains from DPMA, but the training mechanism for NCPP is unspecified.","section":"Section 3.4, Eq. (7)"},{"comment":"The hyperparameters K (number of boundary point pairs) and m (RankMIL margin) are selected using the same test datasets whose final numbers are reported: K on ArT and LSVT, m on ReCTS. With no validation split, this selection procedure can inflate mAP, particularly for the margin m, which directly controls the RankMIL loss and on which the reported improvements depend. The authors should either reselect these hyperparameters on held-out validation portions (with separate test sets reported) or present sensitivity curves over a grid with standard errors, clearly stating which split was used.","section":"Section 4.3.2 and 4.3.3, Tables 3 and Fig. 7"},{"comment":"All mAP numbers appear to come from a single training run. The central claims, such as the 8.04% and 12.71% average improvements over state-of-the-art methods, would be much more convincing with statistics: at least three independent runs with reported mean and standard deviation, and ideally statistical significance tests for the main comparisons. Without these, it is difficult to judge whether the observed gaps are stable or partly due to stochastic variation, especially because several improvements are in the 2-5% mAP range where run-to-run variance can matter.","section":"Tables 4, 5, 7 and 8"},{"comment":"The PPR benchmark is self-constructed: query words are generated by cutting ground-truth substrings, and evaluation is image-level via mAP as in [6]. This is a reasonable design, but the paper does not state how ground-truth positive partial patches are defined for scoring (e.g., whether any detected patch that contains the query substring is considered positive, or whether patch-level overlap is required). The absence of a patch-level metric makes it hard to interpret the PPR mAP numbers, especially for non-continuous patches, and leaves room for the model to rank an image correctly for reasons other than accurate partial-patch localization. Please specify the evaluation protocol precisely and, if feasible, provide a patch-level or localization-based metric to complement image-level mAP.","section":"Section 4.1 and Section 4.4"}],"minor_comments":[{"comment":"The loop condition 'for i = 1; i < |t| - n; i++' appears to exclude the last valid subsequence; it should likely be 'i <= |t| - n' (or equivalently 'i < |t| - n + 1') to cover all substrings of length n.","section":"Algorithm 1"},{"comment":"The sentence 'These MIL-based methods are designed for general objects' is repeated verbatim twice in immediate succession; one copy should be removed.","section":"Section 2.2"},{"comment":"The text says 'the parameters T and C are empirically set to 15 and 128'; these are hyperparameters, not learned parameters, and the wording should be corrected.","section":"Section 3.1.1"},{"comment":"The DPMA transition allows a path to move 'one step to the right and at least zero steps down,' but the recurrence with max over previous rows could be misread as permitting zero horizontal progress; please clarify the exact admissible moves and whether the path must touch every query column exactly once.","section":"Section 3.5, Eq. (8)"},{"comment":"The citation for ABCNet v2 is inconsistent: the text refers to 'ABCNet v2 [57]', but reference [57] is Mask TextSpotter v3 while ABCNet v2 is [55]. Please correct the citation.","section":"Section 4.4.2, Table 8"},{"comment":"The caption of Fig. 7 does not state the dataset and split used for the margin sweep; the text says experiments are on ReCTS, but it should be explicit whether this is the test set or a validation split.","section":"Figure 7"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern raised by the reader is valid and is the main reason for my recommendation. The RankMIL training signal is the load-bearing component for the paper's central claim, and its 'always contains a more similar patch' premise is not verified and conflicts with the paper's own admission that the BCA assumption is strong. That said, the issue is addressable: the authors can measure the Delta_s > 0 condition on positive bags, add ablations with and without the abandonment rule, and clarify the NCPP training mechanism. The hyperparameter selection and lack of error bars are also fixable within a revision. I do not see a fundamental flaw that would require rejection, provided these points are resolved empirically."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe headline: this is a solid extension that defines partial scene text retrieval and backs it with a genuinely useful inference algorithm. The reported gains are large enough to take seriously, but the RankMIL training loss leans on an untested premise that the paper itself half-concedes.\n\nWhat is new: the PPR task (continuous and non-continuous partial patches), RankMIL, DPMA, and the CSVTRv2 Chinese benchmark. The DPMA is the most interesting piece—it replaces exhaustive patch enumeration with a dynamic program that can stitch non-adjacent local features, and Table 6 shows it roughly doubles NCPP mAP compared to bag construction. The ablations are consistent across six datasets, and the links to the conference version are honest. Code and data release is a plus, though without a commit hash.\n\nThe soft spot is the RankMIL justification. Section 3.4 asserts that a positive bag 'always contains a patch that is more similar to the given query text than the text-line instance.' That only holds if BCA's equal-spatial-character windows actually sample the true character span. Section 3.3.2 says that assumption 'is relatively strong and hardly holds' for arbitrary shapes. If windows miss the span, the selected patch can be a false positive; the loss then either reinforces a wrong association or, when Δs ≤ 0, discards the sample and removes supervision from exactly the hard cases. The paper reports no statistics on how often Δs > 0 on positive bags, and Table 4 cannot separate RankMIL's contribution from DPMA's because every PPR row uses DPMA. Also, margin m is selected on ReCTS and K on ArT/LSVT, and there are no error bars. These are fixable, not fatal.\n\nWho this is for: people working on scene text retrieval, weakly supervised sequence matching, and string-to-image embedding. The central claim—a single model retrieves full lines and partial patches without patch annotations—is plausible and the evidence is enough to warrant a serious referee.\n\nMy recommendation: send it to peer review. Ask the reviewers to (a) report the Δs>0 rate on positive bags across datasets, (b) compare RankMIL vs MIL with a shared, non-DPMA inference, and (c) add error bars or at least multiple seeds.","headline":"A solid extension that defines partial scene text retrieval and offers a genuinely useful inference-time search, but the RankMIL training premise is under-verified and the benchmark is self-constructed.","tokens_in":22731,"tokens_out":2419,"would_cite":true,"duration_ms":22740,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that one end-to-end network can retrieve both full text lines and their partial patches from scene images without patch-level annotations, using a ranking MIL training objective and a dynamic-programming partial match…","keywords":["scene text retrieval","partial patch retrieval","cross-modal similarity learning","multiple instance learning","RankMIL","dynamic programming","DPMA","arbitrary-shaped text"],"falsifier":"A direct test would be to synthesize text lines with known unequal character widths (e.g., a wide “W” next to a narrow “i”) and query snippets whose ground-truth patch is known; if the bag built by BCA misses the correct patch or RankMIL's ranking premise fails, the model's PPR mAP on such synthetic lines should drop sharply. Concretely, compare RankMIL against a version trained with oracle patch labels on those same lines: if the gap is large for uneven-width Latin text but small for uniform-width Chinese text, the equal-spatial-character assumption is the culprit. A second check would be to inspect DPMA's optimal path on lines with variable widths and see whether the selected local features align with the true characters of the query.","tokens_in":21670,"feed_emoji":"🔍","tokens_out":7096,"duration_ms":60816,"temperature":0.7,"pith_summary":"The paper is trying to establish that partial scene text retrieval—searching for a snippet of text inside a photographed text line rather than only the whole line—can be solved without any patch-level annotations. It argues that a network trained only on line-level text labels can learn to rank full text lines and localize both contiguous and non-contiguous partial patches in one shared embedding space. If true, it makes text search in images more useful: a query like “BAR” can find the word “BARBEQUE” in a storefront photo and pinpoint the part of the line that matches. The authors report large mAP gains over existing retrieval and spotting methods, especially for the partial-patch task and for Chinese text.","feed_headline":"Partial text search in photos, no extra labels needed","feed_subtitle":"A single network finds whole text lines, continuous snippets, and non-continuous snippets in English and Chinese images.","key_machinery":"The central machinery is a training/inference pair rather than a single identity. RankMIL is a ranking multiple-instance-learning loss, $loss(f_{q_j}, f_{p^\\theta_r}) = \\max(0, -I(t_{q_j} \\in T^r(t_{p_i^l}) \\land \\Delta s >0)(\\Delta s - m))$ with $\\Delta s = sim_f(f_{q_j}, f_{p^\\theta_r}) - sim_f(f_{q_j}, f_{p_i^l})$, which filters noisy bag patches by requiring the best patch to beat the whole-line similarity by margin $m$. DPMA is a dynamic-programming search over a $T \\times T$ grid of local cosine similarities, with state transition $S_{x,y} = \\max(\\{S_{k,y-1}\\}_{k=0}^{x}) + s_{x,y}$ (with boundary cases), which selects an ordered subset of the line's local features and thereby outputs a partial-patch feature without enumerating bags. The word-embedding module (characters to a fixed-length sequence via bilinear interpolation and a bidirectional LSTM) and boundary-point text proposals supply the sequential features that make both mechanisms work.","core_discovery":"On its own terms, the central discovery is that a text-line-level retrieval model can be extended to partial patches by replacing the standard MIL binary classification with a ranking objective (RankMIL) and by replacing bag construction at inference with a dynamic-programming search (DPMA) over the line's sequential features. The model embeds query strings and detected text-line proposals—represented by boundary points—into a common feature space, with similarity supervised by normalized edit distance. For training, each text line is cut into bags of partial patches with pseudo labels; RankMIL abandons noisy patches whose similarity to the query is below the line-level similarity, and pushes the best patch above it by a margin. At inference, DPMA walks monotonically through a $T \\times T$ grid of local similarities between query and line sequence features to select a subset of local features—spatially adjacent or not—that maximizes cumulative similarity, yielding the partial-patch feature directly. The paper reports that this outperforms state-of-the-art approaches by 8.04 and 12.71 mAP on average across three English datasets for line retrieval and patch retrieval, and by 24.45 and 38.06 mAP on Chinese datasets.","pith_inferences":["If the equal-spatial-character assumption is the real bottleneck, a version of BCA that segments by estimated per-character widths (using attention or character recognition) should close most of the English TIR–PPR gap; this is directly testable with the released code.","RankMIL's ranking premise—best patch beats whole line—is a weak-supervision signal that could transfer to other sequence-like objects, such as audio snippets, time series, or video sub-clips, where instance-level labels are missing but sequence-level containment labels exist.","DPMA resembles sequence alignment; coupling it with a learned character-width prior might produce a differentiable soft-alignment that makes training and inference consistent, reducing the train/inference mismatch between RankMIL bags and DPMA search.","The larger reported gains on Chinese text suggest the method's advantage grows where character widths are uniform; extending it to cursive or proportional scripts may require explicit width modeling."],"forward_implications":["A single model can rank and localize text lines and their partial patches simultaneously, so image galleries can be searched at the sub-line level without any patch annotations.","Because DPMA does not require spatially adjacent features, queries can match non-continuous partial patches—characters separated by other text—which bag-based methods cannot retrieve.","Replacing bag construction with DPMA removes the need to enumerate more than 50 patches per line, improving inference speed while raising PPR accuracy (for example, from 70.37 to 75.74 mAP on CSVTRv2).","Learning cross-modal similarity directly generalizes to non-Latin scripts better than PHOC or end-to-end recognition, with the largest reported gains on Chinese datasets.","The remaining English TIR–PPR gap is attributed by the authors to variable character widths, making proportional fonts the harder case for the equal-width assumption."],"supporting_citations":[{"why":"Defines the single-shot scene text retrieval baseline and the mAP metric used throughout the comparisons.","marker":"[6]"},{"why":"Supplies the bag-based weak supervision idea that RankMIL adapts to text patches.","marker":"[12]"},{"why":"Provides the joint detection and cross-modal similarity learning framework that this paper extends to partial patches.","marker":"[13]"},{"why":"Defines the Levenshtein edit distance used as text-level similarity supervision in Eq. (4).","marker":"[18]"},{"why":"Supplies the boundary-point text proposal representation and ArbitraryRoIAlign used to extract features from arbitrary-shaped text.","marker":"[39]"},{"why":"Provides the FCOS anchor-free detector structure underlying the text-line proposal module.","marker":"[44]"},{"why":"Serves as a state-of-the-art spotting baseline whose officially released model is used for comparisons.","marker":"[57]"},{"why":"Serves as the strongest previous spotting baseline on English partial-patch retrieval, against which the reported PPR gain is measured.","marker":"[58]"}],"fun_headline_variants":["Partial text retrieval without extra labels via RankMIL","Search text snippets in images with RankMIL and DPMA","One network finds text lines and partial patches","Partial scene text retrieval: lines and snippets, no labels","RankMIL adaptively filters noisy patches for partial text retrieval"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that whenever a query string appears in a text line, the bag built from that line always contains some patch whose feature is more similar to the query than the whole line's feature is; this relies on the Bag Constructing Algorithm's assumption that characters occupy equal spatial widths, which the paper admits is “relatively strong and hardly holds” for non-horizontal text.","fun_headline_variants_meta":{"raw":{"variants":["Partial text retrieval without extra labels via RankMIL","Search text snippets in images with RankMIL and DPMA","One network finds text lines and partial patches","Partial scene text retrieval: lines and snippets, no labels","RankMIL adaptively filters noisy patches for partial text retrieval"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000386,"raw_usage":{"total_tokens":2079,"prompt_tokens":1023,"completion_tokens":1056,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":977}},"tokens_in":639,"tokens_out":1056,"duration_ms":9671,"temperature":1.0,"reasoning_tokens":977,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:48:13.829767+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would be to synthesize text lines with known unequal character widths (e.g., a wide “W” next to a narrow “i”) and query snippets whose ground-truth patch is known; if the bag built by BCA misses the correct patch or RankMIL's ranking premise fails, the model's PPR mAP on such synthetic lines should drop sharply. Concretely, compare RankMIL against a version trained with oracle patch labels on those same lines: if the gap is large for uneven-width Latin text but small for uniform-width Chinese text, the equal-spatial-character assumption is the culprit. A second check would be to inspect DPMA's optimal path on lines with variable widths and see whether the selected local features align with the true characters of the query.","supporting_citations":[{"cited_title":"Single shot scene text retrieval,","cited_arxiv_id":null,"evidence_quote":"Defines the single-shot scene text retrieval baseline and the mAP metric used throughout the comparisons."},{"cited_title":"Multiple instance boosting for object detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the bag-based weak supervision idea that RankMIL adapts to text patches."},{"cited_title":"Scene text retrieval via joint text detection and similarity learning,","cited_arxiv_id":null,"evidence_quote":"Provides the joint detection and cross-modal similarity learning framework that this paper extends to partial patches."},{"cited_title":"Binary codes capable of correcting deletions, insertions, and reversals,","cited_arxiv_id":null,"evidence_quote":"Defines the Levenshtein edit distance used as text-level similarity supervision in Eq. (4)."},{"cited_title":"All you need is boundary: Toward arbitrary-shaped text spotting,","cited_arxiv_id":null,"evidence_quote":"Supplies the boundary-point text proposal representation and ArbitraryRoIAlign used to extract features from arbitrary-shaped text."},{"cited_title":"Fcos: Fully convolutional one- stage object detection,","cited_arxiv_id":null,"evidence_quote":"Provides the FCOS anchor-free detector structure underlying the text-line proposal module."},{"cited_title":"Mask textspotter v3: Segmentation proposal network for robust scene text spotting,","cited_arxiv_id":null,"evidence_quote":"Serves as a state-of-the-art spotting baseline whose officially released model is used for comparisons."},{"cited_title":"Swintextspotter: Scene text spotting via better synergy between text detection and text recognition,","cited_arxiv_id":null,"evidence_quote":"Serves as the strongest previous spotting baseline on English partial-patch retrieval, against which the reported PPR gain is measured."}],"review_version":1}