{"id":"4a360fc7-76c3-4fc4-8aee-7cde56524d47","arxiv_id":"2412.19917","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Char-SAM automatically converts word-level text annotations into character-level segmentation masks by prompting SAM with refined character boxes and glyph-derived point prompts.","lead":"The authors built a pipeline that turns word-level text boxes into pixel-level masks of individual letters by prompting SAM with character boxes and letter-shape templates. It can auto-generate training data for scene text segmentation from existing detection datasets without training a new model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CGR's fixed-glyph assumption is load-bearing: the claimed MLT17/downstream quality rests on canonical English templates that do not hold for variable glyphs or non-Latin scripts, yet no per-script or per-character evidence is given.","rationale":"The reader's verdict is CONDITIONAL, and my stress-test does not change it; it sharpens the condition. The strongest_claim has two legs: TextSeg accuracy and downstream/MLT17 dataset quality. The TextSeg leg is plausible but confounded by the use of word boxes/transcriptions; the downstream leg depends on CGR working on MLT17. The single most load-bearing condition is the stated glyph fixedness: if it fails for variable glyphs or non-Latin scripts, the MLT S refined dataset is not high-quality and the +0.65 F-score downstream gain may not generalize. The concrete stratified test would settle this by measuring per-character CGR deltas. I give credit for the ablation study (Table II) and the downstream experiment (Table IV); they are real evidence but aggregate only. The absence of code and data makes independent verification impossible, which supports keeping CONDITIONAL rather than ACCEPT. My concern does not move the verdict; it identifies the precise evidence that should be supplied.","tokens_in":7769,"tokens_out":10905,"duration_ms":119435,"concrete_test":"Run a per-character ablation on TextSeg and MLT S using the existing word boxes/transcriptions: for each GT character instance, compute SAM's fgIoU/F-score with Pchar only versus Pchar+Ppos+Pneg, then stratify the delta by character identity and by script. Specifically, (1) for all instances of 'a' and 'g' in TextSeg, report the median CGR delta (with CGR minus without CGR); if the median is <= 0 for either character, the fixed-glyph assumption fails for variable glyphs. (2) For MLT S, report the CGR delta separately for Latin and non-Latin scripts; if the non-Latin delta is not positive, the MLT17 refined dataset claim is unsupported and should be restricted to Latin text.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing point is in Sec. II-C: 'the English character, regardless of font or text direction, follows the fixed glyph structure determined by its category.' This typographic premise justifies placing glyph-derived positive points and hole-derived negative points into every character box. It is false for common Latin letters with multiple frequent forms (single-storey vs double-storey 'a'; single- vs double-loop 'g'), and it is not defined for non-Latin scripts that dominate MLT17 (Arabic, Chinese, Japanese, Korean, Bangla). Because Table II attributes +3.0 fgIoU / +4.2 F-score to CGR, the claimed automatic high-quality annotation, including the MLT S refined dataset and the +0.65 F-score downstream gain in Table IV, rests on these templates being valid for the actual characters. The paper reports only aggregate gains; it gives no per-character, per-font, or per-script breakdown, and no fallback for non-Latin characters. If CGR's usefulness is concentrated in canonical Latin uppercase instances, the MLT17 contribution and the claimed generality are overstated. The concern is a missing-evidence and assumption-validity issue, not a disagreement with community consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Char-SAM, a training-free pipeline that turns word-level scene text annotations (bounding boxes and transcriptions) into character-level segmentation masks by prompting SAM with character boxes produced by the CBR module and with glyph-derived positive/negative point prompts produced by the CGR module. On TextSeg the method reports 84.80 fgIoU and 92.15 F-score, close to supervised SOTA, and the dataset-swap experiment (Table IV) indicates that training on Char-SAM outputs outperforms training on COCO TS / MLT S pseudo-labels and slightly exceeds the TextSeg-trained baseline.","tokens_in":7985,"tokens_out":6153,"duration_ms":59272,"significance":"If the results hold, Char-SAM would be a useful low-cost annotation tool for scene text segmentation: it is training-free, exploits existing detection datasets, and the Table IV dataset-swap experiment directly tests the usefulness of the generated annotations. The paper's strengths are its clear ablations of CBR/CGR and prompt granularity (Tables II and III) and the downstream training comparison. The main weaknesses are the unverified fixed-glyph assumption behind CGR, the protocol mismatch in the SOTA comparison of Table I, and the absence of variance estimates for the small reported differences.","major_comments":[{"comment":"The CGR module's central premise, stated as 'the English character, regardless of font or text direction, follows the fixed glyph structure determined by its category,' is not valid for common Latin letterforms (single-storey versus double-storey 'a', single-loop versus double-loop 'g') and is undefined for the Arabic, Chinese, Japanese, Korean, and Bangla scripts that appear in MLT17. Table II attributes +3.0 fgIoU and +4.2 F-score to CGR on TextSeg, and Table IV's downstream gain uses MLT S refined, so the claimed dataset-quality improvement rests on the templates being valid for the actual characters. The paper provides only aggregate gains; please supply a per-character, per-font, or per-script breakdown, and either provide a fallback for non-Latin and variable-glyph characters or explicitly narrow the scope of the MLT17 claim.","section":"II-C, Table II"},{"comment":"The comparison in Table I is not a head-to-head zero-shot benchmark. Char-SAM is given the dataset's word-level bounding boxes and transcriptions as input (Section II-A), while the supervised methods are evaluated image-only after being trained on TextSeg masks; the reported 'zero-shot' result is therefore an upper-bound annotation-quality estimate under privileged word-level annotations. Please report Char-SAM with automatically detected boxes and recognized transcriptions, or clearly frame Table I as an annotation-pipeline evaluation rather than a zero-shot segmentation comparison.","section":"Table I"},{"comment":"The paper reports single runs without error bars or significance tests. This matters most in Table IV, where the claimed benefit over the TextSeg baseline is +0.13 fgIoU and +0.65 F-score, and in Table I, where Char-SAM is within about one point of SOTA; these differences could be within run-to-run variation. Please report multiple seeds (or at least variance) and significance tests, or otherwise justify that the differences exceed noise.","section":"Tables II-IV"}],"minor_comments":[{"comment":"The CBR module description is underspecified: the recognizer used, the 'elaborated bipartite matching' between character boxes and categories, and the watershed initialization are not described in enough detail for reproduction; please add pseudocode or precise references.","section":"II-B"},{"comment":"The CGR vote threshold (0.6) and the number of collected fonts (80) are introduced without justification; Table II varies only the template number (40 vs. 80), so please add a sensitivity analysis for the threshold or discuss how these choices were selected.","section":"III-B"},{"comment":"There is a small numerical inconsistency: Table II row '#2 (TN=40)' reports 84.75 fgIoU / 92.01 F-score, while Table III row 'Pchar + Ppos + Pneg' reports 84.68 / 91.91, even though the two rows appear to describe the same prompting configuration; please clarify the difference or reconcile the numbers.","section":"Tables II and III"},{"comment":"The visualization in Figure 3 is qualitative; please add per-dataset annotation-quality statistics for COCO TS refined and MLT S refined (e.g., agreement with a manually labeled subset), since Table IV measures downstream performance only.","section":"III-E"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of a computer vision venue and the core idea has value, but the current version overclaims multilingual generality on MLT17 given the fixed-glyph templates. I would be comfortable with acceptance only after the authors either supply per-script and per-character evidence for CGR or narrow the claim, and after the Table I protocol and the variance issue are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nQuick read on Char-SAM: it's a useful engineering paper that combines existing pieces into a plausible annotation pipeline, and the CGR module—using glyph-template votes to place positive and negative SAM point prompts—is the one genuinely new move. On TextSeg the ablations show the modules do real work: CBR adds about 3.5 fgIoU over the word-box baseline, and CGR adds another ~3 fgIoU and ~4 F-score. The dataset-swap experiment (Table IV) also supports the claim that these annotations train better than the existing semi-supervised pseudo-masks. So the core pipeline is a reasonable contribution.\n\nTwo soft spots, one serious, one routine. The serious one is exactly what the stress-test note flags: CGR's premise, stated in Sec. II-C, is that English characters have a fixed glyph structure regardless of font. That's already false in the large (single- vs double-storey 'a'), though a vote over 80 fonts and a 0.6 threshold may tolerate that. But the paper exports CGR to MLT17, which contains Arabic, Chinese, Korean, Bangla, etc., with no explanation of how glyph templates are generated for those scripts, no per-script breakdown, and no fallback. Because Table II attributes a sizable chunk of the gain to CGR, the MLT_S refined dataset quality and the downstream gain in Table IV are overclaimed without that evidence. This is a missing-evidence problem, not a contradiction, but it needs fixing.\n\nThe routine soft spot is Table I. Char-SAM receives word boxes and transcriptions; the supervised baselines only get the image. So 'within 1% of SOTA F-score' is a headline that will mislead. The apples-to-apples claim is the Table IV comparison, which is the one that matters for the annotation use case.\n\nI'd also want error bars or at least multiple runs on the small-number ablations, and the recognizer/matching details are underspecified enough that reproduction would be painful. No code or data release as of v1.\n\nWho gets value: people building annotation tools for text editing/erasing, and anyone working on SAM prompting with glyph priors. It deserves a serious referee. I'd send it to review with a request for per-script analysis, a fairer comparison, and details on the recognizer and matching.","headline":"A clever training-free SAM prompting pipeline for scene text segmentation, with a real glyph-prompt idea; but the headline comparison to supervised SOTA is unfair and the multilingual claims rest on an unexamined English-only glyph assumption.","tokens_in":8534,"tokens_out":3476,"would_cite":true,"duration_ms":31776,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Char-SAM shows that a training-free prompt pipeline can turn SAM into a high-quality scene text segmentation annotator, reaching 92.15 F-score on TextSeg.","keywords":["scene text segmentation","Segment Anything Model","visual prompting","character-level bounding box","glyph template","annotation generation","zero-shot segmentation","TextSeg"],"falsifier":"Run Char-SAM on a test set of curved or artistic Latin text and non-Latin scripts such as Chinese, Japanese, or Arabic with manual character-level masks, and compare per-script F-score; a large drop on scripts without glyph templates, or on stylized letters whose template votes place wrong prompts, would falsify the fixed-glyph assumption.","tokens_in":7563,"feed_emoji":"🔤","tokens_out":6906,"duration_ms":64002,"temperature":0.7,"pith_summary":"The paper's central claim is that SAM can be turned into a scene text segmentation annotator without any training, provided it is prompted at the right granularity. The authors propose Char-SAM, a pipeline that takes existing word-level bounding boxes and transcriptions, refines them into character-level boxes (CBR), and adds glyph-derived positive and negative point prompts (CGR) to cure SAM's over- and under-segmentation on text. On the TextSeg benchmark, the training-free pipeline reaches 84.80 fgIoU and 92.15 F-score, about one point below the supervised state of the art, and it improves a downstream segmentation model's F-score by 0.65 when its generated annotations are added to training. The motivation is to make pixel-level text annotation cheap enough to generate new datasets from existing detection-only datasets such as COCO-Text and MLT17.","feed_headline":"Training-free pipeline scores 92.15 on scene text masks","feed_subtitle":"Character-level boxes and glyph-derived points turn word-level text annotations into pixel-level masks.","key_machinery":"The machinery has two modules. The Character Bounding-box Refinement (CBR) module uses the CRAFT character-aware detector to turn a word-level box into candidate character boxes, then applies a text recognizer and a watershed split on the SAM logical map to separate merged adjacent characters, finally matching character boxes to the characters in the transcription. The Character Glyph Refinement (CGR) module renders each character category across many fonts, thresholds the rendered pixels into foreground and hole pixels, and lets each glyph template vote; pixels whose vote rate exceeds 0.6 become positive point prompts, hole pixels become negative point prompts, and these are fed to SAM together with the character box. The bbox-to-mask capability of SAM is the engine, while the two modules supply the finer character-level visual prompts that make that engine work on text.","core_discovery":"Char-SAM's central discovery is that SAM's failure on scene text is not a segmentation-capability problem but a prompt-granularity problem. When prompted with a word-level box, SAM treats the whole word as one object; when prompted with a character-level box alone, it over-segments holes (the inside of 'D' or 'A') and under-segments large characters. The paper shows that adding a CBR module to split word boxes into character boxes, and a CGR module that votes over font-rendered glyph templates to place positive points on character strokes and negative points on holes, resolves both failure modes. The resulting pipeline, still using the off-the-shelf SAM-B model, achieves 84.80 fgIoU and 92.15 F-score on TextSeg, comparable to supervised methods evaluated with multi-scale inference. The authors further show that the masks it generates on COCO-Text and MLT17 are cleaner than the weakly supervised COCO TS and MLT S annotations, and that training on these generated masks improves a downstream TextSeg-trained model.","pith_inferences":["The paper leaves implicit that CGR's fixed-glyph premise is script-dependent: for Chinese, Japanese, Arabic, or heavily stylized Latin logos, the per-character template votes would need a script-specific template library, and the paper reports no coverage analysis for the multilingual MLT17 images.","A natural transfer would be to any category with a stable canonical structure (digits, icons, anatomical parts), reusing the same refine-and-vote prompt recipe to fix SAM's over- and under-segmentation on small or hollow objects.","Because the downstream training gain comes from replacing weak pseudo-labels with Char-SAM masks, one can imagine a bootstrapping loop—train a text segmentation model on the generated masks, refine the masks with the model, and retrain—which the paper does not explore."],"forward_implications":["Existing detection datasets with only word-level boxes and transcriptions can be upgraded to pixel-level text masks automatically, at zero training cost.","The generated annotations on COCO-Text and MLT17 are of higher quality than previous weakly supervised COCO TS and MLT S masks, so downstream segmentation models trained with them gain +0.65 F-score over the TextSeg baseline.","Adding more glyph templates improves the result (84.75 to 84.80 fgIoU and 92.01 to 92.15 F-score when going from 40 to 80 templates), so the font-template library is a tunable resource.","The per-prompt ablation shows each added prompt type—character box, positive points, negative points—contributes a measurable gain, confirming that prompt granularity is the operative factor in turning SAM into a text annotator."],"supporting_citations":[{"why":"Supplies the promptable SAM model whose bbox-to-mask capability is the engine of the Char-SAM pipeline.","marker":"[24]"},{"why":"Provides the CRAFT character-aware detector used by CBR to split word-level boxes into character-level boxes.","marker":"[32]"},{"why":"Provides the TextSeg benchmark with character-level masks used for evaluation and for the downstream training experiments.","marker":"[1]"},{"why":"Supplies SynthText training data for CRAFT; the domain gap between synthetic and real images motivates the recognition and watershed correction steps in CBR.","marker":"[33]"},{"why":"Defines the COCO TS weak-supervision baseline whose annotation quality is compared against Char-SAM-generated COCO TS refined.","marker":"[15]"},{"why":"Defines the MLT S weak-supervision baseline whose annotation quality is compared against Char-SAM-generated MLT S refined.","marker":"[17]"},{"why":"Provides the supervised state-of-the-art TFT baseline that Char-SAM approaches within about one F-score point on TextSeg.","marker":"[14]"}],"fun_headline_variants":["SAM with character prompts beats word boxes for text","Char-SAM: glyph prompts fix SAM for scene text","Automatic text masks via SAM and character-level prompts","Training-free SAM text annotator reaches 92.15","Char-SAM: from word boxes to text segmentation masks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The CGR module assumes that each character category has a fixed glyph shape across fonts, styles, and writing directions, so that pixels from font-rendered templates can act as reliable positive and negative point prompts for real scene text.","fun_headline_variants_meta":{"raw":{"variants":["SAM with character prompts beats word boxes for text","Char-SAM: glyph prompts fix SAM for scene text","Automatic text masks via SAM and character-level prompts","Training-free SAM text annotator reaches 92.15","Char-SAM: from word boxes to text segmentation masks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000207,"raw_usage":{"total_tokens":1427,"prompt_tokens":996,"completion_tokens":431,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":353}},"tokens_in":612,"tokens_out":431,"duration_ms":4967,"temperature":1.0,"reasoning_tokens":353,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:47:09.118840+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Char-SAM on a test set of curved or artistic Latin text and non-Latin scripts such as Chinese, Japanese, or Arabic with manual character-level masks, and compare per-script F-score; a large drop on scripts without glyph templates, or on stylized letters whose template votes place wrong prompts, would falsify the fixed-glyph assumption.","supporting_citations":[{"cited_title":"Character region aware- ness for text detection,","cited_arxiv_id":null,"evidence_quote":"Provides the CRAFT character-aware detector used by CBR to split word-level boxes into character-level boxes."},{"cited_title":"Rethink- ing text segmentation: A novel dataset and a text-specific refinement approach,","cited_arxiv_id":null,"evidence_quote":"Provides the TextSeg benchmark with character-level masks used for evaluation and for the downstream training experiments."},{"cited_title":"Synthetic data for text localisation in natural images,","cited_arxiv_id":null,"evidence_quote":"Supplies SynthText training data for CRAFT; the domain gap between synthetic and real images motivates the recognition and watershed correction steps in CBR."},{"cited_title":"COCO TS dataset: Pixel–level annotations based on weak supervision for scene text segmentation,","cited_arxiv_id":null,"evidence_quote":"Defines the COCO TS weak-supervision baseline whose annotation quality is compared against Char-SAM-generated COCO TS refined."},{"cited_title":"Weak supervi- sion for generating pixel–level annotations in scene text segmentation,","cited_arxiv_id":null,"evidence_quote":"Defines the MLT S weak-supervision baseline whose annotation quality is compared against Char-SAM-generated MLT S refined."},{"cited_title":"Scene text segmentation with text-focused Transformers,","cited_arxiv_id":null,"evidence_quote":"Provides the supervised state-of-the-art TFT baseline that Char-SAM approaches within about one F-score point on TextSeg."}],"review_version":1}