{"id":"1528fbbb-422d-407b-beab-9360ab5d19a2","arxiv_id":"2506.15010","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"PALETTE improves historical map text spotting by using character centers and boundary points as hyper-local reference points for deformable attention, and SynthMap+ provides synthetic map training data.","lead":"PALETTE is a text spotting model that samples image features at predicted character and boundary point locations rather than a fixed text center, improving text extraction from historical maps. It also introduces SynthMap+, a synthetic map dataset, and the system has been deployed on 60,000 maps to generate 100 million labels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Character-center supervision during iterative training may be circular: predicted centers are validated only by whether they fall inside the GT boundary, so biased centers could be re-added to training and inflate the reported gains.","rationale":"The reader and I both focus on the character-center predictor and the iterative training loop, so there is partial agreement. My concern is more specific and more consequential: the inside-boundary validation criterion in Section 2.5 can admit biased centers, and iterative training then reinforces those biases, potentially inflating reported improvements because the strongest baselines (TESTR, DeepSolo) do not have this self-training loop. I do not think this warrants changing the CONDITIONAL verdict, because the paper provides ablations showing hyper-local sampling components matter (Table 5), SynthMap+ helps multiple models (Table 4 and Appendix B), and the deployment over 60,000 maps is independent evidence of practical utility. However, the suggested experiment should be run before the claim that hyper-local sampling is the decisive factor is taken as established. The author-created benchmarks and absence of significance testing are secondary concerns that also support the CONDITIONAL verdict.","tokens_in":16653,"tokens_out":1463,"duration_ms":14761,"concrete_test":"Run the finetuning loop without the iterative pseudo-label additions (finetune on Total-Text only, disabling character-center loss whenever centers are unavailable, as in pretraining), then evaluate on Grinnell-UMass-31 and Rumsey-309. If the E2E-None gap between PaLeTTe and TESTR shrinks by more than roughly half of the reported 2.5–3.3 point advantage, the iterative character-center pseudo-labeling is a major contributor and the hyper-local sampling claim is weaker than presented. Additionally, on a held-out subset of SynthMap+ images where true character centers are known, compute the distance between predicted and true centers to quantify how often the inside-boundary rule admits biased centers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that hyper-local sampling at predicted character centers drives the improvements. This requires the predicted centers to be accurate enough to sample the right features. Section 2.5 says that during finetuning, predicted centers are added to training only if they fall inside the ground truth boundary, and are otherwise replaced by fake centers on the center line. That criterion is weak: a center can lie inside the boundary yet be far from the true character location, especially for long, curved, or highly rotated text where the boundary encloses a large area. The iterative loop then trains on those same biased predictions, so the model may be re-predicting and reinforcing its own biases rather than learning from independent supervision. Because the ablations remove hyper-local sampling components but do not isolate the quality of the predicted character centers, the reported gains over TESTR and DeepSolo could be partly attributed to this self-training mechanism rather than to genuinely better sampling. The reader's weakest assumption identifies the same module but frames it as a trainability concern; my concern is that the inside-boundary filter can admit systematically biased centers and thereby inflate the headline results.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PaLeTTe, an end-to-end text spotter for historical maps built on Deformable DETR. The core idea is hyper-local sampling: predicted boundary-point locations and character centers are used as reference points for deformable attention, replacing the coarse instance-center reference used in prior DETR-based spotters. A character-center predictor infers character centers from boundary points and character content queries, and an iterative training procedure adds predicted centers as pseudo-labels when they fall inside the ground-truth text boundary. The paper also introduces SynthMap+, a synthetic map-image generator, and a new annotated benchmark, Rumsey-309. Experiments on Rumsey-309 and Grinnell-UMass-31 show that PaLeTTe outperforms ABCNet-v2, SWINTS, TESTR, and DeepSolo, with ablations indicating that hyper-local sampling and hyper-local positional embeddings contribute to the gains.","tokens_in":16880,"tokens_out":7262,"duration_ms":66851,"significance":"If the empirical results hold, this is a practically valuable contribution: the deployment on 60,000 David Rumsey maps with over 100 million extracted labels demonstrates real-world impact. The release of code, synthetic data, and a new benchmark is a concrete strength, as is the standard training and evaluation protocol. The central mechanistic claim—that sampling features around predicted sub-component locations improves spotting on long and rotated map text—is well motivated and supported by the pretrained (non-finetuned) results in Table 1, which show the largest gains and do not rely on the iterative pseudo-label procedure. The main uncertainty is the quality and possible circularity of the pseudo-labeled character centers used in finetuning, which deserves a more careful analysis than the paper currently provides.","major_comments":[{"comment":"The criterion for accepting a predicted character center as 'correct' is that it falls inside the ground-truth text boundary. For long, curved, or highly rotated text, the polygon encloses a large area, so a center can be inside the boundary yet far from the true character position. Because these accepted centers are then used as training labels in subsequent iterations, the model can reinforce its own biased predictions rather than learning from independent supervision. The pretrained results in Table 1 do not use this procedure and are therefore unaffected, but the finetuned results and the claim that iterative training alleviates the need for character-center annotation depend on it. Please provide a quantitative evaluation of pseudo-label quality (e.g., distance of predicted centers to manually annotated centers on a held-out subset of Total-Text or the evaluation maps) and an ablation comparing finetuning with and without iterative training on both benchmarks, reporting E2E and detection F1. This would help determine whether the finetuned gains are robust or partly an artifact of self-training.","section":"§2.5"},{"comment":"The ablation results in Table 5 are not reported consistently with the text. The text states that removing hyper-local sampling gives a 2.9% reduction in detection F1 (first row) and a 5.1% reduction in recognition performance (second row), but Table 5 shows PaLeTTe-wo-HLD at 82.3 vs 84.7 detection F1 (a 2.4-point difference) and PaLeTTe-wo-HLR at 66.4 vs 69.8 E2E-None (a 3.4-point difference). Please correct the numbers and clarify whether the reductions are percentage points or relative percentages, and whether they refer to E2E-None or a separate recognition metric. In addition, please state explicitly whether the ablations are on the pretrained or finetuned models, since the paper does not specify this for Table 5.","section":"§4.4.5"},{"comment":"The newly introduced Rumsey-309 benchmark is central to the evaluation, but no annotation-quality statistics are reported. Please provide details on the annotation protocol: the number of annotators, the annotation tool, inter-annotator agreement on a subset, and any quality-control steps. It would also be useful to clarify whether any model selection or hyperparameter tuning was performed on the evaluation sets, since the authors are also the creators of the benchmark and the developers of the deployment system.","section":"§4.2.2"}],"minor_comments":[{"comment":"The word 'Figrue' is misspelled as 'Figrue' in the caption.","section":"Figure 1 caption"},{"comment":"The name 'PaLeTTe' is sometimes run together with adjacent words, e.g., 'PaLeTTewithSynthMap+' in the abstract and 'PaLeTTeprogressively' in Section 2. Please add spaces after 'PaLeTTe'.","section":"Abstract and Introduction"},{"comment":"There is a missing space in 'Weinman et al. [50]propose' in the first paragraph.","section":"§6.1"},{"comment":"The benchmark name is written as 'Rumsey309' in the limitations section, while the rest of the paper uses 'Rumsey-309'. Please make the naming consistent.","section":"§7"},{"comment":"The orientation intervals are inconsistent: the Grinnell columns use [30, 60) and [60, 90], while the Rumsey columns use (30, 60] and (60, 90]. Please use consistent interval notation.","section":"Table 3"},{"comment":"The reference 'In Figure 2 1' should be 'In Figure 2(a)' or similar for clarity; the current notation appears to be a rendering artifact.","section":"§2.2"},{"comment":"In the character-center predictor equations, the dot notation in 'wq·(...)' is not standard for matrix-vector products; please use explicit linear projection notation for readability.","section":"§2.3"}],"recommendation":"major_revision","confidential_remarks":"The paper's authors are closely connected to both the mapKurator deployment and the newly introduced Rumsey-309 benchmark. This is not itself a conflict, but independent replication or at least a transparent annotation-quality report would substantially strengthen the evaluation. The major comments above are intended to address this concern without questioning the authors' integrity."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: PALETTE is a serious, applied text-spotting paper for historical maps. The hyper-local sampling idea—using predicted boundary points and character centers as deformable-attention reference points instead of a fixed proposal center—is genuinely new relative to TESTR and DeepSolo, and the experiments back it up. This paper is worth a careful referee.\n\nWhat's new: the hyper-local sampling module and hyper-local positional embeddings; SynthMap+, a synthetic map generator using QGIS label placement and real map backgrounds; and the Rumsey-309 benchmark. The deployment on 60,000 Rumsey maps with 100M labels is real-world evidence that the system works at scale. The authors also give an honest limitations section and release code and data.\n\nThe experimental protocol is standard: pretrain on synthetic plus human data, finetune on Total-Text, evaluate on two map datasets. Ablations show that removing hyper-local sampling for detection or recognition drops F1 by 2.9 and 5.1 points, and removing hyper-local positional embeddings also hurts. The gains over TESTR and DeepSolo are consistent across datasets, including the pretrained models.\n\nSoft spots: the new Rumsey-309 benchmark is author-created, but it's public and the annotation process is described. No significance testing, but the margins are large enough that this is minor. The 'first wide-variety map text spotter' claim is a bit overstrong since prior map-specific methods (Weinman, Li et al.) are not compared empirically, though the intro and related work do acknowledge them.\n\nThe bigger concern is the iterative character-center training in Section 2.5. The filter for adding pseudo-labels—all predicted centers must fall inside the ground truth boundary—is weak for long, curved, or rotated text where the polygon encloses a lot of area. Biased centers could be re-added and reinforce themselves. However, this does not sink the paper: the pretrained models (no iterative training) already beat baselines by 3.3 to 5.8 points, and the ablations isolate the hyper-local components. So the headline result is not dependent on the self-training loop. The finetuning gains should be read with that caveat.\n\nBottom line: this is a sound, well-written applied contribution. The math is straightforward empirical validation, no derivation issues. The citation pattern looks fair, with prior work properly referenced. I'd send it to peer review and expect it to be accepted after fixing the overclaim and discussing the weak filter.\n\nRecommendation: this paper deserves a serious referee; the central idea holds.","headline":"Solid applied paper with a real mechanism behind the gains; the iterative character-center training deserves a skeptical read but doesn't sink the central claim.","tokens_in":17378,"tokens_out":3378,"would_cite":true,"duration_ms":29468,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A text spotting model that anchors deformable attention to each predicted character center and boundary point outperforms state-of-the-art spotters on historical maps, with the largest gains on long and highly rotated labels.","keywords":["historical maps","text spotting","text detection and recognition","deformable attention","hyper-local sampling","synthetic map data","character center prediction"],"falsifier":"Rerun the paper's without-hyper-local-character-sampling ablation—fixing all character reference points at the proposal center—on the larger historical map benchmark and verify that end-to-end F-score drops by the reported roughly 5 points and that the drop concentrates on text of length 7 or more and on rotations above 60 degrees. If fixed-center sampling with identical training data matches or beats hyper-local sampling, the mechanism is not what carries the improvement. A second check: jitter the supervised character centers by a few pixels during training and measure whether end-to-end performance degrades, which would confirm the model actually depends on precise reference points.","tokens_in":16460,"feed_emoji":"🗺️","tokens_out":8960,"duration_ms":91081,"temperature":0.7,"pith_summary":"This paper argues that text spotting on historical maps fails when a whole text instance is represented by a single coarse reference point, because long, curved, and highly rotated labels have sub-components—boundary points and characters—that can lie far from that point. It proposes PaLeTTe, which treats each boundary point and each character as its own target, predicts where they are, and uses those predicted locations as the reference points for deformable attention, so image features are always sampled from the immediate neighborhood of the component being predicted. The paper also builds a synthetic map dataset, SynthMap+, that follows cartographic placement rules and reuses textures from real scanned map backgrounds, and shows that adding it to pretraining boosts both PaLeTTe and baseline spotters. Taken together, the claim is that one end-to-end model can read arbitrary-shaped, long, and highly rotated text across a wide variety of scanned historical maps.","feed_headline":"Text spotter beats rivals by sampling around each character","feed_subtitle":"Hyper-local attention anchored to each predicted character center lifts accuracy on long, rotated map text.","key_machinery":"The central mechanism is hyper-local sampling inside a Deformable DETR-style decoder. For each content query representing a boundary point or a character, the reference point for deformable attention is the query's own predicted location rather than the text-instance center, and the network samples image features around that location with learned offsets. A character-center predictor computes character centers from character and boundary-point queries via cross-attention, and the predicted positions are also injected as hyper-local positional embeddings so each sub-component knows where it sits relative to the others. This gives every query an explicit local positional prior and lets intra- and inter-instance self-attention reason about the spatial arrangement of boundary points and characters.","core_discovery":"PaLeTTe establishes that hyper-local reference points—predicted character centers for recognition and predicted boundary points for detection—carry the positional information that deformable attention needs, and that coarse instance-level reference points are the main bottleneck. Starting from box-proposal centers, the decoder predicts boundary points, then a character-center predictor reads character and boundary queries to estimate character centers; in subsequent layers the sampling and positional embeddings are re-anchored to these refined predictions. On two new hand-annotated historical map benchmarks, this yields end-to-end spotting F-score gains of roughly 3 to 6 points over the strongest baselines, with the largest improvements concentrated on text longer than ten characters and on text rotated into the 60–90 degree range. The paper further shows that character-center supervision can be bootstrapped from boundary-only annotations through iterative training, which is what makes the method applicable to real maps where character centers are not labeled.","pith_inferences":["If hyper-local re-anchoring is the cause of the gains, a testable extension is to apply the same decoder modification to scene-text spotters; a similar improvement there would show the mechanism is about sub-component alignment, not about the map domain.","The SynthMap+ recipe—separating geometric text placement from background style and sourcing backgrounds from real scans—could be pointed at other document types, such as manuscripts or engineering drawings, to generate synthetic training data with the same ease.","The failure cases the paper itself reports (very large characters, overlapping text and line features) suggest the method inherits a sensitivity to reference-point accuracy: when the character-center predictor is confidently wrong, hyper-local sampling could lock onto the wrong features, so an uncertainty estimate on predicted centers might extend the approach to its own hard cases.","A further consequence left implicit is that the character-center predictor gives the model a free alignment signal between detection and recognition; one could exploit that alignment as a confidence measure for spotting, scoring a detection as more reliable when boundary points and character centers agree."],"forward_implications":["Long and highly rotated text, the cases where fixed-center sampling is most misaligned, show the largest gains: on text of length 10 or more the end-to-end recall rises by about 8 points on one benchmark, and for rotations of 60–90 degrees it rises by about 4.8 points on the larger benchmark.","Adding SynthMap+ to pretraining improves PaLeTTe's end-to-end F-score by about 24.8 points on the larger benchmark and also improves the TESTR baseline, indicating the synthetic data is a reusable training resource rather than a model-specific fix.","Ablating hyper-local sampling for boundary points costs about 2.9 points of detection F1, and ablating it for characters costs about 5.1 points of end-to-end performance, evidence that re-anchoring to predicted sub-component locations is the load-bearing design choice.","The iterative training procedure lets human annotations without character-center labels be folded into training, which is what allows the model to be finetuned on real maps at scale.","Deployed on more than 60,000 maps, the model generated over 100 million text labels used for full-text map search, an existence proof that the approach scales beyond benchmarks."],"supporting_citations":[{"why":"Supplies the sparse deformable attention mechanism that PaLeTTe re-anchors to hyper-local reference points.","marker":"[55]"},{"why":"TESTR, the dual-decoder text spotter that PaLeTTe extends and its strongest comparison baseline.","marker":"[54]"},{"why":"DeepSolo, the center-line sampling text spotter that PaLeTTe compares against on long text.","marker":"[53]"},{"why":"ABCNet-v2, the Bezier-curve baseline whose recognition drops on highly rotated map text.","marker":"[34]"},{"why":"Prior synthetic map generation that lacks transcriptions and covers only one map style, the gap SynthMap+ addresses.","marker":"[30]"},{"why":"Supplies one of the two hand-annotated historical map benchmarks used for evaluation.","marker":"[39]"},{"why":"Self-training inspiration for the iterative character-center bootstrap used on human-annotated data.","marker":"[51]"},{"why":"Total-Text, the scene-text dataset used for finetuning because it empirically transfers best to the historical map tests.","marker":"[8]"}],"fun_headline_variants":["Hyper-local sampling lifts text spotting on historical maps","Palette: hyper-local deformable transformers beat SOTA on map text","Character-level anchors improve text spotting for long rotated labels","New method and synthetic data boost text extraction from old maps","Hyper-local attention outperforms on angled map text"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the character-center predictor can learn accurate character centers from boundary points and character queries, and that any predicted center landing inside the ground-truth boundary is considered correct during iterative training; if those centers are systematically biased, hyper-local sampling anchors to the wrong image locations and the reported gains should shrink or disappear.","fun_headline_variants_meta":{"raw":{"variants":["Hyper-local sampling lifts text spotting on historical maps","Palette: hyper-local deformable transformers beat SOTA on map text","Character-level anchors improve text spotting for long rotated labels","New method and synthetic data boost text extraction from old maps","Hyper-local attention outperforms on angled map text"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000401,"raw_usage":{"total_tokens":2150,"prompt_tokens":1056,"completion_tokens":1094,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":672,"completion_tokens_details":{"reasoning_tokens":1015}},"tokens_in":672,"tokens_out":1094,"duration_ms":11509,"temperature":1.0,"reasoning_tokens":1015,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:08:49.760493+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the paper's without-hyper-local-character-sampling ablation—fixing all character reference points at the proposal center—on the larger historical map benchmark and verify that end-to-end F-score drops by the reported roughly 5 points and that the drop concentrates on text of length 7 or more and on rotations above 60 degrees. If fixed-center sampling with identical training data matches or beats hyper-local sampling, the mechanism is not what carries the improvement. A second check: jitter the supervised character centers by a few pixels during training and measure whether end-to-end performance degrades, which would confirm the model actually depends on precise reference points.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TESTR, the dual-decoder text spotter that PaLeTTe extends and its strongest comparison baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DeepSolo, the center-line sampling text spotter that PaLeTTe compares against on long text."},{"cited_title":"ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting","cited_arxiv_id":"2105.03620","evidence_quote":"ABCNet-v2, the Bezier-curve baseline whose recognition drops on highly rotated map text."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior synthetic map generation that lacks transcriptions and covers only one map style, the gap SynthMap+ addresses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies one of the two hand-annotated historical map benchmarks used for evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Self-training inspiration for the iterative character-center bootstrap used on human-annotated data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Total-Text, the scene-text dataset used for finetuning because it empirically transfers best to the historical map tests."}],"review_version":1}