{"id":"cedb0cd0-639c-46cd-9f38-0aa80f5fbc72","arxiv_id":"2412.18524","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"HTR-JAND reports state-of-the-art character error rates of 1.23% on IAM, 1.02% on RIMES, and 2.02% on Bentham by combining an efficient CNN-attention hybrid with knowledge distillation and a T5 error-correction stage.","lead":"The paper introduces HTR-JAND, a handwriting recognition system that combines an efficient CNN-attention architecture with knowledge distillation and a T5 text-correction stage, reporting very low character error rates on three standard benchmarks. A smart generalist might read it to see how far a carefully assembled pipeline of known components plus a learned post-processor can push recognition accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported SOTA CERs include a 60M T5 post-processor and lexicon correction absent from baselines, confounding the claimed gains; raw model comparisons are needed before the central claim can be accepted.","rationale":"The reader's reject verdict is justified, but I would put the weight on a different load-bearing assumption. Even if Proxima Attention were fully specified, the SOTA numbers would still not mean what the abstract asserts, because the final evaluation mixes an HTR model with a T5 post-processor, ensemble, and lexicon correction while the baselines are raw single models. This is a fairness and attribution issue, not merely a presentation issue: the headline numbers are for '+LBC', not for the 0.75M Student model, and the paper's own ablation shows the post-processing contribution is large and decisive on the benchmark columns. The proposed concrete check—reporting raw ensemble CERs without the post-processor in Table VI—would settle whether the claimed gain survives without external correction. The undefined Proxima Attention is also a genuine reproducibility defect and should be fixed, but the confounding of the comparison is the more direct threat to the central claim. I therefore agree with rejection, and no new verdict movement is needed.","tokens_in":15339,"tokens_out":5620,"duration_ms":55983,"concrete_test":"Compute and report the CER of the final model with the T5 post-processor and lexicon correction disabled (i.e., raw ensemble predictions) on IAM, RIMES, and Bentham, and place those numbers in Table VI alongside the current 'Ours (+LBC)' row. If the raw CERs (approximately 2.02, 1.89, and 3.12, as implied by Table V) are not below the best baseline in each column, the state-of-the-art claim rests entirely on external post-processing and the comparison is unfair.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The state-of-the-art claim (CER 1.23/1.02/2.02 on IAM/RIMES/Bentham) is not established as a property of the proposed HTR model because the reported best results come from a pipeline that includes components the comparison baselines do not use. Section V introduces a fine-tuned T5-small post-processor (60M parameters) that is said to reduce CER by 23.4% on average. The ablation in Table V shows the final CER on IAM declining from 2.02 (ensemble) to 1.23 only after '+LBC', and similarly 1.89->1.02 on RIMES and 3.12->2.02 on Bentham. Table VI then compares 'Ours (+LBC)' against single-model baselines (Retsinas et al., Puigcerver, etc.) with no equivalent post-processing or lexicon correction. This is an unfair comparison: it does not isolate the HTR architecture, and the reported advantage may be entirely due to the external correction stage. The efficiency claim is also affected: the abstract highlights a 0.75M-parameter Student model, but the final inference pipeline includes T5-small, so the stated parameter count understates the deployed system by a factor of roughly 80. Separately, the architecture as written is incomplete: Proxima Attention is not actually defined beyond 'K = XW_K, V = XW_V' (Eq. 17), despite Eq. 18 requiring O_Proxima, which blocks independent reimplementation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HTR-JAND, a handwritten text recognition framework that combines a FullGatedConv2d/SE CNN encoder, BiLSTM sequence modeling, a 'Combined Attention' mechanism (multi-head self-attention fused with 'Proxima Attention'), knowledge distillation, curriculum/synthetic/multi-task training, and a fine-tuned T5 post-processor. The authors report state-of-the-art CERs of 1.23%, 1.02%, and 2.02% on IAM, RIMES, and Bentham, with a 0.75M-parameter student model. The central claim is that the full pipeline is both more accurate and more parameter-efficient than existing HTR systems.","tokens_in":15643,"tokens_out":5969,"duration_ms":51273,"significance":"If properly supported, the framework would be a useful contribution to efficient HTR: it targets both accuracy and deployment cost, evaluates on several standard benchmarks, and promises released code and pretrained models. The paper also contains a reasonable high-level ablation strategy. However, the current manuscript does not establish the state-of-the-art claim because the headline results include external post-processing and an unspecified 'Proxima Attention' component, and the knowledge-distillation objective is defined inconsistently. The efficiency claim is not assessed for the complete inference pipeline. The ingredients are valuable, but the technical presentation and experimental controls need substantial revision before the claims can be accepted.","major_comments":[{"comment":"Proxima Attention is not defined. Eq. (17) only gives K = XW_K and V = XW_V; there is no definition of Q, no dynamic query update, and no expression for O_Proxima. Eq. (18) then uses O_Proxima in the combined output. The sentence 'Proxima Attention commuted using Eq. 16' does not specify how Proxima differs from standard multi-head attention. Since Proxima Attention is listed as one of the three key contributions and is credited in the conclusion, this gap blocks independent reimplementation and verification of the architecture.","section":"§III-B4, Eqs. (16)–(18)"},{"comment":"The knowledge-distillation loss is defined inconsistently. Eq. (23) defines Lkd = KL(softmax(zS_interp/τ), softmax(zT/τ)) · τ², while Eq. (28) defines Lkd = KL(softmax(zT/τ), softmax(zS/τ)) without the τ² factor and without the interpolation of zS. The arguments are reversed and the scaling differs. The actual training objective is therefore ambiguous, and this discrepancy affects the core distillation mechanism that the paper claims as a contribution.","section":"§III-C1 and §III-D, Eqs. (23) and (28)"},{"comment":"The ablation does not isolate the T5 post-processor. Section V reports that the fine-tuned T5-small model reduces CER by 23.4% on average, and the headline numbers in Table V appear after the '+LBC' column. The table has no row or column that separates the raw HTR model output from the T5-corrected output, and the text does not state whether '+LBC' includes the T5 stage or is a separate lexicon-correction step. Without this separation, the contribution of the proposed HTR architecture to the final CERs cannot be identified.","section":"§V and §VI-C, Table V"},{"comment":"The state-of-the-art comparison is not controlled. 'Ours (+LBC)' is compared with baselines that do not use an equivalent post-processing or lexicon-correction stage, so the reported advantage could come entirely from the external correction. A like-for-like comparison that reports the HTR model's raw CER, or that applies the same post-processing to the baselines, is required to support the 'state-of-the-art' claim. In addition, Table VI covers only IAM and RIMES, not Bentham, despite the abstract claiming state-of-the-art results on Bentham, and recent strong baselines are omitted.","section":"§VI-D, Table VI"},{"comment":"The claimed efficiency is not measured for the full deployed system. The abstract's 0.75M-parameter Student does not include the T5-small post-processor (60M parameters) or the ensemble/LBC stages used for the headline results. Table VII reports the Student model at CER 4.12% with 0.75M parameters, while the headline 1.23% is achieved only after the additional stages. The parameter count and inference time of the complete pipeline should be reported, and the contribution of each stage to both accuracy and cost should be separated.","section":"§VI-C, Table V and §VI-F, Table VII"},{"comment":"The ablation is internally inconsistent regarding multi-task learning. The text states that Table V shows the effects of Multi-Task Learning (MTL) and reports MTL improvements (e.g., 46.17% CER improvement on Saint Gall), but the table has no MTL column. The experimental support for MTL as a component is therefore missing from the reported data.","section":"§VI-C, Table V"}],"minor_comments":[{"comment":"There is a typo: 'including including' should be 'including'.","section":"§III-A"},{"comment":"'Proxima Attention commuted using Eq. 16' should read 'computed', and the sentence should be rewritten to actually define the operation.","section":"§III-B4"},{"comment":"The dataset name is written as 'Rimes' in one place and 'RIMES' elsewhere; please use a consistent spelling.","section":"§VI-C"},{"comment":"References [14] and [35] are duplicate entries of the same paper ('Transformer for handwritten text recognition using bidirectional post-decoding').","section":"References"},{"comment":"The header 'V ocab' contains an errant space; it should be 'Vocab'.","section":"Table II"},{"comment":"The output projection W^O in MultiHead is not defined; please specify its dimensions or state that it is the standard multi-head output projection.","section":"Eq. (16)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is not acceptable in its current form, but the main problems are fixable in principle: the authors can define Proxima Attention, reconcile the KD loss definitions, rerun the ablations with and without T5 post-processing, and provide like-for-like comparisons with baselines. If the Proxima Attention definition or the raw-model comparisons cannot be supplied, the paper should be rejected because the central SOTA claim would be unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before reading. The headline CERs are not produced by the proposed HTR model alone; they include a 60M-parameter T5-small post-processor plus lexicon correction, while the baselines in Table VI are single models without such post-processing. And the architecture as written cannot be reimplemented: Proxima Attention is named but never defined beyond K and V projections in Eq. 17. Both are load-bearing, not cosmetic.\n\nWhat is actually new: a specific combination of known components — FullGatedConv2d, SE blocks, BiLSTM, multi-head attention with something called Proxima Attention, knowledge distillation, curriculum learning, synthetic data, and T5 correction. The paper is straightforward about the pipeline nature of the system, and the ablation in Table V is genuinely useful. It shows the contribution of each stage, including a large CER drop from the lexicon-based correction. The efficiency table comparing teacher and student models is also informative. That is real work, and the authors shipped code.\n\nNow the soft spots, in rough order. First, Proxima Attention is never defined. Eqs. 16–18 mention a combined output and a dynamic query update, but there is no formula for the query update or for O_Proxima, so no one can reproduce the architecture from the text. Second, the knowledge-distillation loss is inconsistent: Eq. 23 uses interpolated student logits and a tau^2 factor; Eq. 28 drops both. Those are different losses. Third, the SOTA comparison is unfair. Table VI compares 'Ours (+LBC)' to baselines that do not have a T5 or lexicon stage, so the reported advantage does not isolate the HTR model. Relatedly, the abstract's 0.75M-parameter claim understates the deployed system by a factor of roughly 80 if T5-small is part of inference. Fourth, the abstract and Section VI claim SOTA on Bentham, but Table VI only reports IAM and RIMES; Bentham numbers are not compared. That is a smaller fix but worth noting.\n\nThe core idea — KD, curriculum, and a correction stage — is plausible and the ablation supports it. The paper deserves a serious referee, but it needs major revision: define or remove Proxima, reconcile the loss equations, and report comparisons with and without post-processing so the HTR model's contribution is separable.","headline":"Reported SOTA CERs come from a pipeline with T5 post-processing and lexicon correction, not the HTR model alone, and Proxima Attention is undefined; the paper needs major revision but deserves a serious referee.","tokens_in":16178,"tokens_out":2419,"would_cite":false,"duration_ms":23167,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HTR-JAND claims state-of-the-art character error rates of 1.23% on IAM, 1.02% on RIMES, and 2.02% on Bentham for handwritten text recognition.","keywords":["handwritten text recognition","knowledge distillation","combined attention","Proxima attention","FullGatedConv2d","Squeeze-and-Excitation","curriculum learning","T5 post-processing"],"falsifier":"The first concrete check is to run the released implementation, or an implementation built strictly from the paper, and confirm the 1.23%, 1.02%, and 2.02% CERs; because Proxima Attention is not mathematically defined, the text alone does not permit this. A second decisive check is to ablate the Combined Attention by replacing Proxima Attention with a second standard multi-head attention head and measuring IAM CER: if it remains 1.23%, Proxima Attention is not the active ingredient; if it degrades, the reported mechanism is doing the work.","tokens_in":15121,"feed_emoji":"✍️","tokens_out":6458,"duration_ms":54907,"temperature":0.7,"pith_summary":"The paper tries to establish that accurate handwritten text recognition, including on historical documents, does not require huge models. It proposes HTR-JAND, a framework that combines gated convolutions with Squeeze-and-Excitation blocks, a fused attention mechanism, knowledge distillation, curriculum learning, and T5-based post-processing. The central claim is that this pipeline achieves character error rates of 1.23% on IAM, 1.02% on RIMES, and 2.02% on Bentham, while a distilled Student model with 0.75M parameters matches much larger systems. If true, high-accuracy handwriting recognition becomes practical for resource-constrained digitization projects.","feed_headline":"Handwriting recognition hits 1.23% error on IAM, 1.02% on RIMES","feed_subtitle":"A 0.75M-parameter distilled student plus T5 post-editing beats far larger systems on modern and historical corpora.","key_machinery":"The load-bearing mechanism is the Combined Attention module, which fuses Multi-Head Self-Attention with Proxima Attention, an attention variant the paper describes as using linear projections for keys and values with dynamic query updates, though its exact mathematical definition is not fully given. The other central mechanism is knowledge distillation: a Teacher-Student training scheme in which the Student is trained with a four-part loss combining CTC, cross-entropy, a KL-divergence distillation term against the Teacher's softened logits, and an auxiliary classifier loss. These mechanisms sit on a CNN backbone of FullGatedConv2d and Squeeze-and-Excitation layers feeding bidirectional LSTMs.","core_discovery":"The paper's central claim is that a compact handwritten text recognition system can reach state-of-the-art accuracy by pairing gated and Squeeze-and-Excitation convolutions with a fused attention module, knowledge distillation, curriculum learning, and neural post-processing. On the IAM, RIMES, and Bentham benchmarks, the full pipeline reports character error rates of 1.23%, 1.02%, and 2.02%, respectively. The Student model, with 0.75M parameters versus the Teacher's 1.5M, keeps competitive accuracy while cutting parameters by 48% and inference time by roughly half. The paper attributes these gains to synergy among its components rather than to any single module.","pith_inferences":["The paper does not separately ablate Multi-Head Self-Attention from Proxima Attention, so its own evidence cannot tell which half of the Combined Attention is responsible for the gains; a reader should treat the fused module as the hypothesis to test.","The undefined Proxima Attention leaves open the possibility that the reported gains come from the training regimen rather than the attention mechanism, so an ablation that replaces Proxima Attention with a second standard attention head would be decisive.","The 23.4% CER reduction attributed to T5 post-processing suggests a testable extension: plugging the fine-tuned T5 into an unrelated HTR system and measuring whether the error correction transfers.","Because the Student retains competitive accuracy at 0.75M parameters, further compression via quantization or pruning is a natural extension not explored in the paper."],"forward_implications":["If the reported error rates hold, modern English and French handwriting recognition can move below 1.5% character error, and historical Bentham material below 2.1%, with models small enough for practical deployment.","The Student's 48% parameter reduction and roughly halved inference time imply that Teacher-level accuracy can be carried into resource-constrained digitization settings without a dedicated GPU at every processing node.","The ablation results imply that knowledge distillation, curriculum learning, ensemble learning, multi-task learning, and post-processing each contribute, so HTR systems should be designed as multi-stage pipelines rather than single architectures.","T5 post-processing correcting about 90% of residual errors suggests that a language-model stage can act as a general error-repair layer on top of an HTR front end."],"supporting_citations":[{"why":"Introduces CTC loss, the sequence-alignment objective that the HTR-JAND loss framework builds on.","marker":"[20]"},{"why":"Supplies gated convolutional layers, which HTR-JAND adapts as FullGatedConv2d for feature extraction.","marker":"[2]"},{"why":"Combines gated convolutions, Squeeze-and-Excitation, and post-processing, the closest architectural template HTR-JAND extends.","marker":"[17]"},{"why":"Provides the knowledge-distillation technique used to compress the Teacher into the Student.","marker":"[18]"},{"why":"Motivates data augmentation and curriculum learning for HTR, which HTR-JAND integrates into its multi-stage training.","marker":"[19]"},{"why":"The T5 model that HTR-JAND fine-tunes for context-aware post-processing.","marker":"[39]"},{"why":"Supplies the IAM benchmark dataset used for the headline 1.23% CER result.","marker":"[36]"},{"why":"Supplies the RIMES benchmark dataset used for the 1.02% CER result.","marker":"[37]"},{"why":"Supplies the Bentham historical dataset used for the 2.02% CER result.","marker":"[38]"}],"fun_headline_variants":["Distilled handwriting model hits 1.23% error on IAM","Half the params, state-of-the-art HTR: 1.23% CER on IAM","Compact HTR student: 0.75M params, 1.23% CER on IAM","Knowledge distillation slashes HTR params by 48% without losing accuracy","HTR-JAND: attention + distillation = 1.02% CER on RIMES"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Combined Attention mechanism's 'Proxima Attention' component can be implemented as intended, yet the paper never defines it beyond saying keys and values are linear projections of the input, so if that component is not reproducible the reported error rates cannot be reproduced either.","fun_headline_variants_meta":{"raw":{"variants":["Distilled handwriting model hits 1.23% error on IAM","Half the params, state-of-the-art HTR: 1.23% CER on IAM","Compact HTR student: 0.75M params, 1.23% CER on IAM","Knowledge distillation slashes HTR params by 48% without losing accuracy","HTR-JAND: attention + distillation = 1.02% CER on RIMES"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00086,"raw_usage":{"total_tokens":3765,"prompt_tokens":1009,"completion_tokens":2756,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":2642}},"tokens_in":625,"tokens_out":2756,"duration_ms":16968,"temperature":1.0,"reasoning_tokens":2642,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:41:13.811467+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The first concrete check is to run the released implementation, or an implementation built strictly from the paper, and confirm the 1.23%, 1.02%, and 2.02% CERs; because Proxima Attention is not mathematically defined, the text alone does not permit this. A second decisive check is to ablate the Combined Attention by replacing Proxima Attention with a second standard multi-head attention head and measuring IAM CER: if it remains 1.23%, Proxima Attention is not the active ingredient; if it degrades, the reported mechanism is doing the work.","supporting_citations":[{"cited_title":"Connection- ist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,","cited_arxiv_id":null,"evidence_quote":"Introduces CTC loss, the sequence-alignment objective that the HTR-JAND loss framework builds on."},{"cited_title":"Gated convolutional recurrent neural net- works for multilingual handwriting recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies gated convolutional layers, which HTR-JAND adapts as FullGatedConv2d for feature extraction."},{"cited_title":"HTR-Flor: A Deep Learning System for Offline Handwritten Text Recognition,","cited_arxiv_id":null,"evidence_quote":"Combines gated convolutions, Squeeze-and-Excitation, and post-processing, the closest architectural template HTR-JAND extends."},{"cited_title":"Learning from multiple teacher networks,","cited_arxiv_id":null,"evidence_quote":"Provides the knowledge-distillation technique used to compress the Teacher into the Student."},{"cited_title":"Data augmentation for recognition of handwritten words and lines using a cnn-lstm network,","cited_arxiv_id":null,"evidence_quote":"Motivates data augmentation and curriculum learning for HTR, which HTR-JAND integrates into its multi-stage training."},{"cited_title":"The iam-database: an english sentence database for offline handwriting recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the IAM benchmark dataset used for the headline 1.23% CER result."},{"cited_title":"Results of the rimes evaluation campaign for handwritten mail processing,","cited_arxiv_id":null,"evidence_quote":"Supplies the RIMES benchmark dataset used for the 1.02% CER result."},{"cited_title":"Building a volunteer community: results and findings from transcribe bentham,","cited_arxiv_id":null,"evidence_quote":"Supplies the Bentham historical dataset used for the 2.02% CER result."}],"review_version":1}