{"id":"17cf5eee-b8a1-4892-a816-1d13498cf6b5","arxiv_id":"2412.10155","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Recoloring words with a hand-crafted letter-to-color scheme improves image-only document classifiers on Tobacco-3482 by 3-5%, reaching a reported 91.14%.","lead":"A preprocessing method that recolors each word in a document image based on its letters lets small image classifiers sort documents more accurately. It offers a cheap alternative to large multimodal models for companies with little data or computing power.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim requires the specific character-score color mapping to cause the gains, but Table 2 only compares against no preprocessing; a random-colorization control is missing, so the semantic mechanism remains untested.","rationale":"The reader's weakest_assumption is the same one I would flag. The numerical results are not impossible: OCR-derived text highlighted in color plausibly helps a CNN on Tobacco-3482, a small dataset with highly distinctive document classes. The consistency across four architectures in Table 2 is genuine supporting evidence that the preprocessing is useful. What the paper does not establish is that the specific similarity-based character-to-color encoding is responsible, as opposed to word-length scaling, word highlighting, or any colored overlay. This matters because the abstract and introduction frame the contribution as embedding textual semantic features in accordance with the similarity of different letters. If a random color mapping performs the same, the method reduces to coloring words, which is a weaker and different claim. I would keep the reader's CONDITIONAL verdict: the empirical claim is promising, but the mechanism test, the exact mapping, and error bars are required before accepting the semantic interpretation. My concern does not change the verdict; it reinforces the main condition.","tokens_in":8274,"tokens_out":6191,"duration_ms":71542,"concrete_test":"Reproduce the Table 2 ResNet50 experiment with three versions of the preprocessing: (i) WordVIS exactly as specified; (ii) the same OCR boxes, M_f, and RGB formula, but with the 36-character score table replaced by a random permutation (5 independent seeds); (iii) the same OCR boxes with a random RGB color per word, removing both character scores and word-length scaling. Use the Sec. 4.2 hyperparameters and the Sec. 4.1 split protocol, reporting mean ± std. If (ii) or (iii) is within noise of (i), the semantic mapping is not the load-bearing ingredient; only if (i) is clearly above both can the central claim stand.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the attribution of the accuracy gains to the semantic character-score encoding of Sec. 3.1. WordVIS changes the input image in three simultaneous ways: OCR word boxes become colored regions; each word's color is scaled by word length through the multiplying factor M_f (Sec. 3.2); and characters are mapped to channels and scores through a hand-chosen lookup. Table 2 compares only WordVIS versus no preprocessing, so any of these three ingredients, or any deterministic colorization of words, could account for the reported 3-5% gains. If a random permutation of the score table, or even per-word random colors, yields similar accuracy, the paper's central claim that the similarity-based text encoding is what helps is not supported. The exact lookup table is not included, and Sec. 3.1's example (d=3, e=5 under an ascending assignment) is not internally consistent with a straightforward alphabetical scoring, which makes the tested mapping non-reproducible and the attribution harder to audit. None of this disproves the empirical improvements; it means the controlled experiment needed to isolate the mechanism has not been run.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes WordVIS, a preprocessing step for document image classification. OCR extracts words, each word is assigned an RGB color by summing hand-assigned per-character scores grouped into R/G/B channels and multiplying by word length (Sections 3.1-3.3, Eq. 1). The colorized documents are then used to train standard image classifiers. On Tobacco-3482, Table 2 reports consistent gains over no preprocessing across ResNet50, ResNet101, DenseNet121, and EfficientNetV2 (e.g., ResNet50 67.9% to 72.5%), and Table 1 reports 91.14% with DocXClassifier-B, claimed as a new benchmark record without document pretraining.","tokens_in":8465,"tokens_out":6000,"duration_ms":61574,"significance":"If the proposed mechanism is responsible for the gains, WordVIS is an attractive low-resource alternative to multimodal document classifiers: it avoids extra pretraining, is in principle language-agnostic, and can be plugged into existing CNN pipelines. The paper also provides qualitative attention heatmaps showing that the colorized model focuses more tightly on text regions. However, the current evidence does not isolate the semantic character-score mapping as the cause of the improvements; the missing controls, incomplete specification of the mapping, and single-dataset point estimates leave the central claim and its reproducibility questionable. The practical direction is plausible but not yet established at the level claimed.","major_comments":[{"comment":"The central claim is that the semantic character-score encoding causes the accuracy gains, but Table 2 only compares WordVIS against no preprocessing. WordVIS changes the input in at least three simultaneous ways: OCR word regions are colorized, colors are scaled by word length through M_f, and characters are mapped through a hand-chosen score table. Without controls such as a randomly permuted score table, random per-word colors, or a uniform colorization of word boxes, the reported 3-5% improvements cannot be attributed to the semantic scoring; they could arise from any deterministic colorization that highlights text. A control experiment isolating the score mapping is necessary to support the paper's main claim.","section":"Sections 3.1, 3.2, and Table 2"},{"comment":"The score assignment is not reproducible as written. Section 3.1 describes a generic ascending assignment over 36 characters, but the 'deep' example in Section 3.3 uses d=3, e=5, p=7, which does not match a straightforward alphabetical ascending assignment (d=4, p=16 in a 1-based alphabet) and is inconsistent with the channel ranges in Eq. (1). The OCR-error example also appears to use p=8 for 'deeq', where the changed character is q. The exact lookup table is never provided. Without the precise mapping, the experiments cannot be replicated and the claimed robustness property is not auditable.","section":"Sections 3.1 and 3.3"},{"comment":"The empirical evaluation rests on a single random 80/20 split of one dataset and reports only point estimates. The text states that experiments were performed multiple times to rule out standard deviation, but no variance, number of runs, or statistical test is reported. Given that the reported gains are 1.7 to 4.6 percentage points, confidence intervals or a paired analysis across repeated splits are needed before the improvements can be considered consistent; otherwise the differences could be within split-to-split noise.","section":"Sections 4.1 and 4.4, Table 2"},{"comment":"The claim of a new state-of-the-art record on Tobacco-3482 is not supported by a controlled comparison under identical evaluation conditions. Prior results in Table 1 come from different publications with potentially different train/validation/test protocols, while this paper uses its own random split; no error bars are reported for any baseline. To substantiate a new record, the WordVIS model and the DocXClassifier baselines should be evaluated on the same splits and with the same OCR pipeline, and the standard benchmark protocol should be followed or explicitly justified.","section":"Section 4.3 and Table 1"}],"minor_comments":[{"comment":"The sentence before Eq. (1) says the formula is 'given below in Fig. 1 and Fig. 2', but the formula appears as equations, not in the figures; this should be corrected.","section":"Section 3.3"},{"comment":"There are typos in the dataset name: 'Tabaccoo-3482' appears in Section 4.1 and in the Table 1 caption, and the name is inconsistent with 'Tobacco-3482' used elsewhere.","section":"Sections 4.1 and 4.3"},{"comment":"The notation 'lim 1→∞ Nc' and 'lim 0→255 Cs' is nonstandard and should be replaced with clear statements about the ranges of Nc and Cs.","section":"Section 3.1"},{"comment":"The stray text 'ferrandodc' appears at the end of the training details paragraph and should be removed.","section":"Section 4.2"},{"comment":"The conclusion states that heatmap analysis 'drastically improve[s] the quantitative results', but the preceding discussion is about qualitative attention patterns; this appears to be a typo for 'qualitative results'.","section":"Section 5"},{"comment":"The example class 'ADVE' is used in the qualitative discussion, but the dataset classes are never listed; please define the ten classes of Tobacco-3482 and specify how sample documents were selected for the figures.","section":"Sections 3.4 and Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is very short for the scope of its claims, and the evaluation section lacks the detail needed to verify either the mechanism or the benchmark record. The main risk is not that the reported numbers are fabricated, but that the semantic-encoding claim is untested; the comparison against no preprocessing is insufficient. I would condition acceptance on the controlled experiments and reproducibility details described in the major comments, and I would note that the paper may fit better as an application-style contribution once those are provided."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does one simple thing: it colorizes each word in a document image using per-character scores, then trains an ordinary CNN on the result. On Tobacco-3482, WordVIS gives consistent 3–5% gains over plain images across ResNet50, ResNet101, densenet121, and EfficientNetV2, and pushes DocXClassifier-B to 91.14%. That is a genuinely useful result for low-resource document classification, and the idea is new to this task even though it borrows from Chargrid and BERTGrid. The writing is clear, the ablations cover several architectures, and the authors are upfront that the mapping is hand-chosen and generic.\n\nThe soft spots are real but fixable. The biggest one: Table 2 only compares WordVIS against no preprocessing. WordVIS changes the input in three ways at once — colored word regions, word-length scaling, and the specific character-to-channel score table. Any deterministic colorization of words might give similar gains. The paper needs a control experiment with, say, random per-word colors or a shuffled score table. Without that, the claim that the semantic encoding is the mechanism is unsupported. Second, the exact score table is not given, and the worked example is internally inconsistent (d=3, e=5, p=7 under an ascending alphabetical assignment; the OCR typo example then uses p=8 for q). This makes the method non-reproducible as written. Third, the authors say they ran multiple splits to rule out variance, but they report no error bars. Fourth, the evaluation is on a single dataset. These are all straightforward to address.\n\nThe core direction is plausible and the improvements, if real, would be valuable to practitioners who cannot run large multimodal models. The paper deserves a serious referee, but it should not be accepted as is. I would send it back with a request for the code and exact mapping, a random-colorization baseline, error bars, and ideally a second dataset. If the control experiment shows that any colorization works, the paper becomes a much weaker empirical observation; if the semantic mapping matters, the result is worth publishing.\n\nThis is a job for a competent referee, not a desk reject, and not a fast accept either.","headline":"Simple word-colorization trick shows consistent gains on Tobacco-3482, but the paper never tests whether the semantic score mapping is what drives them.","tokens_in":8970,"tokens_out":1533,"would_cite":false,"duration_ms":17716,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A word-to-color preprocessing step lets lightweight image classifiers beat multimodal models on Tobacco-3482, reaching 91.14% accuracy without document pretraining.","keywords":["document image classification","document classification","image processing","data pre-processing","visual embeddings","textual embeddings","deep learning","Tobacco-3482"],"falsifier":"Train the same DocXClassifier-B on Tobacco-3482 images whose words are colored by a random permutation of the WordVIS lookup table, or by a hash of each word to a random RGB value, keeping all other settings identical; if accuracy stays near 91.14%, then the specific semantic score assignment is not the cause of the improvement.","tokens_in":1514,"feed_emoji":"🎨","tokens_out":1855,"duration_ms":107268,"temperature":0.7,"pith_summary":"The paper presents WordVIS, a preprocessing step that embeds a document's text into its image by recoloring every word with an RGB value computed from a hand-built character-score table. The authors claim that this recoloring lets standard image classifiers pick up textual cues, giving consistent accuracy gains on the small Tobacco-3482 benchmark, including 4.64% for ResNet50 with no document pretraining. Applied to the image-based DocXClassifier-B, WordVIS pushes accuracy from 88.42% to 91.14%, the best published score on that dataset without document pretraining. If the claim holds, lightweight convolutional models could replace large multimodal transformers in data-scarce document classification.","feed_headline":"Colorizing words by letter scores sets a 91.14% accuracy record","feed_subtitle":"Lightweight image models beat multimodal AI models on the Tobacco-3482 benchmark without document pretraining.","key_machinery":"The load-bearing object is the WordVIS colorization, inspired by Levenshtein distance: for each word, each character is assigned a score between 1 and 9 and one of three channels, the channel scores over the word are summed, and the sums are multiplied by a factor derived from word length to give the word's RGB triplet (for example, the word 'deep' becomes $(52,28,0)$). This maps every word to a color while remaining nearly invariant to OCR swaps of similar letters, and it makes stop words and content words take distinct hues. The mechanism works by converting the document's text into a fixed, deterministic visual pattern that convolutional kernels can learn from, without any learned embedding or pretraining.","core_discovery":"The central discovery is that a document's semantic content can be carried by color: each letter gets a numeric score and one of three channels, each word's channel scores are summed, and the sums are multiplied by the word's length to produce a deterministic RGB color. Colorized words overlaid on the original image convert the text into a visual signal that a CNN can read directly. The paper argues that this is why WordVIS preprocessing improves accuracy across ResNet50, ResNet101, DenseNet121, EfficientNetV2, and DocXClassifier-B, and why the DocXClassifier-B result of 91.14% beats larger multimodal models that require millions of pretraining samples. The claim is that textual semantic features can be embedded into the visual space with a simple, language-agnostic lookup table, removing the need for separate text encoders or document pretraining.","pith_inferences":["The paper never tests whether a random or permuted character-to-color mapping would give the same gains; a colorization that simply gives every word a distinct color might be the real mechanism, not the semantic score assignment.","The same word-to-color embedding could be applied to other document tasks, such as key information extraction or layout analysis, where related color-coding schemes already exist.","On larger document benchmarks, the accuracy gain may shrink because CNNs already exploit layout and image statistics; the method's advantage is most plausible in small-data regimes like Tobacco-3482.","The color space could be tuned per dataset by learning the character-score table from a small validation set, potentially improving on the generic table used in the experiments."],"forward_implications":["Document classification pipelines that already have OCR can add a deterministic colorization step and gain accuracy without retraining multimodal networks or adding pretraining data.","On the Tobacco-3482 benchmark, WordVIS lets an image-based model beat larger multimodal models with substantially fewer parameters and less inference time (the paper reports 73.2% fewer parameters and 60% less inference time than DocXClassifier-XL).","Because the character-score table is language-agnostic, the method should extend to new languages without language-specific pretraining data, unlike transformer-based document models.","The ablation study shows gains across four different CNN architectures, indicating the benefit is not tied to one particular classifier."],"supporting_citations":[{"why":"Supplies the DocXClassifier-B base model whose 88.42% accuracy WordVIS raises to 91.14% on Tobacco-3482.","marker":"[25]"},{"why":"The previous state of the art on Tobacco-3482 (90.3%) that WordVIS surpasses.","marker":"[17]"},{"why":"Provides the EfficientNet and EfficientNet+BERT baselines compared in Table 1.","marker":"[15]"},{"why":"Provides the MobileNetV2+Text multimodal baseline that WordVIS outperforms.","marker":"[5]"},{"why":"Provides the AlexNet, GoogleNet, ResNet-50, and VGG-16 baselines on Tobacco-3482.","marker":"[2]"},{"why":"Chargrid, the prior character-level color-coding approach that WordVIS adapts from key information extraction to classification.","marker":"[19]"},{"why":"BERTGrid, the costly text-embedding-in-image scheme that WordVIS contrasts with on efficiency.","marker":"[12]"},{"why":"Defines the BERT embeddings that the multimodal baselines rely on and that WordVIS avoids needing.","marker":"[13]"}],"fun_headline_variants":["WordVIS: Color-coded words hit 91.14% without document pretraining","Turn text into color, then let a CNN classify documents at 91.14%","Painting words by score yields top accuracy on Tobacco-3482","Colorized letters boost image classifiers to a 91.14% record","WordVIS embeds text as colors, beating multimodal models on documents"],"cache_read_input_tokens":11264,"weakest_assumption_plain":"The paper assumes that its hand-chosen character-to-score and character-to-channel mapping is what produces the accuracy gains, rather than the mere fact that words are given distinct colors; no experiment tests a random or alternative color mapping.","fun_headline_variants_meta":{"raw":{"variants":["WordVIS: Color-coded words hit 91.14% without document pretraining","Turn text into color, then let a CNN classify documents at 91.14%","Painting words by score yields top accuracy on Tobacco-3482","Colorized letters boost image classifiers to a 91.14% record","WordVIS embeds text as colors, beating multimodal models on documents"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000244,"raw_usage":{"total_tokens":1511,"prompt_tokens":902,"completion_tokens":609,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":509}},"tokens_in":518,"tokens_out":609,"duration_ms":6578,"temperature":1.0,"reasoning_tokens":509,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:16:19.020929+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same DocXClassifier-B on Tobacco-3482 images whose words are colored by a random permutation of the WordVIS lookup table, or by a hash of each word to a random RGB value, keeping all other settings identical; if accuracy stays near 91.14%, then the specific semantic score assignment is not the cause of the improvement.","supporting_citations":[{"cited_title":"https://doi.org/10.36227/techrxiv.19310489.v2","cited_arxiv_id":null,"evidence_quote":"Supplies the DocXClassifier-B base model whose 88.42% accuracy WordVIS raises to 91.14% on Tobacco-3482."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the EfficientNet and EfficientNet+BERT baselines compared in Table 1."},{"cited_title":"Multimodal deep networks for text and image-based document classification","cited_arxiv_id":"1907.06370","evidence_quote":"Provides the MobileNetV2+Text multimodal baseline that WordVIS outperforms."},{"cited_title":"Cutting the Error by Half: Investigation of Very Deep CNN and Advanced Training Strategies for Document Image Classification","cited_arxiv_id":"1704.03557","evidence_quote":"Provides the AlexNet, GoogleNet, ResNet-50, and VGG-16 baselines on Tobacco-3482."},{"cited_title":"In: NAACL HLT 2019 - 2019 Conf","cited_arxiv_id":null,"evidence_quote":"Defines the BERT embeddings that the multimodal baselines rely on and that WordVIS avoids needing."}],"review_version":1}