{"id":"6403e2d7-4122-4960-b93e-1a6c2519372d","arxiv_id":"1908.08307","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A shallow Capsule Network with VGG-19 features and MSE loss colorizes grayscale images with mid-range PSNR/SSIM on DIV2K.","lead":"This paper adapts a type of neural network built for image classification, CapsNets, to colorize black-and-white photos. The result is a working colorizer that scores in the mid-range on a standard benchmark, suggesting CapsNets can be applied to image generation tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No non-capsule control is provided, and Figure 4b shows routing has no effect; the central claim that capsule mechanisms drive the colorization is unsubstantiated.","rationale":"The reader's weakest assumption is exactly that routing-by-agreement is doing work, and the paper's own Figure 4b provides evidence against it. I see the missing non-capsule control as the most load-bearing concern: without it, the central claim in the title and abstract--that colorization is achieved by a Capsule Network in a way that exploits capsule-specific segmentation--has not been demonstrated. The reported numbers could come from the VGG-19 front-end and the MSE-optimized decoder alone. This is an internal-consistency issue, not merely a disagreement with current consensus: Section 3.4 says routings are critical while Figure 4b shows no sensitivity. The evaluation-protocol concerns (fine-tuning on the validation set, Table 1's aggregate worst/best rather than named methods) are secondary but reinforce that the empirical support is weaker than the prose implies. The paper does offer a useful proof-of-concept with published loss curves and per-image examples, and the idea is not contradicted by the existing literature, so a reject is not warranted. The control experiment described above would settle the concern: if a matched convolutional decoder matches ColorCapsNet, then the CapsNet-specific contribution is nil and the claim must be substantially weakened; if it does not, the conditional acceptance can be upgraded. Because this is the same condition the reader already attached, the verdict remains CONDITIONAL.","tokens_in":8133,"tokens_out":7917,"duration_ms":84022,"concrete_test":"Train a control network with the same VGG-19 front-end, the same 9x9 patch preprocessing, the same latent dimension and decoder structure, and the same MSE loss, but replace the primary-capsule/capsule-layer/routing block with an ordinary convolutional bottleneck of matched parameter count and receptive field. Train it under the exact protocol of Section 4.1 except that the DIV2K validation set is held out and never used for fine-tuning, and evaluate both models on the held-out split and on the official NTIRE 2019 test set. If the control matches ColorCapsNet within roughly 0.1 dB PSNR, the capsule-specific mechanisms are not responsible for the result; if the control is clearly worse, the paper's central claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's title and Section 3.4 claim that ColorCapsNet 'exploits the idea of routing in order to segment objects internally.' For that claim to hold, the capsule-specific parts of the network must contribute to the reported PSNR/SSIM. No such evidence is given. Section 3.4 calls routing 'one of the most critical hyperparameters' but Figure 4b reports that varying the number of routing iterations has no effect on loss, and the authors select one routing iteration for speed. There is no ablation comparing ColorCapsNet with a plain convolutional decoder that uses the same VGG-19 front-end, latent dimensionality, and MSE loss. Table 1 compares only against aggregate 'worst' and 'best' entries from the NTIRE 2019 challenge, not against named baselines, so 'comparable' is not established. A further confound is that Section 4.1 fine-tunes the model on the DIV2K validation set before evaluating it on the test set, making the reported test numbers a product of an unstandardized protocol. Together these gaps mean the observed performance could be entirely due to the pretrained VGG-19 features and MSE training rather than to capsule routing or capsule representations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ColorCapsNet, an adaptation of the original CapsNet to the task of automatic image colorization. The architecture uses a VGG-19-based feature extractor, a primary capsule layer, and a capsule classification layer, with MSE loss in CIE Lab space. The method is trained on ILSVRC 2012 and DIV2K and evaluated on NTIRE 2019 colorization challenge data. The authors report a validation PSNR of 22.20 and SSIM of 0.88 after 50 epochs, and a test PSNR of 21.08 and SSIM of 0.85 after 10 epochs, and claim these results are promising and comparable.","tokens_in":8380,"tokens_out":4695,"duration_ms":40349,"significance":"The application of capsule networks to colorization is a novel and potentially interesting direction. The paper provides a clear description of the architecture and a comparative analysis of patch size, routing iterations, and topology choices. However, the central claim that capsule-specific mechanisms (routing-by-agreement and capsule representations) contribute to the colorization quality is not supported by the evidence: Figure 4b shows no effect of routing iterations, and no non-capsule control is reported. The evaluation protocol also confounds the test numbers by fine-tuning on the validation set. If the authors add a proper ablation and a standardized comparison, the work could be a useful contribution to the community. The manuscript is honest about the observed lack of routing sensitivity.","major_comments":[{"comment":"The test evaluation is confounded by fine-tuning on the validation set. Section 4.1 states that after training on the DIV2K train split, the network is \"finally trained on this validation set to colorize the test images,\" so the test numbers in Table 1 do not follow the standardized NTIRE evaluation protocol and are not directly comparable to the listed worst/best challenge values. Additionally, the validation and test rows come from different training durations (50 vs 10 epochs), so the two numbers should not be presented as a single model's performance.","section":"Section 4.1; Table 1"},{"comment":"The paper's central claim that routing-by-agreement contributes to colorization is contradicted by its own Figure 4b, which shows no effect of varying the number of routing iterations. No ablation is provided that replaces the capsule decoder with a plain convolutional decoder using the same VGG-19 front-end, latent dimensionality, patch size, and MSE loss. Without such a control, the reported PSNR/SSIM values cannot be attributed to capsule mechanisms; they may arise entirely from the pretrained VGG-19 features and the MSE objective. This is a load-bearing gap for the title and Section 3.4 claims.","section":"Section 3.4; Figure 4b"},{"comment":"The comparison to \"Worst\" and \"Best PSNR/SSIM\" from the NTIRE 2019 challenge is not a comparison to named baselines. Since the worst and best values may come from different participants for each metric, the claim that ColorCapsNet has \"comparable colorization performance\" is not quantitatively supported. The authors should cite specific competing methods from the NTIRE report and evaluate under the identical protocol, including the same training epochs and no validation-set fine-tuning.","section":"Table 1"}],"minor_comments":[{"comment":"There is a typo in Section 1: \"These studies may be classified as in two categories:\" should read \"These studies may be classified into two categories:\".","section":"Section 1"},{"comment":"Equation (6) is ambiguous: it writes \"1/YX\" while the summation bounds are over y=1..Y and x=1..X; this should be written as 1/(XY) or with an explicit double sum.","section":"Equation (6)"},{"comment":"The caption \"the train loss decreases with the patch size goes down exponentially\" is grammatically incomplete, and the figure does not use a logarithmic scale, so the word \"exponentially\" is not supported by the displayed data.","section":"Figure 4(c)"},{"comment":"The description of DIV2K says the train set has 794 grayscale and 800 RGB images, and then \"794 out of 800 images in the train set are used\"; the reason for excluding six images is not explained.","section":"Section 4.1"},{"comment":"The paper does not specify whether patches are sampled densely or randomly, nor the batch size used for training; these details are needed for reproduction.","section":"Section 3.1/3.2"},{"comment":"The footnote markers \"1\" and \"2\" after the ColorCapsNet numbers are not formatted as superscripts, which makes the table harder to read.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is an early-stage report whose central claim about capsule routing is not yet supported by experiments. I recommend major revision: the authors should provide a non-capsule control, compare with named baselines under a standardized protocol, and clarify the evaluation procedure. If they cannot show that capsule mechanisms matter, the paper would not meet the standards of a serious journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a simple application paper: it takes the original CapsNet, swaps the front-end for VGG-19's first two conv layers, changes margin loss to MSE, and trains it patch-wise for colorization on DIV2K. That's the whole novelty, and it's fine as a modest proof-of-concept. The paper does a couple of honest things: it shows in Figure 4b that the number of routing iterations doesn't affect the loss, then picks one iteration for speed; and it openly says the test model was fine-tuned on the DIV2K validation set. Both admissions are to its credit.\n\nThe soft spots are the ones the stress-test note names. There is no comparison to a non-capsule control. If you take the same VGG-19 features, a latent vector of the same size, and MSE loss, a plain convolutional decoder would likely do as well or better. Without that ablation, the title's implication—that capsule mechanisms drive the colorization—is unsubstantiated. The routing claim is especially shaky: Section 3.4 calls routing 'one of the most critical hyperparameters,' but Figure 4b says changing it does nothing. The paper doesn't resolve that tension. 'Comparable' also isn't established: Table 1 only gives the challenge's aggregate worst and best, not named baselines, and the validation-set fine-tuning confounds the test numbers. Code is not provided, so the numbers can't be checked.\n\nNone of this is fatal in the sense of the paper being nonsense. It's a preliminary empirical report. But the central claim is overreach: the evidence doesn't show ColorCapsNet's capsule-specific machinery is responsible for the observed PSNR/SSIM. A serious referee should ask for ablations against a convolutional decoder, named baselines or at least a proper held-out protocol, and a discussion of why routing appears to have no effect.\n\nI'd send this to peer review rather than desk-reject, because the question is legitimate and the paper is readable and honest in its reporting. The likely outcome is major revision or a workshop-level paper. My own verdict is skeptical: I wouldn't cite it for the capsule claim, but I'd maybe bring it up in a reading group as an example of an under-ablated application paper.","headline":"A modest CapsNet colorization adaptation that honestly reports routing has no effect, but never shows the capsule machinery matters.","tokens_in":8897,"tokens_out":2454,"would_cite":false,"duration_ms":25065,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A shallow capsule network, ColorCapsNet, adapted from the original CapsNet with VGG-19 features and MSE loss, produces automatic colorization results comparable to NTIRE 2019 challenge participants on DIV2K.","keywords":["image colorization","capsule network","routing-by-agreement","CIE Lab colorspace","VGG-19","transfer learning","DIV2K","PSNR"],"falsifier":"Train ColorCapsNet on DIV2K with routing iterations set to zero or with the capsule layer replaced by a plain convolutional layer of similar size, keeping VGG-19 features and MSE loss identical; if validation PSNR and SSIM remain at roughly 22.20 and 0.88, the central role claimed for routing is refuted.","tokens_in":7933,"feed_emoji":"🎨","tokens_out":4574,"duration_ms":41518,"temperature":0.7,"pith_summary":"This paper attempts to show that capsule networks, originally designed for classification, can be turned into automatic image colorizers. The proposed ColorCapsNet maps grayscale image patches to CIE Lab color patches using a shallow encoder-decoder capsule topology, VGG-19 pretrained features, batch normalization, and mean squared error loss. Evaluated on DIV2K, it reports validation PSNR of 22.20 and SSIM of 0.88 at 50 epochs, and test PSNR of 21.08 and SSIM of 0.85 at 10 epochs, placing it among the comparable entries in the NTIRE 2019 colorization challenge. The significance would be that capsule routing, not just convolutions, can support image-to-image generation tasks.","feed_headline":"Capsule network colorizes images with challenge-level scores","feed_subtitle":"A shallow CapsNet with VGG features and MSE loss hits PSNR 22.20 on DIV2K validation, on par with NTIRE 2019 entries.","key_machinery":"The central object is ColorCapsNet, an autoencoder-like capsule network in which a capsule is a group of neurons whose output vector encodes an entity's instantiation parameters. The encoder starts with the first two convolutional layers of VGG-19 (with transferred weights) followed by batch normalization, a primary capsule layer, and a capsule layer with six capsules. Dynamic routing-by-agreement transfers information between capsule layers and is intended to segment objects internally; the decoder then reconstructs the $a$ and $b$ chrominance channels from the capsule vectors, with the $L$ channel taken from the grayscale input. Patch size and routing iterations are treated as critical hyperparameters, with 9x9 patches and one routing iteration chosen.","core_discovery":"The paper's central claim is that the generative and segmentation capabilities of the original CapsNet carry over to colorization when the feature detector is replaced with the first two convolutional layers of VGG-19 (initialized with pretrained weights), batch normalization is added, the number of class capsules is reduced to six, and margin loss is exchanged for MSE in CIE Lab space. The resulting ColorCapsNet is patch-based and fully automatic, and it achieves the reported PSNR/SSIM values on DIV2K validation and test sets. The authors interpret this as evidence that capsule networks deserve further investigation for colorization.","pith_inferences":["The paper does not isolate routing's contribution: since changing routing iterations from 1 to 3 barely changed train loss, a plausible inference is that the VGG-19 features and the decoder, not the routing-by-agreement mechanism, drive most of the colorization quality.","A direct testable extension would replace the capsule decoder with a plain convolutional decoder of similar capacity; if PSNR/SSIM stay near 22.20/0.88, capsule routing is not load-bearing for this task.","The same architecture could be extended to multimodal colorization by predicting a per-pixel color distribution instead of a single Lab value, which would address the ill-posed ambiguity the paper acknowledges.","The observed invariance to routing iterations suggests that dynamic routing may matter more for classification-type tasks than for dense prediction; this is an inference beyond the paper's claims."],"forward_implications":["If ColorCapsNet's results hold, capsule networks are viable for image generation tasks beyond classification, not just for object recognition.","The reported scores imply that a shallow capsule architecture, combined with transferred VGG-19 features, can compete with more complex automatic colorization models on a standard benchmark.","Because the network is patch-based and trained in CIE Lab space, the method suggests that local color distribution learning with MSE loss is sufficient for plausible colorization in this setting.","The authors' observation that loss curves had not plateaued at 50 epochs implies that longer training on DIV2K may push the scores higher, and the paper explicitly leaves this as an open direction."],"supporting_citations":[{"why":"Supplies the original CapsNet topology, routing-by-agreement, and the generative decoder that ColorCapsNet adapts.","marker":"[14]"},{"why":"Provides the first two convolutional layers and pretrained weights used as the feature detector.","marker":"[15]"},{"why":"Defines the DIV2K dataset used for final training and for validation/test evaluation.","marker":"[1]"},{"why":"Defines the NTIRE 2019 colorization challenge whose validation and test sets provide the reported PSNR/SSIM comparison.","marker":"[6]"},{"why":"Justifies the use of MSE loss in CIE Lab space and represents the automatic colorization baseline approach.","marker":"[17]"}],"fun_headline_variants":["CapsNet colorizes via VGG features and MSE loss","Shallow CapsNet matches NTIRE colorization scores","ColorCapsNet: capsule network for image colorization","Capsule network turns grayscale to color with VGG boost","CapsNet colorization on par with challenge entries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that capsule routing-by-agreement, rather than the transferred VGG-19 features and the decoder, is what gives ColorCapsNet its colorization ability; the paper calls routing one of the most critical hyperparameters but offers no ablation that removes or replaces routing to prove that.","fun_headline_variants_meta":{"raw":{"variants":["CapsNet colorizes via VGG features and MSE loss","Shallow CapsNet matches NTIRE colorization scores","ColorCapsNet: capsule network for image colorization","Capsule network turns grayscale to color with VGG boost","CapsNet colorization on par with challenge entries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000146,"raw_usage":{"total_tokens":1134,"prompt_tokens":852,"completion_tokens":282,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":199}},"tokens_in":468,"tokens_out":282,"duration_ms":3086,"temperature":1.0,"reasoning_tokens":199,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:42:20.431682+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train ColorCapsNet on DIV2K with routing iterations set to zero or with the capsule layer replaced by a plain convolutional layer of similar size, keeping VGG-19 features and MSE loss identical; if validation PSNR and SSIM remain at roughly 22.20 and 0.88, the central role claimed for routing is refuted.","supporting_citations":[{"cited_title":"Dy- namic routing between capsules","cited_arxiv_id":null,"evidence_quote":"Supplies the original CapsNet topology, routing-by-agreement, and the generative decoder that ColorCapsNet adapts."},{"cited_title":"Ntire 2017 challenge on single image super-resolution: Dataset and study","cited_arxiv_id":null,"evidence_quote":"Defines the DIV2K dataset used for final training and for validation/test evaluation."},{"cited_title":"Ntire 2019 challenge on image colorization: Report","cited_arxiv_id":null,"evidence_quote":"Defines the NTIRE 2019 colorization challenge whose validation and test sets provide the reported PSNR/SSIM comparison."},{"cited_title":"Colorful image colorization","cited_arxiv_id":null,"evidence_quote":"Justifies the use of MSE loss in CIE Lab space and represents the automatic colorization baseline approach."}],"review_version":1}