{"id":"e2894042-728e-46fa-94c1-08b65ed7ecf9","arxiv_id":"1908.08584","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A semantic segmentation method that reuses a VGG-16 encoder in reverse as its decoder via weight permutation reports higher Cityscapes IoU with fewer parameters than three VGG-based baselines.","lead":"The authors propose a semantic segmentation network that uses the encoder's own convolutional layers, with transposed weight matrices, as the decoder, removing the need for separate decoding layers. On the Cityscapes benchmark, their VGG-16-based model reports higher accuracy than SegNet, DeepLab LargeFOV, and FCN-8s while using fewer parameters.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Uncontrolled baselines and a missing tied-vs-untied ablation leave the central empirical claim unverified; the permuted-weight decoder's specific contribution is not isolated.","rationale":"I chose the missing tied-versus-untied ablation as the load-bearing point because the paper's headline is an empirical claim ('achieves higher accuracy ... while using fewer parameters'), and Table 1 is not a controlled comparison. The comparison numbers come from different papers with different training protocols and data sizes, and the margins over DeepLab LargeFOV and FCN-8s are small enough to be affected by single-run noise. Most importantly, the backward pass is a full extra processing path with skip connections and separate BNs; without a control that removes only the permuted weight tying, the proposed mechanism is not identified as the cause. A standard untied decoder of the same parameter and FLOP budget is the natural control. This does not contradict the reader's conditional verdict; it specifies the experiment needed to move from conditional to accept.","tokens_in":8659,"tokens_out":12176,"duration_ms":125512,"concrete_test":"Implement the Section 4 architecture and training protocol (same data split, loss, optimizer, augmentation, no coarse data, early stopping) and train a control model with the same encoder and a decoder of equal parameter count and effective depth, but with untied, randomly initialized decoder weights replacing the backward permutation path. Run both with at least 3 seeds. If the untied control reaches or exceeds VGG-PWN's 67.3 IoU class, the feedbackward weight permutation is not the operative cause; if it is clearly below, the assumption is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 1 is the only evidence for the central claim, but Section 5 compares VGG-PWN against numbers taken from 'frequently cited papers' rather than re-running SegNet, DeepLab LargeFOV, and FCN-8s under the Section 4 protocol. Section 6 even notes DeepLab used 19,998 extra coarse-annotation samples, so the comparison mixes training-data size, loss, augmentation, input resolution, and evaluation split with architecture. The reported margins over DeepLab and FCN are only about 2-3 IoU points, and VGG-PWN is a single early-stopped run with no seeds or error bars, so run-to-run variance could erase those margins. Compounding this, no ablation isolates the 'encoder-as-decoder' mechanism: the backward path includes skip additions, extra BN layers, and 27 effective convolutional passes, so a conventional decoder of the same parameter/computation budget might achieve the same result without reusing the encoder weights through the Section 3 permutation. The paper's weakest assumption—that permuted pretrained filters are a useful decoding projection—is therefore never directly tested.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes 'feedbackward decoding' for semantic segmentation: instead of attaching a separate decoder, the encoder's own convolutional layers are used in reverse, with weight tensors derived by permuting the input and output channel dimensions of the original convolution kernels. The authors adapt VGG-16 by keeping its 13 convolutional layers, adding one classification layer, and adding two batch-normalization operations per layer, one for each pass direction. On the Cityscapes benchmark they report that their best variant, VGG-PWN, reaches 67.3 IoU class while using 14.7M parameters, outperforming published scores for SegNet, DeepLab LargeFOV, and FCN-8s with fewer parameters. The paper argues that this demonstrates the encoder itself can serve as the decoder, and that the permutation operation preserves useful filter structure.","tokens_in":8859,"tokens_out":3651,"duration_ms":39854,"significance":"If the empirical claims hold, the idea is significant: it offers a way to construct a decoder without training additional decoding layers, reducing parameter count and potentially improving generalization by reusing pretrained features. The mathematical observation in Section 3, that a convolution weight tensor can be transposed by swapping channel dimensions to reverse the direction of a convolution, is correct and is a clean, useful conceptual contribution. The paper also makes a falsifiable quantitative claim about Cityscapes. However, the experimental support is currently not strong enough to substantiate the central claim: the comparisons are against published baseline numbers rather than re-run baselines, the contribution of the permutation mechanism is not isolated by ablations, and all results come from a single training run. The promise of public code is not fulfilled in the manuscript, further limiting reproducibility.","major_comments":[{"comment":"The central experimental claim rests entirely on comparisons with scores taken from previously published papers for SegNet, DeepLab LargeFOV, and FCN-8s, rather than on re-running those models under the protocol described in Section 4. Section 6 concedes that DeepLab LargeFOV was trained with 19,998 additional weakly labeled samples, while VGG-PWN uses only the 2,975 fully labeled Cityscapes training images. Because training data, loss function, augmentation, input resolution, and evaluation protocol all differ, the reported margins of roughly 2-3 IoU class points in Table 1 cannot be attributed to feedbackward decoding. The authors should re-run the baseline models under identical conditions or substantially weaken the comparative claims.","section":"§5, Table 1"},{"comment":"No ablation isolates the permuted-weight encoder-as-decoder mechanism. The backward pass includes skip additions, two additional batch-normalization operations per layer, and 27 effective convolutional passes, so a conventional decoder with the same parameter and computation budget could plausibly produce the same or better results without reusing the encoder weights through the Section 3 permutation. Please include controls such as an untied decoder trained from scratch, a decoder with the same topology but without weight reuse, and a tied decoder using nontransposed or randomly projected weights.","section":"§4, Figures 1-2 and §6"},{"comment":"All reported numbers come from a single training run with early stopping on the validation set, and no seeds or error bars are reported. Given that the advantage over DeepLab LargeFOV and FCN-8s is only 1.9-2.5 IoU class points, run-to-run variance could erase those margins. The authors should report mean and standard deviation over multiple seeds, or at least two complete runs, and state the precise early-stopping rule.","section":"§4 and §5"},{"comment":"The derivation in Section 3 establishes dimensional compatibility of the permuted weight tensor W', but it does not explain why pretrained filters should remain useful as decoding projections after permutation. This is the core assumption of the method, and it is currently supported only by the unablated experiments. A direct test, such as comparing against a randomly initialized or learned decoder of the same topology, or an analytical argument about filter structure preservation, would make the claim load-bearing rather than purely empirical.","section":"§3"}],"minor_comments":[{"comment":"The statement 'No other layers are added' is misleading because each convolutional layer receives two additional batch-normalization operations, one for the forward pass and one for the backward pass; please clarify that no additional decoder layers are added, rather than no layers at all.","section":"§4"},{"comment":"In the paragraph defining W', the text writes W' ∈ R^{h×w×c2×c1}, but the original weight tensor was defined as W ∈ R^{m×n×c1×c2}; the spatial dimensions should be m and n, not h and w.","section":"§3"},{"comment":"Training details are incomplete: the number of epochs, the exact early-stopping criterion, batch-normalization momentum and epsilon, whether batch-normalization statistics are computed separately for the forward and backward passes, and the precise handling of pooling indices in the backward pass are not reported.","section":"§4"},{"comment":"The difference between VGG-PWP and VGG-PWN is not visually evident from the figures, and the captions do not explain the permutation policy; please annotate the figures or add a table that clearly marks which layers permute their weights in the backward pass.","section":"Figures 1 and 2"},{"comment":"The abstract and conclusion both promise 'Our code will be publicly available later,' but no code or repository link is provided; please include an actual link or state the release plan in the final version, since the experiments are otherwise difficult to reproduce.","section":"Abstract and §7"},{"comment":"The rows for SegNet, DeepLab LargeFOV, and FCN-8s should cite the specific source papers and, if available, the exact evaluation configuration (e.g., training set size and post-processing) so that readers can assess comparability.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is interesting and the weight-permutation derivation is sound, but the empirical evaluation is not currently adequate for a journal-level claim of state-of-the-art parameter efficiency. The authors should be required to provide code, re-run baseline methods under the same protocol, and add ablations that isolate the feedbackward mechanism. If they cannot re-run the baselines, the claims should be explicitly reframed as comparisons with published numbers rather than controlled experimental improvements."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is neat and the parameter savings are real, but the empirical claims are not supported by the experiments as reported.\n\nThe paper proposes using the encoder's conv layers in reverse as the decoder by swapping the input/output channel dimensions of each weight tensor. That is a simple, original trick, and the architecture does cut parameters sharply: 14.7M vs SegNet's 29.4M. Also to their credit, they explicitly say they did not use the 19,998 coarse-annotated images that DeepLab LargeFOV had, so the comparison is at least directionally favorable to them.\n\nThe math of the permutation itself is straightforward: W' is just W with last two dims swapped. But the paper overstates it by calling this 'reversing' the layer's effect. For a convolution, the actual adjoint/transpose requires flipping the kernel spatially; permuting channels gives a different operation. That may be fine in practice, but the conceptual justification needs more care.\n\nThe bigger problem is the experiments. Table 1 compares against SegNet, DeepLab, and FCN-8s scores taken from the literature, not rerun under the same training protocol, input size, loss, or evaluation setup. Single run, no seeds, no error bars. No code is released. The margins over DeepLab and FCN are only a few IoU points, so run-to-run variance could flip them. There is also no ablation that isolates the reusing-encoder-weights mechanism. The backward path includes skip connections, extra BN layers, and 27 effective passes, so a standard decoder with the same parameter/compute budget might do just as well.\n\nFor someone working on compact segmentation this is an idea worth discussing, and it deserves a serious referee. But the paper needs re-run baselines, multiple seeds, and a tied-vs-untied ablation before the performance claims can be believed.\n\nRecommendation: send to peer review, but expect major revision.","headline":"Neat parameter-reuse idea, but the experimental support is too uncontrolled to back the performance claims.","tokens_in":9337,"tokens_out":4065,"would_cite":false,"duration_ms":38917,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes feedbackward decoding: after a convolutional encoder processes an image forward, the same network runs backward to recover spatial resolution, with the encoder's weights reused as the decoder's weights.","keywords":["semantic segmentation","feedbackward decoding","weight transpose","encoder-decoder architecture","Cityscapes benchmark","parameter efficiency","VGG-16","fully convolutional network"],"falsifier":"Hold the training pipeline fixed and replace the transposed encoder weights in the backward pass with randomly initialized weights of identical shape; if the random decoder matches or beats the transposed-weight decoder on Cityscapes IoU class, then the claimed benefit of reusing encoder weights is refuted. A more direct check is per-layer round-trip reconstruction: encoding a batch of images and decoding with the transposed weights should reconstruct layer inputs far better than decoding with random weights, and that margin should grow with the final segmentation score.","tokens_in":8509,"feed_emoji":"🔁","tokens_out":10587,"duration_ms":93240,"temperature":0.7,"pith_summary":"The paper proposes feedbackward decoding, a way to do semantic segmentation with no dedicated decoder: after the encoder processes the image forward, the same convolutional layers are run backward to restore spatial detail, reusing the encoder's weights. The key operation is to transpose each convolution's weight tensor by swapping the input-channel and output-channel dimensions, turning an encoder layer into a decoder layer without adding parameters. On the Cityscapes benchmark, a network built from just the 13 convolutional layers of VGG-16 plus one tiny classifier reports an IoU-class score of 67.3, higher than SegNet, DeepLab LargeFOV, and FCN-8s while using 50.0%, 28.3%, and 89.1% fewer parameters, respectively. If the claim holds, the information needed to reconstruct pixel-level detail is already present in trained encoder filters, and decoders do not need their own learned weights.","feed_headline":"No decoder needed: one encoder decodes, beating three baselines","feed_subtitle":"Feedbackward decoding reuses VGG-16's own filters backward, raising Cityscapes IoU class with up to 89.1% fewer parameters.","key_machinery":"The carrying object is the transposed weight tensor of a convolutional layer, formed by swapping the input-channel and output-channel dimensions of $W$ rather than by transposing spatial dimensions as in transposed convolution. This gives $W'$ with the same size and the same filters, preserving the inner structure of the original filters while changing which channel grouping they feed. In the forward pass each column of the filter matrix produces one output channel; in the backward pass each former row becomes a column, so the layer can expand or contract channel counts in the opposite direction. The network's effective depth is then counted by how many times information passes through a convolutional layer, which is why the paper describes its 14-layer model as having comparable capacity to the 26-convolutional-layer SegNet.","core_discovery":"The central claim is that a convolutional layer is inherently bidirectional. Because the weight tensor for mapping $c_1$ input channels to $c_2$ output channels has exactly the same size as the tensor for the reverse mapping, $W' \\in \\mathbb{R}^{m\\times n\\times c_2\\times c_1}$ can be obtained from $W \\in \\mathbb{R}^{m\\times n\\times c_1\\times c_2}$ by permuting the channel dimensions. Grouping filters into columns in the forward pass becomes grouping them into rows in the backward pass, so the original trained filters are kept intact while their role is reversed. The paper implements this on VGG-16, using nearest-neighbor interpolation where max pooling occurred, summing forward and backward feature maps at matching depths, and adding one classification layer. Its best variant, VGG-PWN, reaches 67.3 IoU class on Cityscapes, the highest among the compared VGG-16-based models, with only 14.7M parameters.","pith_inferences":["If transposed encoder weights genuinely act as an approximate inverse projection, then encouraging orthogonality or spectral balance in encoder filters should improve feedbackward decoding; this is testable by regularizing layer weights and measuring segmentation accuracy.","The method predicts that the round-trip reconstruction error of a forward-backward pass should be much smaller for trained encoder weights than for random weights of the same shape, and that this gap should correlate with final segmentation accuracy.","The same weight-reuse trick could be tried on other dense prediction tasks, such as depth estimation or edge detection, where an encoder must recover spatial detail from high-level features without a separately trained decoder.","The PWN-versus-PWP comparison implies that arbitrary permutations can break the team structure of filters; a further extension would be to learn which channel permutations to apply in the backward pass rather than choosing them only when necessary."],"forward_implications":["Semantic segmentation can be built as an encoder plus one tiny classification layer, with no upsampling decoder that has its own parameters.","Reusing encoder weights in the backward pass roughly halves the parameter count relative to a symmetric encoder-decoder such as SegNet, at comparable effective depth.","Decoder weights no longer have to be trained from scratch, so training can work with many fewer labeled images; the paper's model used only the 2,975 fully labeled Cityscapes training images, while DeepLab LargeFOV used 19,998 extra weakly labeled samples.","Permuting weights only when the channel dimension changes, as in the PWN variant, gives better results than permuting at every layer as in PWP, indicating that preserving existing filter groups matters for the backward projection.","Because the encoder's structure is unchanged, the method can be applied to other base networks rather than only VGG-16."],"supporting_citations":[{"why":"Defines SegNet, the symmetric VGG-16 encoder-decoder baseline whose parameter count and IoU scores feedbackward decoding is compared against.","marker":"Badrinarayanan et al., 2015"},{"why":"Introduces fully convolutional networks and the transposed-convolution decoding approach whose separately learned weights feedbackward decoding avoids.","marker":"Long et al., 2014"},{"why":"Defines DeepLab with atrous convolution and fully connected CRFs, the LargeFOV baseline and one of the three comparison models.","marker":"Chen et al., 2016"},{"why":"Provides the Cityscapes dataset and the four IoU metrics on which all reported results are computed.","marker":"Cordts et al., 2016"},{"why":"Supplies the VGG-16 architecture whose 13 convolutional layers are reused in both the forward and backward passes.","marker":"Simonyan & Zisserman, 2014"},{"why":"Defines the intersection-over-union loss that the training process minimizes to directly approximate the Cityscapes evaluation metric.","marker":"Rahman & Wang, 2016"}],"fun_headline_variants":["Reverse the encoder to decode: no separate decoder needed","Same VGG-16 weights, backward pass beats three baselines","Save up to 89% parameters: encode forward, decode backward","Bidirectional conv filters: one network for encoding and decoding","Feedbackward decoding: reuses encoder weights, cuts parameters"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a convolution's weight tensor, after swapping its input and output channels, provides a genuinely useful decoding projection: that the same filters which extract semantic features forward can meaningfully map those features back toward spatial detail.","fun_headline_variants_meta":{"raw":{"variants":["Reverse the encoder to decode: no separate decoder needed","Same VGG-16 weights, backward pass beats three baselines","Save up to 89% parameters: encode forward, decode backward","Bidirectional conv filters: one network for encoding and decoding","Feedbackward decoding: reuses encoder weights, cuts parameters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000275,"raw_usage":{"total_tokens":1683,"prompt_tokens":1028,"completion_tokens":655,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":644,"completion_tokens_details":{"reasoning_tokens":571}},"tokens_in":644,"tokens_out":655,"duration_ms":6686,"temperature":1.0,"reasoning_tokens":571,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:34:38.546211+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold the training pipeline fixed and replace the transposed encoder weights in the backward pass with randomly initialized weights of identical shape; if the random decoder matches or beats the transposed-weight decoder on Cityscapes IoU class, then the claimed benefit of reusing encoder weights is refuted. A more direct check is per-layer round-trip reconstruction: encoding a batch of images and decoding with the transposed weights should reconstruct layer inputs far better than decoding with random weights, and that margin should grow with the final segmentation score.","supporting_citations":[{"cited_title":"Optimizing intersection-over-union in deep neural networks for image segmentation","cited_arxiv_id":null,"evidence_quote":"Defines the intersection-over-union loss that the training process minimizes to directly approximate the Cityscapes evaluation metric."}],"review_version":1}