{"id":"08b405e8-9e0e-44e7-9742-f7ac746e031a","arxiv_id":"1908.05168","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A hooking layer (LinearScope) freezes the nonlinear decisions of a CNN to expose the network as a single linear map, revealing bias-dominated classifier scores, wavelet-like super-resolution bases, and copy-move/template strategies in CycleGAN.","lead":"LinearScope lets a convolutional network run alongside a linearized copy in which all nonlinear decisions are frozen, so the whole network becomes one input-dependent linear map. The authors use it to inspect classifiers, super-resolution networks, and CycleGAN, reporting bias-heavy class decisions, wavelet-like bases, and copy-move strategies.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Wavelet-type basis claim rests on visual inspection of SVD modes, with no quantitative metrics, null models, or stability analysis.","rationale":"The reader's weakest assumption identifies the same core gap: the wavelet-type basis conclusion depends on visual similarity to Gabor patches rather than on quantitative measurement. My stress-test sharpens this concern in two ways. First, the visual claim is not merely under-quantified; it is made on SVD modes of a single input-dependent linear interpreter, and no null model or input-dependence analysis is provided. Second, the numerical SVD procedure itself can produce unstable modes when singular values are close, which is especially relevant for the high-frequency modes that the paper emphasizes. These issues do not undermine the LinearScope tool itself: the linear algebra behind Eq. (5), the transposed-probe trick in Eq. (6), and the residual decomposition in Theorem 1 from Ref. [28] are coherent, and the implementation effort is real. The concern is about the interpretation layer: the headline discoveries are currently selected visual examples rather than tested hypotheses. A concrete quantitative evaluation with null controls would settle whether the wavelet-type basis and copy-move/template claims are genuine network strategies. Since the reader already issued a CONDITIONAL verdict on exactly this basis, my read does not change the verdict; it reinforces it with a specific stability concern and a concrete validation protocol.","tokens_in":20412,"tokens_out":4208,"duration_ms":48416,"concrete_test":"Run Algorithm 1 SVD on EDSR 4x (and one CycleGAN model) for N=20 diverse validation images, and compute for each V/U mode: (i) spatial participation ratio as a localization measure, (ii) orientation selectivity from the 2D power spectrum (energy in a narrow angular wedge divided by total energy), and (iii) radial bandwidth (e.g., standard deviation of log-frequency). Compare the resulting distributions against the same metrics for (a) bicubic upsampling, (b) a matched-size Gaussian random matrix, and (c) frozen interpreters of the same architecture with random weights. If the metrics do not separate the trained networks from these nulls, and if the modes are not stable across inputs or across multiple SVD runs with different initializations, then the wavelet-type-basis claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central inference that SR and I2I networks use wavelet-type bases similar to the human visual system is supported only by visual inspection of SVD modes in Section 4 Case 2 (Figures 5 and 12) and Section 8.C (Figure 14). No quantitative definition of spatial localization, orientation selectivity, or bandpass behavior is given, and no null model is computed. This is load-bearing because the linear interpreter F(x0) in Eq. (5) is one frozen linear system for a single input; its SVD modes are not automatically stable across inputs, and with near-degenerate singular values the power method with deflation (Algorithm 1) can return arbitrary mixtures of modes. The high-frequency modes emphasized in the wavelet claim are precisely the least stable ones. Without comparing against null distributions (e.g., bicubic upsampling, random linear maps of matched size, or frozen interpreters of randomly initialized networks), Gabor-like appearance in a few selected images does not establish that the network actually uses such bases. The authors themselves acknowledge in Section 5 that the effect of these visualizations on human understanding is 'not direct' and requires future research; the paper's future-impact claims, such as using the basis as an interpretable target, depend on the basis being a stable, measurable network property rather than a visually plausible artifact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces LinearScope, a hooking layer that runs a network in two batches: one batch executes the original nonlinear network, and the second batch executes a linear interpreter of the form y1 = F(x0)x1 + r(x0) obtained by freezing the decisions of all nonlinear units for the input x0. The authors show how to compute the residual, columns, rows, transposed system, and SVD of F(x0) without materializing the matrix or storing activations, using an accelerated power method with deflation. They then apply this tool to classification networks (reporting residual-dominated scores and a 'pixel-wise vote' interpretation), to super-resolution models (reporting wavelet-type bases similar to the human visual system), and to CycleGAN image-to-image translation (reporting copy-move and template-creation strategies). The paper is explicitly exploratory and positions the LinearScope as a general instrument for such studies.","tokens_in":20586,"tokens_out":4407,"duration_ms":44347,"significance":"If the interpretive claims hold, the paper makes a useful methodological contribution: LinearScope is a simple, efficient extension of Deep Filter Visualization, the linear algebra for probing F(x0) is sound, and Theorem 1 is clear and independently published. The paper also ships concrete demonstrations and gives reproducible definitions for the residual and filter computations. However, the significance of the headline interpretive claims is currently limited by the lack of quantitative validation: the wavelet-type basis claim is supported only by visual similarity of selected SVD modes, the residual-dominance claim relies on an undefined contribution statistic, and the pixel-vote claim is anecdotal. These are not merely presentation issues; they are load-bearing for the paper's stated future-impact claims.","major_comments":[{"comment":"The claim that SR and I2I networks use 'wavelet-type basis similar to the human visual system' is not supported by the evidence presented. The paper never defines spatial localization, orientation selectivity, or bandpass behavior quantitatively, computes no null comparison (e.g., SVD modes of bicubic interpolation, random linear maps, or randomly initialized networks), and does not study stability of the modes across inputs x0. Because F(x0) is an input-dependent frozen linear system and Algorithm 1 with deflation can return arbitrary mixtures in the presence of near-degenerate singular values, the Gabor-like appearance in selected figures is a visual interpretation rather than a measured network property; this is load-bearing for the paper's central future-impact claims. The authors' candid statement in Section 5 that the effect of visualizations on human understanding is 'not direct' does not address this.","section":"§4 Case 2, §8.C (Figs. 5, 12, 14)"},{"comment":"The central classification claim that residuals contribute 'typically more than 80%' is not evaluable because the manuscript never defines the reported contribution statistic. If the quantity is r/(Fx+r) for each score, the percentage can exceed 100% or be negative due to cancellations; if it is an average over images or classes of some normalized quantity, the normalization must be specified. Equation (8) gives a decomposition of r but not a definition of 'contribution'; without this, the claimed dominance of residuals and the bias-dependence conclusion cannot be tested. Please add the exact formula and report individual-image distributions, not only means.","section":"§4 Case 1, Table 3"},{"comment":"Algorithm 1 is presented as the method for extracting the SVD basis, but no convergence analysis, stopping criterion, or chosen values for momentum m and step count S are given, and the algorithm is not validated against a directly materialized SVD even for a small model. The update vprev ← vcurr/||vnext|| also appears to be a typo, since vprev should be a normalized version of vcurr or some consistent momentum state. Given that the wavelet-type basis conclusion rests on these modes and that deflation is sensitive to clustered singular values, the paper should verify the computed singular values and vectors, report numerical error, and state the parameter settings.","section":"§3, Algorithm 1"},{"comment":"The 'pixel-wise vote' interpretation is based on qualitative inspection of selected images. The paper does not provide a quantitative measure of agreement among pixel votes, a comparison to chance-level voting, or a count of how often the top-1 label receives the most votes; without such evidence the statement that pixels 'clearly focus on objects' is an anecdotal observation rather than a demonstrated network strategy. Since this is one of the paper's headline discoveries, it needs an evaluable metric.","section":"§4 Case 1, Fig. 3"}],"minor_comments":[{"comment":"The abbreviation 'HSV' is used for the Human Visual System; this is incorrect and confusing because HSV commonly denotes hue-saturation-value. Please use 'HVS' throughout.","section":"§1, bullet list"},{"comment":"The captions assert that the basis is 'spatially localized, oriented and bandpass, comparable to wavelet basis'; this is the conclusion being argued, not a measured property. Please make the captions descriptive rather than interpretive, or add the quantitative evidence.","section":"§4 Case 2, Figs. 5 and 12 captions"},{"comment":"In the related-work paragraph, the citation list '[12, 35, 40, 27, 3, 11, 15, 12, 35, 40, 27, 3, 11, 15, 37]' contains duplicated entries; prune the repeated references.","section":"§2"},{"comment":"The linear interpreter for Sigmoid is described only as a 'continuous mask' in Figure 2; the precise choice of A(x0) and c(x0) for sigmoid (and for other non-ReLU units) should be stated explicitly in the text.","section":"§3, Eq. (2)"},{"comment":"The normalized layer-wise contributions shown in the histograms exceed 100% and can be negative; adding a sentence explaining that this is expected because contributions have opposite signs would help readers interpret the plots.","section":"§4, Fig. 8"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about the wavelet-type basis claim is valid and is the main barrier to acceptance. The methodological core is sound, but the interpretive claims consistently outrun the quantitative evidence. I would encourage the editor to invite a major revision rather than reject, because the LinearScope tool and the residual/column/row machinery are useful and can be strengthened with modest additional experiments (null models, stability checks, and explicit definitions). The reliance on the authors' prior work [28] is acceptable since that theorem is independently published and the assumptions are stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper in one line: it ships a genuinely useful engineering contribution and then wraps it in interpretive claims that go beyond what the evidence supports.\n\nWhat is actually new: LinearScope avoids storing activations by running a second batch through the network and hooking each nonlinear unit with a linear interpretation chosen from the first batch. That is a real improvement over the authors' earlier DFV work, and it lets them probe the frozen linear system without materializing F: residuals, columns, rows, a transposed interpreter, and an SVD via an accelerated power method. The linear algebra is sound. The residual-contribution statistic for classifiers is well-defined, and the layer-wise bias decomposition (Theorem 1 from their prior paper) is correctly applied. The pixel-vote idea is genuinely interesting, and the observation that residual bias dominates classifier scores is a concrete, reproducible measurement.\n\nThe soft spots are concentrated in the interpretation sections. The wavelet-like basis claim for SR and I2I rests almost entirely on visual similarity between SVD modes and Gabor patches in Figures 5, 12, and 14. There is no quantitative definition of spatial localization, orientation selectivity, or bandwidth, no comparison against null models like bicubic upsampling or random linear maps, and no stability analysis across inputs. That matters because the frozen linear system is per-input; near-degenerate singular values can produce arbitrary mixtures of modes, and the high-frequency modes emphasized in the wavelet claim are the least stable ones. The authors do acknowledge in Section 5 that the effect on human understanding is not direct, which is honest, but it does not fix the gap. The pixel-vote analysis also lacks a success rate or any quantitative evaluation; it is a curated set of examples. And no code for LinearScope itself is released, only precomputed filter images.\n\nNone of this sinks the tool. The residual, row/column, and transposed-probe analyses are solid and will be useful to people inspecting SR or I2I models. The paper is a tour, and tours are allowed to be qualitative. But the wavelet-type basis claim is load-bearing for the paper's broader impact, and it does not yet meet the standard the authors themselves set elsewhere.\n\nFor peer review: yes, send it. The method deserves referee time, and the manuscript is coherent. But the reviewers should push for quantitative metrics on the SVD modes, null-model comparisons, a stability check across inputs, and ideally code release. With those additions this could be a solid venue contribution; without them, it is a useful tool plus suggestive observations.","headline":"A real, efficient interpretability tool (LinearScope) that deserves referee time, but its headline claims about wavelet-like bases and pixel votes are qualitative and need quantitative backing.","tokens_in":21170,"tokens_out":1389,"would_cite":true,"duration_ms":16952,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CNN can be replayed as an input-dependent linear map, and probing that map reveals bias-dominated classifier scores, wavelet-like bases in super-resolution, and copy-move tricks in CycleGAN.","keywords":["convolutional networks","interpretability","linear interpreter","LinearScope","singular value decomposition","pixel voting","super-resolution","CycleGAN"],"falsifier":"Compare the SVD eigen-modes of EDSR against those of a bicubic upscaler and a random linear map with identical input and output dimensions, measuring spatial localization, orientation selectivity, and spectral bandwidth with numerical indices rather than visual inspection; if the CNN modes are statistically indistinguishable from the generic linear baselines, the wavelet-basis claim would be refuted, and if they separate cleanly, it would be supported.","tokens_in":20162,"feed_emoji":"🔬","tokens_out":9586,"duration_ms":89786,"temperature":0.7,"pith_summary":"The paper sets out to show that once a CNN has made its non-linear decisions on one image, the whole network behaves as an input-dependent affine map $y = F(x_0)x + r(x_0)$ that can be inspected without storing activations or materializing $F$. A hooking layer called LinearScope makes this practical: it runs the live network on one batch and a frozen linear interpreter on a second batch, and by probing with impulses, transposed gradients, and an SVD power method one can read off residuals, rows, columns, and singular modes. Applying these probes to popular classifiers, super-resolution networks, and CycleGAN, the paper reports that classification scores are dominated by bias residuals, that pixel-wise contributions act as independent votes which focus on objects, that SR and I2I networks use spatially localized, oriented, bandpass bases resembling wavelets, and that CycleGAN uses off-diagonal filters to copy textures and create templates. The reason to care is that this is one general instrument turning interpretability questions into concrete linear-algebra probes whose answers are directly viewable images.","feed_headline":"A single hook lets you replay a CNN as a linear map","feed_subtitle":"Freeze a network's decisions and probe the linear map to expose bias-driven votes, wavelet bases, copy-move tricks.","key_machinery":"The machinery is the LinearScope hook combined with the freezing interpretation of non-linearities. For a unit $y_0=h(x_0)$ the hook emits $[h(x_0), A(x_0)x_1+c(x_0)]$; for ReLU and Sigmoid $A$ is a mask, for max-pooling a selection matrix, and for instance normalization a fixed mean and scale, all chosen from the first batch and applied to the second. Because every non-linearity becomes linear once its decision is fixed, composing the hooks yields the whole-network interpreter $y_1=F(x_0)x_1+r(x_0)$. The remaining machinery is probing: the residual is $y_1(0)$, a column is $y_1(\\delta_k)-r$, the transposed map is obtained from the automatic-differentiation identity $F^T y_2 = \\nabla_{x_1}y_1 \\cdot y_2$, and an accelerated power method with deflation computes SVD eigen-inputs and eigen-outputs without materializing $F$. A theorem for sequential networks decomposes the residual into forward-projected masked biases, which is what makes layer-wise contribution histograms and pixel votes possible.","core_discovery":"The paper's central claim is that a convolutional network run on one input can be replayed, for that same input, as a single affine map $y = F(x_0)x + r(x_0)$, provided each non-linear unit's action is frozen according to the decisions it made on $x_0$: ReLU becomes a binary mask, max-pooling becomes a selection, and instance normalization becomes a fixed mean and scale. A hooking layer, LinearScope, returns $[h(x_0), A(x_0)x_1+c(x_0)]$ for a second batch, so the interpreter runs in parallel without storing activations. Using probe inputs $x_1=0$ and $x_1=\\delta_k$, the gradient identity $F^T y_2 = \\nabla_{x_1}y_1 \\cdot y_2$, and a deflated power method, the paper obtains residuals, rows, columns, transposed maps, and SVD modes of $F(x_0)$ without ever forming the matrix. On this basis it reports that ImageNet classifiers derive more than 80% of their scores from the bias residual, that back-projected pixel contributions behave like independent per-pixel votes that focus on objects, that SR and I2I networks have SVD bases that are spatially localized, oriented, and bandpass like wavelets, and that CycleGAN uses off-diagonal filter entries to copy textures from fixed regions and to create output templates such as window boxes.","pith_inferences":["A quantitative Gabor-fit extension could settle the wavelet claim: fit orientation, spatial frequency, and bandwidth to each eigen-mode and compare distributions across learned CNNs, bicubic upscaling, and random linear maps.","Because the interpreter is differentiable in $x_0$, one could add a loss term that shapes $F(x_0)$ or $r(x_0)$ toward a desired structure, making interpretability an optimization target rather than a post-hoc analysis.","The CycleGAN off-diagonal patterns could be used as an audit tool: by inspecting columns, practitioners can detect when a generative model copies textures from a few fixed source regions, which may indicate dataset shortcuts rather than true style transfer.","The residual-dominance result suggests that explanations limited to input-gradient heatmaps systematically miss the main driving term in classifier scores, since the bias contribution typically exceeds 80%."],"forward_implications":["LinearScope extends the earlier freezing approach to networks too large to store activations for, making row, column, transpose, and SVD probes feasible on models like EDSR and CycleGAN.","For classifiers, the decomposition into $F(x_0)x_0$ plus forward-projected masked biases locates where scores are built; batch normalization shifts these contributions deeper, and adversarial attacks spread pixel votes across all classes.","For super-resolution, SVD modes show that better models such as EDSR concentrate high-singular-value responses on recognizable high-level parts of the image, while a simpler PixelShuffle model does not.","For CycleGAN, off-diagonal rows and columns directly document a copy-move mechanism for textures and a template-creation mechanism for facades, explaining how new objects and textures are generated.","Residuals are small in SR but larger in I2I, so the interpreter can distinguish which part of an output is a fixed shift versus input-driven filtering."],"supporting_citations":[{"why":"Supplies the freeze-the-decisions interpretation of non-linearities and the affine form $y=Fx+r$ that LinearScope extends.","marker":"[28]"},{"why":"Supplies the EDSR model whose SVD modes and projective filters anchor the super-resolution and wavelet-basis conclusions.","marker":"[22]"},{"why":"Supplies the CycleGAN models whose off-diagonal filter entries reveal copy-move and template-creation strategies.","marker":"[51]"},{"why":"Supplies the VGG classifiers used for residual-contribution measurements, layer-wise histograms, and pixel votes.","marker":"[41]"},{"why":"Provides the sparse-coding link between simple-cell receptive fields and wavelet bases that the SVD comparison uses as reference.","marker":"[31]"},{"why":"Introduces layer-wise relevance propagation, the pixel-wise decomposition baseline against which the paper's pixel discussions are contrasted.","marker":"[2]"},{"why":"Provides the FGSM attack that the paper uses to show how adversarial noise changes layer contributions and spreads pixel votes.","marker":"[19]"},{"why":"Provides the ImageNet validation images and benchmark used for all classifier measurements.","marker":"[38]"}],"fun_headline_variants":["Freeze CNN non-linearities to replay it as a linear map","One hook exposes CNN's bias votes, wavelets, copy-move","Replay a CNN as a linear system using a LinearScope hook","CNN's per-input linear view: bias-dominated votes, wavelet bases","See CNN's linear skeleton: bias scores, wavelet SVD, copy-move"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that SR and I2I networks use a wavelet-type basis rests on visual similarity between SVD eigen-images and Gabor patches, with no quantitative orientation or bandwidth measurement and no comparison against generic linear upsamplers, so the human-visual-system conclusion is an interpretation of figures rather than a measured property.","fun_headline_variants_meta":{"raw":{"variants":["Freeze CNN non-linearities to replay it as a linear map","One hook exposes CNN's bias votes, wavelets, copy-move","Replay a CNN as a linear system using a LinearScope hook","CNN's per-input linear view: bias-dominated votes, wavelet bases","See CNN's linear skeleton: bias scores, wavelet SVD, copy-move"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000758,"raw_usage":{"total_tokens":3454,"prompt_tokens":1119,"completion_tokens":2335,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":735,"completion_tokens_details":{"reasoning_tokens":2222}},"tokens_in":735,"tokens_out":2335,"duration_ms":17808,"temperature":1.0,"reasoning_tokens":2222,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:21:29.868415+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the SVD eigen-modes of EDSR against those of a bicubic upscaler and a random linear map with identical input and output dimensions, measuring spatial localization, orientation selectivity, and spectral bandwidth with numerical indices rather than visual inspection; if the CNN modes are statistically indistinguishable from the generic linear baselines, the wavelet-basis claim would be refuted, and if they separate cleanly, it would be supported.","supporting_citations":[{"cited_title":"Multigrid Backprojection Super-Resolution and Deep Filter Visualization","cited_arxiv_id":"1809.09326","evidence_quote":"Supplies the freeze-the-decisions interpretation of non-linearities and the affine form $y=Fx+r$ that LinearScope extends."},{"cited_title":"Enhanced deep residual networks for single image super–resolution","cited_arxiv_id":null,"evidence_quote":"Supplies the EDSR model whose SVD modes and projective filters anchor the super-resolution and wavelet-basis conclusions."},{"cited_title":"Emergence of simple–cell receptive field properties by learning a sparse code for natural images","cited_arxiv_id":null,"evidence_quote":"Provides the sparse-coding link between simple-cell receptive fields and wavelet bases that the SVD comparison uses as reference."},{"cited_title":"On pixel-wise explanations for non-linear classi- fier decisions by layer-wise relevance propagation.PloS one, 10(7):e0130140, 2015","cited_arxiv_id":null,"evidence_quote":"Introduces layer-wise relevance propagation, the pixel-wise decomposition baseline against which the paper's pixel discussions are contrasted."},{"cited_title":"Explaining and harnessing adversarial examples","cited_arxiv_id":null,"evidence_quote":"Provides the FGSM attack that the paper uses to show how adversarial noise changes layer contributions and spreads pixel votes."},{"cited_title":"Berg, and Li Fei-Fei","cited_arxiv_id":null,"evidence_quote":"Provides the ImageNet validation images and benchmark used for all classifier measurements."}],"review_version":1}