{"id":"f506ebd5-4306-48f8-aacf-46f4a3910bfa","arxiv_id":"2607.06918","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":4,"one_line_summary":"Decoupling spatial basis refinement from low-rank channel mixing in convolutional layers yields better parameter-efficient fine-tuning for vision foundation models.","lead":"This paper introduces LoCA, a method for efficiently fine-tuning the convolutional layers of large vision models by separating spatial pattern updates from cross-channel mixing. It matters because convolutional layers are ubiquitous in modern vision architectures, and this approach improves adaptation quality while using fewer trainable parameters than standard low-rank methods.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Ablation Table E does not isolate decoupling from module placement; channel mixing path (Eq 4) is mathematically identical to naive LoRA (Eq 2), yet shows +1.21 gain.","rationale":"The reader identified the SVD basis utility for 1×1 kernels as the weakest assumption. This is a valid limitation but is acknowledged by the authors (Sec H) and doesn't undermine the central empirical claim since ConvNeXt's 7×7 depthwise convolutions are where LoCA's spatial basis has the most room to help. My concern is more about the causal attribution: the ablation doesn't cleanly isolate whether the gains come from the decoupling/SVD design or from confounded engineering choices (module placement borrowed from Conv-Adapter, initialization). The channel mixing path being mathematically identical to naive LoRA (Eq 2 vs Eq 4) makes the +1.21 gain in Table E unexplained by the paper's own logic. However, this concern affects the mechanistic explanation, not the empirical performance claim. The final LoCA method does achieve strong results across multiple benchmarks (Tables A-D), and the DGSS comparison is relatively fair in parameter count (LoCA 3.4M vs LoRA-Linear 2.9M on ConvNeXt-B). The concern is about interpretability of contributions rather than validity of results. Since the reader's verdict is ACCEPT based on empirical performance and reasonable novelty, and my concern doesn't invalidate the empirical results, the verdict should remain UNCHANGED. The paper would benefit from a cleaner ablation that controls for module placement, but the overall contribution is solid.","tokens_in":25646,"tokens_out":5051,"duration_ms":200371,"concrete_test":"Re-run the Table E ablation with a control: replace LoCA's channel mixing path with standard LoRA (identical formulation, Eq 2) using the SAME module placement, initialization, and rank allocation as LoCA's channel path, then add a randomly-initialized depthwise diagonal LoRA (same parameter count as the spatial basis path, no SVD initialization). If this control matches or exceeds the 'Spatial Basis' row (67.03), the SVD-based spatial basis refinement is not the load-bearing component, and the gains come from placement + depthwise adaptation rather than spatial prior preservation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central mechanistic claim is that decoupling channel and spatial adaptation resolves spatial-channel entanglement. However, the channel adaptation path (Eq 4: ΔW♭_ch = (α/r_ch) B_c A_c, with B_c ∈ R^{C_out×r_ch}, A_c ∈ R^{r_ch×(C_in·k_h·k_w)}) is mathematically identical to naive flattened LoRA (Eq 2: W'♭ = W_0♭ + (α/r) BA, same dimensions). The ablation in Table E shows 'Channel Mixing' gaining +1.21 over 'LoRA Convolution' (64.95→66.16), but since these formulations are the same, the gain must come from confounded factors—likely module placement (the paper states it follows Conv-Adapter [4] placement strategies) or initialization—not from the decoupling design itself. This means the paper cannot cleanly attribute its performance gains to the spatial-channel decoupling that is its core contribution. The spatial basis term (Eq 5-7) adds a depthwise diagonal correction, which is essentially a structured depthwise adaptation; whether the SVD-based basis structure matters versus a simpler depthwise LoRA is only partially tested (Table F shows small differences: 66.5 vs 66.2-66.3 on DGSS). If the gains primarily come from module placement and depthwise adaptation rather than SVD-based spatial priors, the novelty claim is weakened, though the empirical performance claim stands.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"The paper proposes Low-Rank Convolutional Adaptation (LoCA), a Parameter-Efficient Fine-Tuning (PEFT) method designed specifically for convolutional layers in Vision Foundation Models (VFMs). Standard LoRA flattens 4D convolutional kernels into 2D matrices, which entangles spatial and channel information. To resolve this, LoCA decouples the adaptation into two paths: (1) a low-rank channel adaptation for dense cross-channel mixing, and (2) an SVD-based spatial basis refinement that operates on the depthwise diagonal. The method also introduces a hierarchical rank scheduling strategy that allocates adaptation capacity based on the stage-specific channel width. The authors evaluate LoCA on fine-grained classification (VTAB-1k, FGVC), domain-generalized semantic segmentation (DGSS), and subject-driven text-to-image generation (DreamBooth), demonstrating competitive or state-of-the-art performance across various convolutional backbones such as ConvNeXt, ResNet, and MambaVision.","tokens_in":25960,"tokens_out":1193,"duration_ms":122941,"significance":"Adapting convolutional layers efficiently is a relevant problem, as convolutions remain fundamental in modern hybrid VFM backbones. The paper provides a clear mathematical formulation for decoupling channel and spatial adaptation and provides reproducible code. The empirical evaluation is broad, spanning classification, segmentation, and generation. The mechanistic analysis of the Effective Receptive Field (ERF) and singular value evolution provides useful insights into how structural preservation aids adaptation. However, the central mechanistic claim regarding the benefits of spatial-channel decoupling is somewhat undermined by confounding factors in the ablation studies, which prevents the significance from being fully realized without further clarification.","major_comments":[{"comment":"The ablation in Table E does not cleanly isolate the contribution of the decoupled design from module placement and initialization. The 'Channel Mixing' variant (Eq. 4) is mathematically identical to naive flattened convolutional LoRA (Eq. 2), as both use B ∈ R^{C_out×r} and A ∈ R^{r×(C_in·k_h·k_w)}. However, Table E shows 'Channel Mixing' achieving +1.21 over 'LoRA Convolution' (64.95 → 66.16). Since the formulations are the same, this gain must stem from confounded factors, likely module placement (as the paper mentions following Conv-Adapter [4] placement strategies) or initialization. The authors must clarify exactly what differs between the 'LoRA Convolution' and 'Channel Mixing' rows in Table E to validate the claim that the decoupled design itself drives the improvement.","section":null},{"comment":"The paper claims that the SVD-based spatial basis is crucial for preserving pre-trained spatial priors. However, Table F shows that the 'Covariance SVD' initialization yields only marginal gains over simpler initializations (e.g., 66.5 vs 66.2-66.3 on DGSS). Furthermore, the spatial basis term (Eq. 5-7) essentially acts as a structured depthwise adaptation. The paper does not directly compare the SVD-based spatial basis against a simpler depthwise LoRA baseline (without the SVD initialization). To support the claim that the SVD-based structure is load-bearing, the authors should provide a direct comparison against a depthwise LoRA variant with standard initialization.","section":null},{"comment":"In Section 4.2, the spatial rank is fixed to r_sp = k_h * k_w. The paper notes in Section H (Limitations) that this fixed spatial basis design constrains the representation capacity for 1x1 kernels, which are prevalent in modern architectures like ConvNeXt. Given that the primary experiments heavily feature ConvNeXt, this limitation seems significant. The authors should discuss how this constraint impacts the reported results and whether the gains are primarily driven by the channel path rather than the spatial path for 1x1 convolutions.","section":null}],"minor_comments":[{"comment":"In Table A, the formatting of the ResNet-50 LoRA row appears to have spacing issues (e.g., '65.06 82.5356.2165.01'). This should be corrected for readability.","section":null},{"comment":"Figure labels (e.g., Fig. A, Fig. B, Fig. C) are used in the main text but appear to be placeholders. Ensure that figures are numbered sequentially as per standard journal formatting (Fig. 1, Fig. 2, etc.).","section":null},{"comment":"In Section 5.1, the paper mentions using 'sequential CA insertion for ConvNeXt and parallel k×k CA adaptation for ResNet'. It would be helpful to briefly define what 'CA insertion' refers to in this context, as it is not a standard term and relies heavily on the Conv-Adapter [4] reference.","section":null},{"comment":"The reference to 'DINOv3-ConvNeXt [38]' in Section 5.3 appears to cite a 2025 arXiv preprint. Ensure that all citations are complete and up-to-date.","section":null}],"recommendation":"major_revision","confidential_remarks":"The skeptic's concern regarding the mathematical equivalence of the channel mixing path to naive LoRA is valid and lands squarely. The authors need to address this confound in the ablation. If the gains are primarily from module placement rather than the decoupled formulation, the novelty claim is weakened, though the empirical performance remains strong. The paper is a solid contribution if the authors can clarify these experimental controls."},"author_rebuttal":null,"desk_editor":{"model":"glm-5.2","letter":"The headline: LoCA decouples convolutional adaptation into a channel-mixing low-rank path and an SVD-initialized spatial basis path, and it works well across classification, segmentation, and generation. The core idea—don't flatten a 4D convolution kernel into a 2D matrix and pretend spatial structure doesn't matter—is sound and well-motivated. The experiments are broad and the code is public, which I appreciate. The hierarchical rank scheduling is a practical touch that costs nothing conceptually and helps on the deeper stages. The ERF and singular-value evolution analyses are genuinely informative and go beyond the usual bar charts. Credit where due: the paper takes the convolutional PEFT problem seriously rather than hand-waving it. The stress-test concern about Table E is partly valid. The channel-mixing path (Eq. 4) is mathematically identical to flattened convolutional LoRA (Eq. 2)—same matrix dimensions, same rank structure. Yet the ablation shows +1.21 going from 'LoRA Convolution' to 'Channel Mixing.' That gap has to come from confounded factors, most likely module placement (the paper follows Conv-Adapter placement strategies for LoCA but presumably applies LoRA more broadly) or initialization differences. So the paper cannot cleanly attribute that particular gain to its decoupling design. That said, the spatial basis refinement (+0.87 beyond channel mixing) and hierarchical rank (+0.43 beyond that) are genuine additions that the ablation does isolate, and the full method clearly beats the LoRA baseline. A second soft spot: Table F shows the covariance-SVD initialization provides only marginal gains over uniform or flatten-SVD alternatives (66.5 vs 66.2–66.3 on DGSS). The SVD spatial prior helps, but not dramatically. The 1×1 kernel limitation acknowledged in Section H is also non-trivial—ConvNeXt uses many 1×1 convolutions where the spatial basis collapses to a scalar, making the spatial path vacuous. The parameter comparisons against LoRA are also somewhat apples-to-oranges given different module placement. None of these are fatal. The method is a reasonable, well-tested contribution to convolutional PEFT. It's for researchers working on efficient adaptation of convolutional backbones—ConvNeXt, ResNet, hybrid Mamba architectures. A reader building PEFT tooling for vision models will get value here. I'd give it a serious referee. The ablation gap should be addressed in revision but doesn't sink the paper.","headline":"Solid convolutional PEFT method with a real ablation gap","tokens_in":26394,"tokens_out":2007,"would_cite":true,"duration_ms":89204,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Split convolution adaptation into channels and space, keep priors intact","keywords":["parameter-efficient fine-tuning","low-rank adaptation","convolutional neural networks","singular value decomposition","spatial inductive bias","vision foundation models"],"falsifier":"A direct test would compare LoCA against a variant that uses the same channel mixing path and hierarchical rank scheduling but replaces the SVD-based spatial basis with either (a) random spatial bases, (b) no spatial path at all, or (c) a joint low-rank update in flattened space with matching parameter count. If such ablations match LoCA's performance, the spatial basis refinement is not the load-bearing component. The paper's own ablation (Table E) shows the spatial basis adds 0.87 mIoU over channel mixing alone in DGSS, but this margin is modest and task-specific.","tokens_in":25979,"feed_emoji":"🔧","tokens_out":1319,"duration_ms":156595,"temperature":0.7,"pith_summary":"Low-Rank Adaptation (LoRA) has become the standard method for efficiently fine-tuning large pre-trained models, but it was designed for the two-dimensional weight matrices found in transformer attention layers. When applied to convolutional layers—which store weights as four-dimensional tensors coupling spatial kernel shape with input-output channels—LoRA flattens the kernel into a 2D matrix, destroying the spatial topology that gives convolutions their inductive bias. This paper proposes Low-Rank Convolutional Adaptation (LoCA), which separates the adaptation update into two independent paths: a low-rank channel mixing path that handles dense cross-channel dependencies without touching spatial structure, and a spatial basis refinement path that extracts principal spatial patterns from the pre-trained kernel via singular value decomposition of the spatial covariance matrix and learns to adjust them. The two paths are composed additively, with the spatial update placed only on the depthwise diagonal to keep it decoupled from cross-channel mixing. The method also introduces hierarchical rank scheduling that allocates larger adaptation ranks to deeper, wider network stages. Experiments span fine-grained classification, domain-generalized semantic segmentation, and text-to-image generation, with LoCA matching or exceeding prior PEFT methods while using fewer trainable parameters than convolutional LoRA.","feed_headline":"Splitting convolution updates into channels and space beats flattened LoRA","feed_subtitle":"A new PEFT method decouples cross-channel mixing from spatial basis refinement, preserving kernel topology and cutting trainable parameters.","key_machinery":"The core mechanism is a two-path reparameterization of the convolutional weight update. Path one is a standard low-rank channel update (B_c times A_c) operating in flattened kernel space but reshaped back to 4D, handling cross-channel mixing. Path two is a spatial basis expansion: the pre-trained kernel slices are standardized and their spatial covariance matrix is decomposed via SVD to produce a set of principal spatial basis patterns S, which become learnable parameters; depthwise diagonal coefficients U_dw control how each channel combines these spatial patterns. The final update adds the channel mixing tensor to a depthwise-diagonal spatial tensor. A hierarchical rank scheduler scales r^","core_discovery":"The central claim is that decoupling a convolutional weight update into separate channel-mixing and spatial-basis-refinement components preserves the spatial inductive bias of pre-trained convolutional kernels, and that this preservation is what enables better downstream adaptation than naive flattened-kernel low-rank methods. The mechanism carrying this claim is the SVD-based spatial basis: by computing the spatial covariance of the pre-trained kernel, extracting its principal spatial components as a learnable basis, and restricting spatial adaptation to depthwise diagonal coefficients, the method maintains the kernel's inherent spatial structure while still allowing both channel and space-","pith_inferences":["The paper acknowledges that the fixed spatial basis (set to kh*kw) constrains representation capacity for 1x1 convolutions, which dominate architectures like ConvNeXt. If 1x1 kernels are the majority of adapted layers, the spatial path may contribute little for those layers, and the gains may come primarily from the channel mixing path plus the hierarchical rank scheduler rather than from spatial ","The covariance-SVD initialization is evaluated against alternatives (Table F) with modest margins (0.1-0.3 points). This raises the question of whether the specific SVD initialization matters or whether any structured initialization preserving spatial topology would yield similar benefits.","If the spatial basis is frozen (learning rate multiplier 0.0 as reported in the DGSS setup), then the spatial path reduces to learning diagonal coefficients on fixed bases—essentially a depthwise scaling—which resembles existing methods more than the fully learnable basis described in the general formulation."],"forward_implications":["If spatial-channel decoupling is the key, then any PEFT method applied to convolutional or mixed architectures should explicitly preserve kernel geometry rather than flattening it, potentially shifting how adapter modules are designed across vision and multimodal models.","The SVD-based spatial basis approach could extend to other structured weight tensors beyond convolutions—such as grouped linear layers, mixture-of-experts routing weights, or multi-axis attention—where flattening similarly destroys structural priors.","Hierarchical rank scheduling tied to stage width suggests that uniform rank allocation is suboptimal for any hierarchical backbone, which may influence PEFT design for transformer hierarchies as well as convolutional ones.","The finding that singular value spectra evolve more smoothly under decoupled adaptation (Figure G) provides a diagnostic tool: one could monitor SV dynamics during training as an early indicator of whether a PEFT method is preserving or collapsing representational diversity."],"fun_headline_variants":["LoCA decouples channel and spatial adaptation for vision model fine-tuning","Low-rank convolution adaptation preserves spatial priors from pre-trained kernels","SVD-based spatial basis refinement outperforms flattened LoRA on convolution layers","Convolution-aware PEFT separates cross-channel mixing from spatial structure","Decoupled spatial-channel updates beat matrix-flattened LoRA for vision models"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method assumes that the principal spatial patterns extracted from pre-trained convolutional kernels—via SVD of the spatial covariance matrix—constitute a useful, compact basis for downstream spatial adaptation. If the spatial structure most relevant to a new task is not well-captured by the pre-trained kernel's dominant spatial components, or if the fixed spatial rank (kernel height times width) is too small or too large for the task, the spatial basis refinement path may","fun_headline_variants_meta":{"raw":{"variants":["LoCA decouples channel and spatial adaptation for vision model fine-tuning","Low-rank convolution adaptation preserves spatial priors from pre-trained kernels","SVD-based spatial basis refinement outperforms flattened LoRA on convolution layers","Convolution-aware PEFT separates cross-channel mixing from spatial structure","Decoupled spatial-channel updates beat matrix-flattened LoRA for vision models"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":653,"prompt_tokens":558,"completion_tokens":95,"prompt_tokens_details":null},"tokens_in":558,"tokens_out":95,"duration_ms":82355,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T22:55:39.162255+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"A direct test would compare LoCA against a variant that uses the same channel mixing path and hierarchical rank scheduling but replaces the SVD-based spatial basis with either (a) random spatial bases, (b) no spatial path at all, or (c) a joint low-rank update in flattened space with matching parameter count. If such ablations match LoCA's performance, the spatial basis refinement is not the load-bearing component. The paper's own ablation (Table E) shows the spatial basis adds 0.87 mIoU over channel mixing alone in DGSS, but this margin is modest and task-specific.","supporting_citations":[],"review_version":1}