{"id":"5685386f-6c08-47b6-af50-5f7492d21af8","arxiv_id":"2506.19118","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A 7x7 depthwise convolution inside a parameter-efficient adapter improves fine-tuned medical image classification by a few percent over standard PEFT baselines.","lead":"The paper adds a large depthwise convolution inside a small trainable adapter module so that frozen pretrained vision models can be fine-tuned for medical images with few extra parameters. The result is a simple parameter-efficient fine-tuning recipe that reports higher accuracy than 11 standard fine-tuning methods on five medical datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim confounded: no 7×7 ConvPass baseline, so LKA may simply be ConvPass with a larger kernel.","rationale":"The reader's weakest-assumption analysis centers on ERF attribution versus parameter count and inductive bias; I agree that the causal claim is under-supported. My sharper concern is the missing control against the strongest convolutional baseline. ConvPass is the nearest PEFT method in Table 2: similar parameter count, depthwise convolution inside a frozen transformer. The paper tests LKA at 3×3, 5×5, and 7×7 (Table 1) but never tests ConvPass at anything but 3×3. Therefore the headline comparison 'LKA outperforms 11 PEFT baselines by 3.5%' is not a controlled test of the purported mechanism: it compares the proposed method at its selected hyperparameter value against a baseline at a different hyperparameter value. If a 7×7 ConvPass matches LKA, the novelty and SOTA claims are materially weakened; if it does not, the reader's attribution concern remains but the empirical ranking is more credible. The paper's own Table 5 (different RF-expansion recipes) suggests kernel shape matters beyond theoretical receptive field, so measuring ERF for the 7×7 ConvPass variant would also clarify the mechanism. I also note the lack of error bars and test-set-based design selection; these are valid and reinforce the CONDITIONAL verdict, but the matched ConvPass experiment is the single most decisive missing check. Verdict unchanged: CONDITIONAL.","tokens_in":8796,"tokens_out":13738,"duration_ms":137085,"concrete_test":"Use the released code to instantiate ConvPass with a 7×7 depthwise kernel, bottleneck width equal to LKA's (8), the same parallel placement as LKA, and the same training recipe (AdamW, cosine schedule, 100 epochs, 80/20 split). Run all five datasets with at least 5 seeds and report mean ± std top-1 accuracy. If 7×7 ConvPass's average is within one standard error of LKA's 0.957, the SOTA and 'kernel-pivotal' claims reduce to retuning an existing adapter's kernel size; if it is clearly below, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Table 2, LKA (0.652M trainable params; 0.957 average) is compared with ConvPass [16] (0.661M; 0.937 average), the closest convolutional adapter, but only at ConvPass's original 3×3 kernel size. The centrality of the 7×7 kernel is asserted in Section 3.2, and the abstract's 3.5% margin is against the average of 11 baselines, not the best baseline (which LKA beats by about 1.4 points). Since LKA and ConvPass both insert a depthwise convolution into a frozen transformer, the reported gain may be attributable to changing ConvPass's kernel to 7×7 and/or its insertion point, rather than to LKA as a new method. No experiment gives ConvPass a 7×7 kernel, and no ERF measurement is reported for such a variant. Without this matched-kernel control, the claim that LKA outperforms the state of the art and that its specific design is pivotal is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Large Kernel Adapter (LKA), a parameter-efficient fine-tuning module for medical image classification. LKA consists of a down-projection, a channel-wise depthwise convolution with a 7x7 kernel, and an up-projection, inserted in parallel with the MSA and FFN modules of a frozen transformer. The authors report top-1 accuracy on five medical datasets using Swin-L, ConvNeXt-L, and ViT-L backbones, comparing against 11 PEFT baselines and full fine-tuning. They claim that a larger kernel is pivotal because it expands the effective receptive field (ERF), and they report a 3.5% average accuracy gain over the baselines. The paper also includes ablations on kernel size, integration position, bottleneck width, and alternative receptive-field-expanding convolutions.","tokens_in":8971,"tokens_out":8610,"duration_ms":81645,"significance":"If the claims hold, LKA is a simple, parameter-efficient recipe for adapting vision transformers to medical imaging, and the paper's kernel-size sweep is a useful empirical contribution. The breadth of the evaluation—three backbones, five datasets, and eleven baselines—is commendable, and the public code link supports reproducibility. However, the central attribution of the gain to kernel size, the state-of-the-art claim, and the statistical reliability of the reported margins are not fully established by the current experiments. The missing matched-kernel ConvPass control and the absence of repeated-seed results are the main factors limiting confidence.","major_comments":[{"comment":"The abstract and Section 3.2 state that LKA surpasses the state of the art by 3.5% in top-1 accuracy. This margin is computed against the average of the 11 PEFT baselines, not against the strongest baseline. In the average column of Table 2, the best PEFT baselines (LoRA and CIAT) reach 0.943, while LKA reaches 0.957, a gap of 1.4 percentage points. Please report the margin over the best baseline and revise the state-of-the-art claim accordingly.","section":"Section 3.2, Table 2"},{"comment":"The comparison with ConvPass is confounded. LKA and ConvPass both insert a depthwise convolution into a frozen transformer, but ConvPass is evaluated only at its original 3x3 kernel size. Since the paper's central claim is that the larger kernel size is pivotal, a ConvPass variant with a 7x7 depthwise kernel is needed, together with an LKA variant using ConvPass's exact placement and module design. Without this matched-kernel control, the reported differences cannot be attributed specifically to the kernel size rather than to placement, activation, or the presence of the down/up projections.","section":"Section 3.2, Table 2"},{"comment":"Hyperparameters are selected on the same test split that produces the final accuracies. The kernel size (Table 3), bottleneck width (Table 6), and integration position (Table 4) are all chosen using the 80/20 test split described in Section 3.1, and all tables report a single run with no error bars. Please add a validation split for model selection and report the mean and standard deviation over at least three seeds. This is essential for supporting the small 1.4-point margin over the best baseline.","section":"Sections 3.1 and 3.3"},{"comment":"The ERF analysis is presented only as visualizations; no quantitative ERF metric is reported, and no ERF estimate is given for the ConvPass baseline. The statement that ERF expansion is the causal mechanism behind the accuracy gain is therefore an inference from correlation. Please provide a quantitative ERF measure (for example, the standard deviation or effective area of the contribution map) for each configuration, and distinguish the kernel-size sweep as the only intervention that directly supports the causal claim.","section":"Section 3.2, item 2 and Figure 2"}],"minor_comments":[{"comment":"The residual term in Eq. (2) is dimensionally inconsistent: the term 'x' is added inside the parentheses before the up-projection, but 'x' has dimension d while 'W_down * x' has dimension d_hat. The intended formula is likely 'x_LKA = W_up * GeLU(DWConv(W_down * x)) + x' or an equivalent form; please correct it.","section":"Equation (2)"},{"comment":"The total parameter count is stated as '2*d*d_hat + (k^2+2)*d_hat + d', but the text then says LKA adds only '(k^2+1)*d_hat' parameters over the vanilla adapter (which has '2*d*d_hat + d_hat + d'). These expressions differ by one bias term; please make the count consistent.","section":"Section 3.3, parameter-efficiency paragraph"},{"comment":"The column header 'Kernel Size' is meaningful only for LKA; the baseline rows should leave this cell empty or report each baseline's convolutional kernel when applicable, because currently the combined header implies a quantity that is not defined for most baselines.","section":"Table 2"},{"comment":"Dataset details are incomplete: please report the number of images per class, input resolution, and the random split procedure, and correct the typo 'Nnajing' in the affiliations.","section":"Section 3.1"},{"comment":"There is a doubled article in 'we report the the top-1 accuracy'; also, Section 3.2 item 1 states the 4.9% margin without noting that it is in percentage points against the 'None' (vanilla adapter) row of Table 1.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is promising and the kernel-size sweep is valuable, but the missing ConvPass-7x7 control, the absence of repeated-seed statistics, and the use of the test set for hyperparameter selection are load-bearing for the central claim. These issues are repairable with additional experiments, so I would not reject at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is a straightforward empirical contribution: stick a 7×7 depthwise conv inside a parallel adapter and it beats a spread of PEFT baselines on five medical datasets by a useful margin. The architecture is simple, the ablations are fairly thorough, and the code is public. The improvement over the best baseline is about 1.4 points, not the headline 3.5% (which is against the average of 11 baselines). That's still meaningful if it holds.\n\nWhat's genuinely new is the targeted kernel-size sweep for adapters in medical imaging and the parallel placement result. There is prior work using convolutions in adapters — Convpass, ST-Adapter, RepAdapter — so this lives in an established design space. The paper does not hide that; it cites them. The kernel-size ablation (Table 1) and the comparison with dilated and stacked 3×3 convolutions (Table 5) give some evidence that a single large kernel is doing the work.\n\nNow the soft spots, in proportion to how soft they are. The biggest is the missing control: LKA uses a 7×7 depthwise conv in a parallel adapter, and Convpass is a 3×3 conv in a parallel adapter. No experiment gives Convpass a 7×7 kernel. So the marginal claim that a large kernel is pivotal, rather than just a bigger kernel on an existing design, is not actually established. This is a fixable addition, but it should be done before the paper is published. Second, there are no error bars or repeated seeds anywhere. With differences of 1-2 points, that matters. Third, the kernel size, bottleneck width, and placement are selected on the same test sets that produce the final numbers, which inflates the result and makes the comparisons unfair to baselines that weren't tuned the same way. The ERF visualization is interesting but correlational; it does not prove that ERF expansion causes the accuracy gain, though that is a minor overstatement.\n\nThere are also a few citation gaps—e.g., they don't cite the large-kernel CNN literature beyond one paper—and the text has some typos (\"Nnajing\"). None of that changes the empirical core.\n\nWho is this for? Anyone working on parameter-efficient fine-tuning for medical imaging. It's a credible baseline and a useful design note. The paper deserves peer review: the recipe is clear, the experiments are broad, and the issues are addressable with a validation split, repeated runs, and one matched-kernel Convpass comparison. I would not desk-reject it; I would send it back for a solid revision.","headline":"A clean, useful empirical recipe for medical-image PEFT, but the SOTA claim is undercut by a missing matched-kernel ConvPass control and the absence of error bars.","tokens_in":9546,"tokens_out":1417,"would_cite":false,"duration_ms":16042,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 7x7 convolutional adapter beats 11 PEFT baselines on medical images","keywords":["Parameter-Efficient Fine-Tuning","Large Kernel Adapter","Medical Image Classification","Effective Receptive Field","Channel-wise Convolution","Transformer Fine-Tuning","Transfer Learning","Depthwise Convolution"],"falsifier":"Replace the LKA's $7\\times 7$ channel-wise convolution with the same kernel whose spatial positions are randomly shuffled, keeping the training recipe unchanged; if accuracy does not drop on the five medical datasets, the large receptive field is not the active mechanism. A second check is a vanilla adapter with identical parameter count, optimizer, and schedule; if it matches LKA, the paper's central claim fails.","tokens_in":8576,"feed_emoji":"🩺","tokens_out":8180,"duration_ms":73673,"temperature":0.7,"pith_summary":"Parameter-efficient fine-tuning (PEFT) underperforms on medical images, this paper argues, because existing adapters never enlarge the model's effective receptive field, even though medical images need long-range spatial context. The Large Kernel Adapter (LKA) addresses this by putting a channel-wise $7\\times 7$ convolution inside the adapter bottleneck, and the paper reports that this small change beats 11 PEFT methods by $3.5\\%$ average top-1 accuracy across five medical datasets. It also usually beats full fine-tuning while training only 0.652M parameters. The paper's contribution is a concrete mechanism, not just a new module: adapter kernel size, not parameter count, is what drives transfer performance on medical images.","feed_headline":"A 7x7 convolutional adapter beats 11 PEFT baselines on medical images","feed_subtitle":"Small depthwise large-kernel module lifts average top-1 accuracy 3.5 points over state-of-the-art PEFT methods.","key_machinery":"The central machinery is the Large Kernel Adapter (LKA), a bottleneck block defined by $x_{\\text{LKA}} = W_{\\text{up}}\\, \\text{GeLU}(\\text{DWConv}_{k\\times k}(W_{\\text{down}}\\, x)) + x$, with $W_{\\text{down}}\\in\\mathbb{R}^{d\\times \\hat{d}}$, $W_{\\text{up}}\\in\\mathbb{R}^{\\hat{d}\\times d}$, and $\\text{DWConv}_{k\\times k}$ a channel-wise (depthwise) convolution, typically $7\\times 7$. The channel-wise design adds only about $(k^2+1)\\hat{d}$ parameters relative to the plain adapter, keeping the receptive-field gain cheap. The paper also places LKA in parallel with the multi-head self-attention and feed-forward blocks rather than sequentially, which its ablations show is the best wiring.","core_discovery":"The paper's central claim is that kernel size in the adapter bottleneck is the decisive variable for adapting frozen pre-trained vision models to medical imaging, and that a single channel-wise $7\\times 7$ convolution is sufficient to capture the needed context. The evidence is a controlled sweep: as the kernel grows from $1\\times 1$ to $7\\times 7$, accuracy rises monotonically across Swin, ConvNeXt, and ViT backbones, with $7\\times 7$ beating the vanilla adapter by $4.9\\%$ on average. Visualizations show the effective receptive field expanding with kernel size. Additional ablations show $7\\times 7$ channel-wise convolution outperforms stacked $3\\times 3$ layers and dilated $3\\times 3$ with dilation 3, and kernels beyond $7\\times 7$ begin to hurt, giving the method a sweet spot rather than a monotonic rule.","pith_inferences":["The causal story rests on the correlation between measured ERF and accuracy; the paper never intervenes on the kernel's spatial structure while holding everything else fixed, so a randomized-kernel control would tell whether the receptive field or the depthwise convolution's optimization dynamics drives the gain.","Because the gain appears across three very different pre-trained architectures, the recipe may transfer to other small-sample, low-contrast imaging domains such as pathology or retinal images, but that is an extrapolation the paper does not test.","The same bottleneck may help segmentation and detection, where context is also critical, but the paper's evidence is limited to classification top-1 accuracy.","The $7\\times7$ sweet spot hints at a scale-matching rule: the adapter's receptive field should match the typical spatial extent of the structures being classified; datasets with different lesion scales could shift the optimal kernel size."],"forward_implications":["Kernel size, not parameter count, is the active ingredient: increasing the LKA kernel from $1\\times1$ to $7\\times7$ adds few parameters yet raises accuracy by roughly 5 points on Covid and 7 points on BUSI.","Seven-by-seven is an optimum: kernels of $9\\times9$, $11\\times11$, and $31\\times31$ all underperform $7\\times7$, so the design rule is a moderate large kernel, not an arbitrarily large one.","Parallel integration with the Transformer block (alongside MSA and FFN) is better than sequential insertion, so adapter placement matters as much as adapter structure.","A single $7\\times7$ channel-wise convolution beats both a stack of three $3\\times3$ convolutions and a dilated $3\\times3$ kernel, indicating that one large kernel captures the global context more effectively than compounding small receptive-field expansion recipes.","LKA exceeds full fine-tuning on most datasets while updating only 0.652M parameters, so the receptive-field fix is compatible with the core promise of parameter-efficient fine-tuning."],"supporting_citations":[{"why":"Defines the vanilla adapter (down-projection, up-projection) that LKA extends with a large-kernel convolution.","marker":"[5]"},{"why":"Supplies channel-wise convolution, the parameter-efficient way to implement the large kernel inside LKA.","marker":"[8]"},{"why":"Motivates large-kernel designs in vision models and the expansion of effective receptive fields.","marker":"[12]"},{"why":"A convolutional bypass adapter baseline that LKA is compared against and outperforms.","marker":"[16]"},{"why":"Provides the receptive-field computation background for understanding and measuring ERF.","marker":"[29]"},{"why":"The ERF measurement protocol (sampling images and accumulating pixel contributions) used to visualize receptive-field expansion.","marker":"[30]"},{"why":"Supplies dilated convolution, one of the alternative receptive-field-expansion recipes compared in the ablation.","marker":"[31]"}],"fun_headline_variants":["7x7 kernel sweet spot beats stacked 3x3s and dilated convs","LKA: one 7x7 depthwise conv lifts PEFT by 3.5% on medical","Why kernel size matters: LKA outperforms 11 PEFT baselines","Adapter kernel size: 7x7 sweet spot beats bigger and smaller","Single large-kernel adapter matches or beats 11 PEFT methods"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the accuracy gains come from the larger effective receptive field, not from the extra parameters, the depthwise convolution's inductive bias, or an incidental optimization effect.","fun_headline_variants_meta":{"raw":{"variants":["7x7 kernel sweet spot beats stacked 3x3s and dilated convs","LKA: one 7x7 depthwise conv lifts PEFT by 3.5% on medical","Why kernel size matters: LKA outperforms 11 PEFT baselines","Adapter kernel size: 7x7 sweet spot beats bigger and smaller","Single large-kernel adapter matches or beats 11 PEFT methods"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1446,"prompt_tokens":912,"completion_tokens":534,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":427}},"tokens_in":528,"tokens_out":534,"duration_ms":5152,"temperature":1.0,"reasoning_tokens":427,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:36:57.174531+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the LKA's $7\\times 7$ channel-wise convolution with the same kernel whose spatial positions are randomly shuffled, keeping the training recipe unchanged; if accuracy does not drop on the five medical datasets, the large receptive field is not the active mechanism. A second check is a vanilla adapter with identical parameter count, optimizer, and schedule; if it matches LKA, the paper's central claim fails.","supporting_citations":[{"cited_title":"International Conference on Machine Learning, pp","cited_arxiv_id":null,"evidence_quote":"Defines the vanilla adapter (down-projection, up-projection) that LKA extends with a large-kernel convolution."},{"cited_title":"Advances in Neural Information Processing Systems 31 (2018)","cited_arxiv_id":null,"evidence_quote":"Supplies channel-wise convolution, the parameter-efficient way to implement the large kernel inside LKA."},{"cited_title":"Distill 4(11), pp","cited_arxiv_id":null,"evidence_quote":"Provides the receptive-field computation background for understanding and measuring ERF."},{"cited_title":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"The ERF measurement protocol (sampling images and accumulating pixel contributions) used to visualize receptive-field expansion."}],"review_version":2}