{"id":"cbbf9c4d-5d96-4452-a18c-ca3ae1b1623a","arxiv_id":"2412.14100","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Conv-adapter PEFT of MedNeXt on BraTS-Africa achieves 0.80 mean Dice, comparable to full fine-tuning (0.77) and better than training only on BraTS-Africa (0.72).","lead":"The paper tests a lightweight 'adapter' fine-tuning method for a brain tumor segmentation model, applied to a small dataset of MRI scans from Sub-Saharan Africa. It reports that this method matches full fine-tuning while using less training time, and improves over training only on the small dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 0.80 PEFT result is measured on the same 35-case validation set used to choose the sequential adapter, with a single run per method; the reported gain is a selection-optimized estimate, not a validated generalization estimate.","rationale":"The paper is honest about variance and reports P-values, and the code is publicly available, which are positive signs. However, the confirmatory status of the main number is undermined by selection on the validation set and single-run training. The paper's own Section 3.5 explicitly says the sequential adapter was selected after comparing with parallel on the validation set; this self-report confirms the selection bias. The reader's weakest assumption already identified this same issue, and my proposed cross-validation or test-set evaluation would settle whether the 0.80 estimate is robust. Because the central empirical claim depends on this estimate, the CONDITIONAL verdict is appropriate: the paper should add this validation before the claim is accepted as a general result. I do not see an internal inconsistency in the architecture or a reason to reject the paper outright; the concern is about the strength of the evidence, not the integrity of the work.","tokens_in":9302,"tokens_out":7291,"duration_ms":65125,"concrete_test":"Run a 5-fold cross-validation on the 60 BraTS-Africa training cases for the three conditions (from-scratch, full-FT, PEFT-sequential), with at least 3 random seeds per condition and the same hyperparameters as reported, then measure mean and standard deviation of Dice on held-out folds. If the PEFT-vs-from-scratch advantage (0.08 in the paper) is not reproduced across folds and seeds, or if full-FT is not meaningfully worse than PEFT, the central claim of PEFT's superior or comparable generalization is not supported. If the BraTS-Africa hidden test set is accessible through MedPerf, submit the trained PEFT and baseline models for a direct test-set comparison as a decisive check.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the 35-case BraTS-Africa validation set being a reliable estimator of PEFT's generalization. Section 3.5 states that the sequential adapter was chosen over parallel because it gave 0.80 vs 0.78 on this same validation set (Table S5), and all reported metrics are from that selected model. Because the selection criterion is the evaluation metric and no separate test set is used, the reported 0.80 and the P=0.000116 comparison to the from-scratch baseline are post-selection numbers: the selection step biases the mean upward and invalidates the naive test. Additionally, each method is trained once; with n=35 patients and Dice std about 0.19, seed-to-seed variation could be comparable to the 0.08 PEFT-vs-baseline gap or the 0.03 PEFT-vs-full-FT gap. The full-FT comparison (P=0.63) is used to claim 'comparable,' but a non-significant difference on a 35-case set is not evidence of equivalence. The BraTS-Africa challenge includes a hidden test set; without evaluating on it (or, failing that, cross-validating on the 60 training cases), the headline gain is not established. This is the load-bearing weakness: the empirical advantage could shrink, disappear, or reverse on unseen data.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a parameter-efficient fine-tuning (PEFT) approach for brain tumor segmentation on the BraTS-Africa dataset, inserting convolutional adapter blocks into a MedNeXt-S backbone pre-trained on BraTS-2021. The authors report that PEFT reaches 0.80 mean Dice on the 35-case BraTS-Africa validation set, compared with 0.72 for a MedNeXt-S trained from scratch on BraTS-Africa (P=0.000116) and 0.77 for full fine-tuning (P=0.63), while reducing training time from about 10 to about 4 hours. The manuscript also documents the domain-shift drop from BraTS-2021 to BraTS-Africa (0.70 to 0.50 Dice) and disaggregates sensitivity/specificity. Source code is provided.","tokens_in":9633,"tokens_out":5272,"duration_ms":42225,"significance":"The paper addresses a relevant and under-served problem: adapting models to a low-resource, domain-shifted African glioma dataset. The central empirical design is reasonable, the compute-savings claim is concrete and useful, and the public code release supports reproducibility. If the claims survive a properly held-out evaluation with multiple seeds, the reported 0.80 Dice result would be a solid practical benchmark for PEFT in this setting. However, the current manuscript does not yet establish that point because the primary numbers are selected and single-run, as detailed in the major comments.","major_comments":[{"comment":"The choice of sequential over parallel adapters was made by comparing 0.80 versus 0.78 average Dice on the same 35-case validation set used for all reported metrics. Consequently, the reported 0.80 mean Dice and the P=0.000116 comparison to the from-scratch baseline are post-selection estimates: the selection step on the evaluation metric inflates the mean and invalidates the naive significance test. The authors should evaluate the final model on the BraTS-Africa challenge test set, or alternatively perform nested cross-validation on the 60 training cases, and report the unselected comparison showing both adapter placements.","section":"Section 3.5 and Supplementary Table S5"},{"comment":"Every method is trained once, so no measure of training stochasticity is presented. With n=35 cases and reported Dice standard deviations of about 0.19, the standard error of the mean is roughly 0.03, which is comparable to the 0.03 gap between PEFT and full fine-tuning. The authors should report results over multiple seeds (at least three, ideally five) with confidence intervals, and use an equivalence test such as TOST if they want to claim that PEFT is comparable to full fine-tuning; P=0.63 is not evidence of equivalence.","section":"Tables 2 and 3"},{"comment":"The baseline labeled \"Without Fine-Tuning\" is confusing and possibly misnamed. The text says the comparison is to \"the baseline MedNeXt model trained only on the BraTS-Africa dataset,\" which is a from-scratch training, not a pre-trained model without fine-tuning. This ambiguity is load-bearing because the headline comparison \"0.80 vs 0.72\" mixes up the notion of fine-tuning. Please rename the baseline (e.g., \"from-scratch\") and clarify how it relates to Table 1's BraTS-Africa evaluation of the BraTS-2021-trained model, which gives 0.50.","section":"Table 2 and Section 4"},{"comment":"The statement that \"full finetuning results is much lesser variance in performance\" is contradicted by Table 3, where full fine-tuning has Avg Dice standard deviation 0.21 and PEFT has 0.19. Either correct the claim or provide the boxplot statistics that support it.","section":"Abstract and Section 4"}],"minor_comments":[{"comment":"The paper switches between \"MedNeXT\" and \"MedNeXt\"; please standardize the spelling throughout.","section":"General"},{"comment":"The preprocessing section lists steps but omits concrete training hyperparameters such as optimizer, learning rate, epochs, batch size, loss function, and number of training steps. Including these in the main text or supplement would improve reproducibility.","section":"Section 3.3"},{"comment":"The statistical tests used to obtain P=0.000116 and P=0.63 are not described; please specify the test (e.g., paired t-test, Wilcoxon signed-rank) and whether it was one- or two-sided.","section":"Section 4"},{"comment":"The phrase \"lower sensitivity (0.75)\" is ambiguous: 0.75 is higher than the full fine-tuning WT sensitivity of 0.68 and the without-fine-tuning value of 0.36. Rephrase to \"lower than its own specificity\" or specify the intended comparison.","section":"Table 4"},{"comment":"The caption says \"Full Fine-Tuning\" and \"With PEFT Fine-Tuning\"; note in the caption that full fine-tuning is initialized from BraTS-2021 weights, as stated in the text, to avoid confusion.","section":"Table 3"},{"comment":"Reference [24] has overlapping authors with the current manuscript; please ensure that the citation does not imply prior independent validation of the exact adapter block if that is not the case.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is quite short and reads like a workshop contribution; for a full journal paper, the missing multi-seed evaluation and test-set validation are substantial. The authors cite their own previous work (ref. 24) for the adapter design, which is acceptable but should be transparently flagged. The BraTS-Africa challenge provides a hidden test set; evaluating there would greatly strengthen the paper. I do not see a novelty problem, but the empirical claim needs to be secured before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a practical paper from the SPARK Academy group: they adapt MedNeXt-S to BraTS-Africa using a small convolutional adapter, report Dice 0.80 on the Africa validation set versus 0.72 when training only on BraTS-Africa, and claim parity with full fine-tuning (0.77) at 4 hours versus 10. The code is on GitHub, the motivation is genuine, and the paper is easy to follow. The compute comparison is honest and useful for low-resource settings.\n\nWhat is actually new is limited. The adapter is a conv-adapter [23] with an extra expansion layer, and applying PEFT to 3D medical segmentation is an expected extension. The newness is the specific BraTS-Africa application, which is fine for a workshop-grade paper but not a new mechanism.\n\nThe soft spots are exactly where the stress-test puts them. The sequential adapter was chosen because it scored 0.80 versus 0.78 for the parallel adapter on the same 35-case validation set used for all reported results. So the headline gain is selection-optimized, and the p=0.000116 is post-selection, not a valid test. Single runs compound this: with Dice std around 0.19 and n=35, seed-to-seed variation could plausibly cover the 0.03 PEFT-vs-full-FT gap and even part of the 0.08 versus from-scratch gap. The \"comparable\" claim also rests on a non-significant difference, which is not evidence of equivalence. To their credit, the authors acknowledge the variance difference in the boxplots, but they still draw the comparability conclusion.\n\nSmaller issues: the \"without Fine-Tuning\" baseline is ambiguous (in Table 2 it seems to mean from-scratch training on BraTS-Africa, while the abstract uses the same phrase for the BraTS-2021-only model that scores 0.50). Missing baselines, such as other PEFT methods or a regularized full fine-tune, limit the interpretation. I do not see deeper incoherence; the sensitivity/specificity breakdown is candid about oversegmentation.\n\nWho is this for? People working on efficient adaptation for medical segmentation in underserved populations will find it a useful datapoint. With multiple seeds and either cross-validation on the 60 training cases or evaluation on the hidden BraTS-Africa test set, this would be a solid conference submission. As it stands, the headline number should not be taken at face value.\n\nI would send it to peer review: the topic matters, the code is available, and a good referee can push for the necessary reanalysis. I would not cite the 0.80 figure as an established result.","headline":"Useful, honest PEFT-for-segmentation application whose headline Dice gain is selection-optimized on the validation set; the compute savings and code are real, but the evidence needs a proper test set before the 0.80 number can be cited.","tokens_in":10105,"tokens_out":2463,"would_cite":false,"duration_ms":22210,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning only small convolutional adapters on a frozen brain-tumor segmentation backbone lifts mean Dice on a shifted African MRI dataset from 0.72 to 0.80, matching full fine-tuning at less than half the training time.","keywords":["parameter-efficient fine-tuning","medical image segmentation","brain tumor segmentation","domain shift","convolutional adapters","BraTS-Africa","low-resource settings","MedNeXt"],"falsifier":"Run PEFT, full fine-tuning, and target-only training each with at least five random seeds on the same BraTS-Africa 60/35 split, and evaluate on a held-out test set never used to choose the adapter design; if full fine-tuning matches or beats PEFT on average across seeds, the central claim of superior average performance weakens to a claim about equal performance at lower compute.","tokens_in":9168,"feed_emoji":"🧠","tokens_out":7269,"duration_ms":54668,"temperature":0.7,"pith_summary":"Brain-tumor segmentation models trained on large Western MRI datasets lose about 20 points of Dice when applied to the smaller, lower-resolution BraTS-Africa dataset from Sub-Saharan Africa. This paper tries to fix that transfer gap cheaply by freezing a pretrained MedNeXt-S model and training only small convolutional adapter blocks on the 60-case African training set. The claim is that adapter-based fine-tuning reaches mean Dice 0.80 on the 35-case validation set, beating the 0.72 of training only on the African data (P=0.000116) and matching full fine-tuning's 0.77 (P=0.63) while needing about 4 hours instead of 10. If correct, this gives low-resource sites a recipe: keep one strong pretrained backbone and adapt it per site with a few million trainable parameters.","feed_headline":"Adapter fine-tuning lifts glioma segmentation Dice from 0.72 to 0.80","feed_subtitle":"Freezing the backbone and tuning small adapters matches full retraining on scarce African MRI data in under half the time.","key_machinery":"The key object is the ConvNeXt-Adapter, a small trainable module inserted sequentially inside each frozen MedNeXt superblock. It applies layer normalization, a 3×3×3 depthwise convolution, a channel-expansion convolution, GELU activation, and a pointwise projection back to the original channel width, then adds the result to the frozen feature stream via a residual skip. Its job is to rewrite the pretrained features just enough to fit the BraTS-Africa distribution while the backbone stays untouched; only 11.2% additional parameters (34.99 million total) are trained.","core_discovery":"On the paper's own terms, the central claim is that parameter-efficient fine-tuning with convolutional adapters enables a lightweight convolutional segmenter to cross a severe MRI domain shift. Pre-training MedNeXt-S on BraTS-2021 yields 0.50 mean Dice on BraTS-Africa without any adaptation, versus 0.70 in-domain. Inserting the proposed ConvNeXt-Adapter blocks—layer norm, 3×3×3 depthwise convolution, channel expansion, GELU, and pointwise projection, added to the frozen feature stream through a residual skip—and training only those adapters on the 60-case BraTS-Africa training set raises validation mean Dice to 0.80. That is 0.08 higher than training only on BraTS-Africa (P=0.000116) and statistically indistinguishable from full fine-tuning (0.77, P=0.63), at roughly 4 hours versus 10 hours of training. The authors attribute the edge over full fine-tuning to the small trainable parameter count preventing overfitting on the small target set, and they report the adapted model oversegments, with specificity 0.99 against sensitivity 0.75.","pith_inferences":["A frozen backbone plus per-site adapters implies a practical multi-site deployment pattern: one shared pretrained model with small site-specific adapter weights, a pattern the paper does not itself propose.","The sequential-over-parallel gap is a testable architectural hypothesis: adapters that modulate the main feature path in place may align features more strongly than a parallel branch; a controlled study varying insertion points could verify it.","Because the adapter configuration was chosen on the same 35-case validation set it is evaluated on, the 0.80 figure is likely optimistic; a held-out test set or cross-validation would give a fairer estimate.","The high-specificity/low-sensitivity profile suggests that a recall-weighted or boundary-aware loss could push sensitivity up without full retraining, since only the adapter layers would need retuning."],"forward_implications":["A network pretrained on BraTS-2021 alone drops from 0.70 to 0.50 mean Dice when evaluated on BraTS-Africa, so cross-population deployment without adaptation is unsafe.","Adapter-based fine-tuning on the small BraTS-Africa training set recovers and exceeds target-only training, improving mean Dice from 0.72 to 0.80 with P=0.000116.","PEFT performance is statistically comparable to full fine-tuning (P=0.63) while cutting training time from about 10 hours to about 4 hours.","Adapter placement matters: sequential insertion scores 0.80 mean Dice, while parallel insertion scores 0.78.","The resulting model oversegments rather than undersegments, with specificity 0.99 and sensitivity 0.75 on whole tumor, which should inform how its output is used clinically."],"supporting_citations":[{"why":"Supplies MedNeXt-S, the lightweight convolutional architecture whose frozen backbone hosts the adapters.","marker":"[17]"},{"why":"Defines BraTS-Africa, the small Sub-Saharan glioma dataset with marked MRI distribution shift that is the target domain.","marker":"[19]"},{"why":"Introduced adapter modules, the parameter-efficient fine-tuning idea this work translates into convolutional form.","marker":"[21]"},{"why":"Provides Conv-Adapter, the depthwise-separable convolutional adapter design this work extends with an expansion layer.","marker":"[23]"},{"why":"Gives the projection-style adapter formulation that the ConvNeXt-Adapter block adapts with depthwise convolutions.","marker":"[24]"},{"why":"Provides BraTS-2021, the 1251-case pre-training dataset used before fine-tuning on BraTS-Africa.","marker":"[25]"},{"why":"MedPerf, the standardized platform used to compute the reported validation Dice and HD95 metrics.","marker":"[31]"}],"fun_headline_variants":["Adapter tuning lifts brain tumor Dice from 0.72 to 0.80 on scarce MRI","Lightweight adapters boost tumor segmentation Dice to 0.80 on scarce data","Freeze backbone, tune adapters: tumor Dice improves 0.72→0.80","Adapter PEFT achieves tumor Dice 0.80 on small African MRI set","PEFT adapters match full tuning, lifting tumor Dice to 0.80"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 35-case BraTS-Africa validation set is treated as the measure of generalization, and the sequential adapter placement was chosen on that same set, so the reported advantage over full fine-tuning may be optimistic; each result also comes from a single training run.","fun_headline_variants_meta":{"raw":{"variants":["Adapter tuning lifts brain tumor Dice from 0.72 to 0.80 on scarce MRI","Lightweight adapters boost tumor segmentation Dice to 0.80 on scarce data","Freeze backbone, tune adapters: tumor Dice improves 0.72→0.80","Adapter PEFT achieves tumor Dice 0.80 on small African MRI set","PEFT adapters match full tuning, lifting tumor Dice to 0.80"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001643,"raw_usage":{"total_tokens":6629,"prompt_tokens":1148,"completion_tokens":5481,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":764,"completion_tokens_details":{"reasoning_tokens":5370}},"tokens_in":764,"tokens_out":5481,"duration_ms":33996,"temperature":1.0,"reasoning_tokens":5370,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:28:35.614887+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run PEFT, full fine-tuning, and target-only training each with at least five random seeds on the same BraTS-Africa 60/35 split, and evaluate on a held-out test set never used to choose the adapter design; if full fine-tuning matches or beats PEFT on average across seeds, the central claim of superior average performance weakens to a claim about equal performance at lower compute.","supporting_citations":[{"cited_title":"ArXiv (2023)","cited_arxiv_id":null,"evidence_quote":"Defines BraTS-Africa, the small Sub-Saharan glioma dataset with marked MRI distribution shift that is the target domain."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Provides Conv-Adapter, the depthwise-separable convolutional adapter design this work extends with an expansion layer."},{"cited_title":"VLSM-Adapter: Finetuning Vision-Language Segmentation Efficiently with Lightweight Blocks","cited_arxiv_id":"2405.06196","evidence_quote":"Gives the projection-style adapter formulation that the ConvNeXt-Adapter block adapts with depthwise convolutions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MedPerf, the standardized platform used to compute the reported validation Dice and HD95 metrics."}],"review_version":1}