{"id":"47cf99a6-c529-47d5-a3d3-b7f730a3e29c","arxiv_id":"2505.08932","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On a 0.3 ha drone orthomosaic, adapter-based fine-tuning of SAM reached 0.416 mIoU, just ahead of LoRA at 0.410, and far above zero-shot SAM at 0.151.","lead":"UAVs may soon map forest floors automatically: this paper tunes the Segment Anything Model with adapters and LoRA to find stumps, logs, and vegetation in drone images. The adapter version reaches the highest mIoU, with LoRA close behind at half the trainable parameters.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Top-two mIoU gap of 0.006 rests on 36 noisy test windows; no test-window confidence interval or corrected-label evaluation is provided.","rationale":"The reader's weakest assumption identifies exactly the load-bearing concern: the measured mIoU differences are small and the test set is small and noisily labeled. My stress-test agrees and adds specificity. The 0.006 gap between AdapterH and LoRA is smaller than the acknowledged labeling error scale, and the reported seed-to-seed standard deviations do not quantify uncertainty from test-window sampling or label noise. The paper itself provides the evidence for this concern in Section IV-C and Figs. 8-9, so this is not an external or speculative objection. The claim is otherwise scoped honestly: it does not claim superiority over full fine-tuning or other architectures, and the code is available, which supports reproducibility once the dataset is released. The absence of a full-fine-tuning baseline and the imperfect comparability of the zero-shot SAM experiment are secondary and do not change the central conditionality. Therefore the reader's CONDITIONAL verdict should stand unchanged until the proposed corrected-label or bootstrap evaluation is run.","tokens_in":7147,"tokens_out":4588,"duration_ms":52639,"concrete_test":"Re-annotate the 36 test windows with a second independent annotation pass (or a consensus pass that fixes the missed and incorrectly labeled CWD regions described in Section IV-C), then recompute the Table I mIoU for AdapterH and LoRA on the corrected labels while keeping the same trained models. If the AdapterH minus LoRA difference changes sign, or drops below the across-seed standard deviation of 0.001, the claim that adapter-based PEFT achieves the highest mIoU is not supported. As a supplementary check, bootstrap the 36 test windows with 10,000 resamples to obtain a 95% confidence interval for the AdapterH minus LoRA mIoU difference on the original labels; if the interval includes 0, the ranking is statistically unresolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim in Table I is that AdapterH (mIoU 0.416) outperforms LoRA (mIoU 0.410) by 0.006, with the paper's conclusion that adapter-based PEFT achieves the highest mIoU. This difference is the main evidence for the ranking, but it is evaluated on only 36 test windows drawn from a single orthomosaic (Section IV-A), and the reported standard deviations are across three training seeds, not across test windows or label-noise realizations. Section IV-C explicitly acknowledges that many CWD instances are 'incorrectly labeled or entirely missed,' that CWD boundaries are subjective, and that stumps and upside-down logs look alike (Figs. 8 and 9). Label noise of exactly this kind can shift mIoU by more than 0.006: a missed CWD region penalizes the model that correctly predicts it, while a wrong label rewards the model that reproduces the error. Because AdapterH and LoRA produce different false-positive patterns, the 0.006 margin may reflect annotation artifacts rather than method quality. No bootstrap or permutation test over the 36 windows is reported, so we cannot tell whether the ranking would survive resampling. This is a fixable conditionality rather than a logical contradiction, but it is the load-bearing weak point of the paper's headline claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript adapts the Segment Anything Model (SAM) to forest floor segmentation from UAV orthomosaics using parameter-efficient fine-tuning (PEFT). The image encoder is frozen; AdapterH, AdapterL, LoRA modules, or no-PEFT (decoder-only) are trained together with a modified mask decoder that outputs class masks without prompts. Evaluation is on field-D of the Garrulus dataset (0.3 ha, ~10 cm/pixel), with 1,000 training windows and 36 fixed-grid test windows, three seeds. Table I reports AdapterH mIoU 0.416, LoRA 0.410, AdapterL 0.359, decoder-only 0.321, and zero-shot SAM 0.151. The paper concludes that adapter-based PEFT achieves the highest mIoU, with LoRA as a lightweight alternative for UAV platforms.","tokens_in":7435,"tokens_out":5779,"duration_ms":57108,"significance":"If the ranking is robust, the paper gives a practical recipe for adapting a vision foundation model to a small-domain remote sensing task with limited trainable parameters, and the code release plus the honest discussion of label noise are commendable. The main contribution is empirical and domain-specific; it does not claim new PEFT theory. The result is potentially useful for reforestation and seed-dispersal pipelines, but its strength is bounded by the single 0.3 ha field, the 36-window test set, the acknowledged annotation ambiguity, and the lack of a full-fine-tuning baseline.","major_comments":[{"comment":"The headline comparison between AdapterH (mIoU 0.416) and LoRA (0.410) rests on a 0.006 difference evaluated on 36 fixed-grid windows from a single orthomosaic. The reported ±0.001 standard deviations are across three training seeds and do not capture test-window sampling variability or label noise, which Section IV-C explicitly documents as severe for CWD (incorrectly labeled, missed instances, subjective boundaries). With only 36 test windows, a bootstrap or permutation test over windows is needed to show that the AdapterH-vs-LoRA ordering is not a sampling artifact, and ideally the evaluation should be repeated after correcting or excluding the most ambiguous CWD labels. Without this, the statement that AdapterH achieves the highest mIoU is not supported to the precision implied by Table I.","section":"§IV-A, Table I"},{"comment":"mIoU is reported as a single macro-average, but Fig. 10 shows that the BACKGROUND class dominates the per-class results, and MISC and STUMP have significantly lower mIoU. Because the target application is obstacle detection (CWD, STUMP, vegetation), a method could win on the background-dominated macro mIoU while being worse on exactly the classes that matter. The paper should report per-class mIoU in a table for all four classes and, if the macro average is kept, also give a foreground-only mIoU. This would clarify whether the AdapterH/LoRA advantage is concentrated in the classes relevant to the seeding application.","section":"§IV-C, Fig. 10"},{"comment":"The zero-shot SAM baseline is not measured on the same task as the supervised methods. SAM's automatic mask generator produces class-agnostic proposals; the authors then decide a predicted mask is a true positive if its IoU with a ground-truth mask exceeds 50%, and this is aggregated into an mIoU. That is a detection and proposal matching heuristic, not semantic segmentation, and the 50% threshold is an additional design choice. The 0.151 figure should be labeled as a proxy, or replaced by a class-agnostic metric such as average recall of proposals or mask IoU without class assignment, so that the comparison with the PEFT mIoU values is not misleading.","section":"§IV-C, zero-shot protocol"}],"minor_comments":[{"comment":"The learning-rate description 'cosine annealing schedule that decreased the rate by a factor of 0.1 at each step' is inconsistent with cosine annealing; please describe the schedule and the final learning rate precisely.","section":"§IV-B"},{"comment":"The loss weighting between weighted cross-entropy and Dice loss is not given; please specify the weights used.","section":"§IV-B"},{"comment":"The caption does not state that the ± values are standard deviations over three training seeds; please state this and, ideally, provide per-window mIoU variability as well.","section":"Table I"},{"comment":"The dense-embedding ablation reports improvements of '0.02% and 0.04%'; these appear to be absolute mIoU percentage points rather than relative percentages, and the text should state which one is meant.","section":"§IV-C"},{"comment":"The zero-shot experiment is described as 'following the approach by Hu et al. [7]', but [7] is the LoRA paper and is not the source for SAM's automatic mask generator; please correct the citation.","section":"§IV-C, references"},{"comment":"The dataset is not yet released, and the test set consists of 36 windows; even a small release of the test annotations or a data-generation script would materially improve reproducibility.","section":"§IV-A"},{"comment":"There are small typos: 'UA V' appears with a spurious space in the title and abstract, and 'usign' appears in Section IV-C in the ablation description.","section":"Title and §IV-C"}],"recommendation":"major_revision","confidential_remarks":"The paper is an application-oriented empirical study that fits a robotics and agriculture workshop. The main risk is statistical, not methodological: the top-two difference is small, the test set is tiny, and the authors themselves document substantial label noise. I would encourage the editor to require the authors to address Major Comment 1 before archival publication, even if the workshop version is accepted. The authors' acknowledgment of label noise is a positive sign that they are aware of the issue. No concerns about citation integrity or novelty disclosure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a solid, plainly-written empirical study: they take SAM, freeze the image encoder, bolt on three PEFT variants (serial adapter, parallel adapter, LoRA), train SAM's mask decoder on a new UAV orthomosaic of a post-harvest forest floor, and compare against decoder-only fine-tuning and zero-shot SAM. What is actually new is the dataset (Garrulus field-D, 0.3 ha, 10 cm/pixel, CWD/stump/vegetation/misc classes) and the head-to-head numbers; the methods are all established. The paper does not oversell a new mechanism, and the conclusions are scoped to this dataset and protocol.\n\nThe execution is careful in ways that matter: three random seeds, per-class mIoU, an ablation of the learned dense embedding, and—refreshingly—an explicit admission that the CWD labels are noisy and subjective, with examples shown. They also release code, which is real evidence. The zero-shot SAM baseline at 0.151 mIoU is a useful sanity check.\n\nThe soft spots are real, but they are conditions rather than fatal flaws. First, the headline result is a 0.006 mIoU gap between AdapterH (0.416) and LoRA (0.410) on only 36 test windows drawn from a single orthomosaic. The reported standard deviations are across training seeds, not across test windows or label-noise realizations. Given that the authors themselves note many CWD instances are incorrectly labeled or missed, and that stumps and upside-down logs look alike, that 0.006 margin could plausibly be reshuffled by annotation artifacts. A bootstrap or permutation test over the 36 windows, or a re-evaluation on a cleaned subset, would tell us whether the ranking is stable. Second, without a full fine-tuning baseline, we do not know whether the PEFT gains are meaningful relative to just training everything—though the paper does not claim otherwise. Third, the dataset is withheld, so the numbers are not independently reproducible yet; the code helps, but only for the same sample.\n\nI would take this paper seriously as a workshop-level contribution for the precision-agriculture and forestry community. It is exactly the kind of empirical comparison that community needs, even if the top-two ranking is not robust enough to be load-bearing. My recommendation: engage with it, send it to peer review, but as a referee I would ask the authors to either soften the superiority claim or back it with window-level uncertainty estimates and a label-noise analysis. The underlying engineering is sound, the reporting is honest, and the dataset will likely be useful once released.\n\nBest,\n\n[You]","headline":"A straightforward, honest comparison of PEFT adapters vs. LoRA for SAM on a new UAV forest-floor dataset—useful as a workshop study, but the 0.006 mIoU gap between the top two methods needs much stronger evidence before anyone should lean on it.","tokens_in":7943,"tokens_out":1255,"would_cite":false,"duration_ms":14211,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adapter-based parameter-efficient fine-tuning gives the best forest-floor segmentation from UAV imagery, with LoRA nearly matching it at half the trainable parameters.","keywords":["forest floor segmentation","UAV imagery","Segment Anything Model","parameter-efficient fine-tuning","LoRA","adapter tuning","coarse woody debris","orthomosaic"],"falsifier":"Re-annotate the 36 test windows with a stricter, consensus-based definition of coarse woody debris and with stumps separated from upside-down logs, then retrain all four variants under the same protocol; if the serial adapter no longer leads LoRA, or if the order reshuffles across random seeds, the reported ranking is an artifact of annotation noise.","tokens_in":6996,"feed_emoji":"🌲","tokens_out":5976,"duration_ms":48868,"temperature":0.7,"pith_summary":"The paper asks whether a vision foundation model, SAM, can be adapted to segment forest-floor objects in UAV orthomosaics—tree stumps, coarse woody debris, and vegetation—without retraining the entire model and without manual prompts. It reports that injecting small trainable modules into the frozen image encoder while fine-tuning only the mask decoder is enough: the serial-adapter variant reaches mIoU 0.416, LoRA reaches 0.410 with roughly half the trainable parameters, and both clearly beat fine-tuning the mask decoder alone (0.321) and zero-shot SAM (0.151). The value would be a practical route to automatic obstacle detection for UAV-assisted reforestation, where model size and compute are constrained.","feed_headline":"Adapter-tuned SAM beats decoder-only tuning on forest floor","feed_subtitle":"On a 36-window orthomosaic test set, serial adapters reach mIoU 0.416 and LoRA 0.410 with half the trainable parameters.","key_machinery":"The mechanism is a set of PEFT modules placed in the frozen image encoder of SAM, combined with a modified mask decoder. Adapters are bottleneck MLPs (down-projection, nonlinearity, up-projection) inserted after attention or beside the MLP, with a scale factor on the residual. LoRA adds low-rank matrices B and A to the query and value weight matrices, W_q and W_v. The paper keeps the image encoder frozen, trains only the PEFT modules, and replaces SAM's prompt-dependent decoding so the decoder outputs N class masks for N categories, removing the prompt encoder entirely.","core_discovery":"On its own terms, the paper establishes that PEFT can take SAM from a prompt-driven generic segmenter to an automatic class-specific segmenter for a difficult natural scene. With SAM's image encoder frozen, adding two serial bottleneck adapters per transformer block yields mIoU 0.416; LoRA added to the query and value projections yields 0.410 from 4.99M trainable parameters; a single post-MLP adapter yields 0.359; training only the mask decoder yields 0.321; and zero-shot SAM with point prompts yields 0.151. The paper concludes that adapter- and LoRA-based tuning outperform direct decoder fine-tuning, that where the modules sit matters more than parameter count, and that LoRA is the lightweight option for resource-constrained platforms.","pith_inferences":["In our reading, the 0.006 mIoU gap between the serial adapter and LoRA is within the noise floor of a 36-window test set with known label inconsistency, so the ranking should not be treated as stable until labels are cleaned.","The paper's finding that module placement matters more than parameter count suggests a testable extension: sweeping adapter positions across attention and MLP blocks to find a sparser configuration that still matches the serial adapter.","Extending the same protocol to the other unannotated fields of the surveyed forest area would show whether the ranking transfers across sites and flight conditions."],"forward_implications":["Forest-floor obstacle maps can be produced automatically from UAV orthomosaics without manual prompting.","LoRA, with 4.99M trainable parameters, is a viable lightweight choice for on-board deployment on resource-constrained UAV platforms.","PEFT avoids full fine-tuning of SAM, cutting memory and compute requirements.","Performance is uneven across classes, with background dominating and coarse woody debris the lowest, pointing to annotation quality as the next bottleneck."],"supporting_citations":[{"why":"Supplies the frozen SAM architecture (image encoder, prompt encoder, mask decoder) and its pre-trained weights.","marker":"[4]"},{"why":"Defines the LoRA method used in the paper, including the low-rank update to the query and value weight matrices and the scaling factor.","marker":"[7]"},{"why":"Introduces the serial adapter design that the AdapterH variant follows.","marker":"[8]"},{"why":"Introduces the single post-MLP adapter design that the AdapterL variant follows.","marker":"[9]"},{"why":"Combines serial and parallel adapters and supplies the AdapterH configuration with bottleneck dimension 32.","marker":"[10]"}],"fun_headline_variants":["Adapter-tuned SAM tops LoRA on forest floor","Frozen SAM, tiny adapters: forest floor segmentation","Serial adapters beat LoRA in UAV forest floor mapping","PEFT lets SAM segment stumps, debris from drone shots"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results depend on the forest-floor annotations being accurate enough that the measured mIoU differences (notably 0.416 versus 0.410) reflect method quality rather than label noise, a premise the paper itself weakens by reporting many missed or mislabeled coarse-woody-debris instances and only 36 test windows.","fun_headline_variants_meta":{"raw":{"variants":["Adapter-tuned SAM tops LoRA on forest floor","Frozen SAM, tiny adapters: forest floor segmentation","Serial adapters beat LoRA in UAV forest floor mapping","PEFT lets SAM segment stumps, debris from drone shots"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1408,"prompt_tokens":895,"completion_tokens":513,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":446}},"tokens_in":511,"tokens_out":513,"duration_ms":5429,"temperature":1.0,"reasoning_tokens":446,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:44:13.021303+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate the 36 test windows with a stricter, consensus-based definition of coarse woody debris and with stumps separated from upside-down logs, then retrain all four variants under the same protocol; if the serial adapter no longer leads LoRA, or if the order reshuffles across random seeds, the reported ranking is an artifact of annotation noise.","supporting_citations":[{"cited_title":"Segment Anything,","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen SAM architecture (image encoder, prompt encoder, mask decoder) and its pre-trained weights."},{"cited_title":"Parameter-Efficient Transfer Learning for NLP,","cited_arxiv_id":null,"evidence_quote":"Introduces the serial adapter design that the AdapterH variant follows."},{"cited_title":"Adaptformer: Adapting Vision Transformers for Scalable Visual Recog- nition,","cited_arxiv_id":null,"evidence_quote":"Introduces the single post-MLP adapter design that the AdapterL variant follows."}],"review_version":1}