{"id":"6a80f660-1158-4fc2-a6ec-6e7ddc99b745","arxiv_id":"2501.00527","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Adding focal and boundary loss to a Mask2Former network with a separate leaf decoder improves leaf and weed segmentation and modestly improves leaf counting on the PhenoBench agricultural dataset.","lead":"The paper shows that adding two existing training tricks, focal loss and boundary loss, to a standard segmentation network improves its ability to pick out small leaves and weeds in drone images of sugar beet fields. The approach also slightly improves automatic leaf counting, which matters for crop monitoring in precision agriculture.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Focal loss contributes nothing detectable: Lf+Lb beats Lb alone by 0.03 PQ† and 0.01 RMSE_GT on single runs; only boundary loss is supported, so the abstract's 'focal and boundary loss' claim lacks evidence.","rationale":"The reader's weakest assumption—that sub-half-point differences without error bars may be noise—is the right broad concern, and this stress-test sharpens it to the specific comparison that matters for the abstract's claim. The package-level conclusion depends on the joint benefit of focal loss plus boundary loss, but the ablation shows boundary loss alone accounts for almost all of the gain, while focal loss alone slightly hurts PQ†. The 0.03 PQ† and 0.01 RMSE_GT differences between Lf+Lb and Lb are exactly the kind of single-run effects that cannot be distinguished from seed variation. This is a correctness risk, not an internal inconsistency: the loss formulas and training setup are clear, and the code is released, but the empirical support for the 'focal and boundary loss' combination is missing. The appropriate verdict remains conditional acceptance, requiring multi-seed experiments or a narrowed claim. I do not see a separate defect in the boundary-loss implementation or the hierarchical decoder that would change the verdict, and the paper is honest about the modest leaf-count gains and the imperfect comparison with the 1st-place benchmark.","tokens_in":9662,"tokens_out":5729,"duration_ms":57427,"concrete_test":"Retrain ResNet50 Base, Lb, and Lf+Lb with at least 5 different seeds and report mean +/- standard deviation (or 95% confidence intervals) for PQ†, PQleaf, PQweed, and RMSE_GT on the PhenoBench validation set. If the difference Lf+Lb minus Lb has a confidence interval straddling zero on every metric, focal loss contributes nothing measurable and the abstract should credit boundary loss alone. Also report whether the 81.89 test-set number comes from the same seed/variant used in Table 1.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's advertised contribution is the combination of focal loss and boundary loss (abstract; Eq. 6), and the conclusion credits 'focal loss, boundary loss' jointly. The ablation in Table 1 does not support this. On a single training run each, ResNet50 Lf (focal only) gives PQ† 81.46, slightly below Base 81.48; Lb (boundary only) gives 81.78; Lf+Lb gives 81.81. The incremental benefit of adding focal loss on top of boundary loss is 0.03 PQ† on the validation set and 0.01 RMSE_GT (2.31 to 2.30 in Table 3), i.e., within any plausible seed noise. Leaf PQ and weed PQ do improve more (72.06 to 72.29 and 51.16 to 51.56), but those are still sub-half-point changes without repeated runs. Since focal-only actually reduces the headline PQ†, the only ingredient with clear standalone benefit is boundary loss. The central claim that the combination improves small-instance segmentation is therefore not established; the paper should either report multi-seed variance or reword the claim to credit boundary loss alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an adapted Mask2Former for hierarchical panoptic segmentation on the PhenoBench dataset: a second transformer decoder is added to produce leaf-level masks alongside plant-level masks, and the mask loss is changed to a combination of focal loss, Dice loss, and Kervadec et al.'s boundary loss. The authors report ResNet-50 ablations on the validation set, a SwinL model compared with competition baselines on the private test set, and leaf-counting RMSE metrics. They claim that the proposed losses improve segmentation of small instances (leaves and weeds) and leaf-count accuracy without affecting inference speed or model footprint.","tokens_in":9950,"tokens_out":8058,"duration_ms":77787,"significance":"If the claims were fully supported, the contribution would be practically useful: a drop-in loss modification that improves small-instance segmentation on a standard agricultural benchmark, with released code and no runtime cost. The validation-set ablations are single training runs, however, and the reported differences—especially the incremental effect of focal loss on top of boundary loss—are small enough to be explained by seed variation. The paper has useful engineering content, but the central claim that the focal-plus-boundary combination drives the improvement needs stronger empirical support before it can be accepted as stated.","major_comments":[{"comment":"The central claim that the combination of focal loss and boundary loss improves leaf and weed segmentation is not established by the reported single-run ablations. Focal-only (ResNet50 (Lf)) gives PQ+ 81.46, slightly below the base 81.48, while boundary-only gives 81.78 and Lf+Lb gives 81.81; the incremental effect of focal loss over boundary loss is 0.03 PQ+ on the validation set and 0.01 RMSE_GT (2.31 to 2.30). The larger sub-metric gains (PQ_leaf 72.06 to 72.29; PQ_weed 51.16 to 51.56) are also on the order of a few tenths of a point without repeated runs. These numbers support boundary loss as the active ingredient, not the focal-plus-boundary combination advertised in the abstract and conclusion. Please report repeated-run means and variance, or reword the contribution to credit boundary loss alone.","section":"Section 5, Tables 1 and 3"},{"comment":"Test-time augmentation is applied selectively after inspecting validation-set results: the authors state that they tried TTA but 'only found any benefit when we consider the average results on leaf segmentation.' Because this selection is made on the same validation set used for the reported numbers, the TTA rows in Tables 1 and 2 (e.g., PQ_leaf 72.29 to 72.61 and PQ+ 81.81 to 81.89 for ResNet-50) are optimistic. The paper should either report all TTA configurations tried, treat the validation TTA results as a hypothesis to be confirmed on the private test set, or remove TTA from the headline claim.","section":"Section 4.3 and Tables 1-2"},{"comment":"All SwinL results use the proposed Lf+Lb loss, but no SwinL baseline or SwinL boundary-only ablation is provided. Therefore the paper does not demonstrate that the loss changes, rather than the stronger backbone or the separate leaf decoder, are responsible for the SwinL performance (83.62/83.70 on validation, 81.89 on the test set). Since the state-of-the-art comparison in Table 4 is made with this SwinL model, at least one SwinL control is needed to support the attribution of the gains to the proposed losses.","section":"Section 5, Table 2"}],"minor_comments":[{"comment":"The phrase 'achieving a PQ+ of 81.89 on the standard training set' is misleading: the result is obtained on the private test set by a model trained on the standard 1407-image training set. Please rephrase to 'trained on the standard training set and evaluated on the PhenoBench test set.'","section":"Abstract and Table 4"},{"comment":"In the definition of RMSE_GT, the sentence 'This shows the counting error accounting for the role of false positive crop predictions' should say 'false negative crop predictions,' because unmatched ground-truth crops (with predicted leaf count zero) are missed detections, not false positives.","section":"Section 3.3"},{"comment":"The metric names in the table header are corrupted in the manuscript (e.g., 'RM SET P', 'RM SEP red', 'RM SEGT'). They should be formatted as RMSE_TP, RMSE_Pred, and RMSE_GT.","section":"Table 3"},{"comment":"Please clarify what 'TTA ... on the leaf segmentation' means operationally: is the leaf mask averaged over all augmentations while plant and weed outputs are computed from a single pass, and are the same augmentations used for the SwinL test-set results in Table 4?","section":"Section 4.3"},{"comment":"The reference to previous work should read 'Aich et al.' rather than 'Aich et el.'","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is borderline: the methodology is reasonable and the code release is a plus, but the headline claim of a combined focal-plus-boundary loss benefit is not backed by repeated runs. I would encourage the editor to require multi-seed experiments or a clear re-scoping of the contribution before acceptance. The selective application of TTA after validation-set inspection is a further risk that should be addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper is a useful data point on boundary loss, and a cautionary example of overclaiming a combination. The title says boundary loss; the abstract credits both focal and boundary loss. The ablation table only supports the boundary half.\n\nWhat's new: a hierarchical Mask2Former with a separate leaf decoder, applied to PhenoBench, plus focal loss and boundary loss. That specific package hasn't been tested on this benchmark before. The paper also reports leaf-counting RMSE, which is the metric that matters for the intended application. Code is on GitHub, so the work is reproducible.\n\nWhat it does well: the ablation is clean to read. Boundary loss alone raises PQ† by 0.30 over the base ResNet-50 model (81.48 -> 81.78), and weed PQ jumps from 49.17 to 51.16. Leaf RMSE_GT drops from 2.35 to 2.31. The authors are candid that leaf-counting gains are modest and that PQ improvements don't always reduce counting error. That honesty earns credit.\n\nThe soft spots are exactly where the stress-test note lands. Focal loss alone is a wash: 81.46 vs. 81.48 base. Adding focal loss on top of boundary loss changes PQ† by 0.03 and RMSE_GT by 0.01. That is not a measurable effect on single runs. The conclusion that the combination improves small-instance segmentation is therefore not established. The focal loss should either be backed by multi-seed variance or dropped from the claim. Relatedly, there are no error bars anywhere; TTA is applied only to leaves after the authors saw it help there; and the loss weights and boundary alpha schedule are hand-set with no explicit note about validation-set tuning.\n\nBottom line: the boundary-loss result is real, modest, and worth having in the literature. The focal-loss claim should be reworded or supported. A serious referee should see this—the benchmark comparison and code are useful to anyone working on plant/leaf panoptic segmentation. I'd ask for a few seeds or a corrected claim before acceptance.","headline":"The title is more honest than the abstract: boundary loss gives a small real gain, but focal loss is a dead end in the ablation as reported.","tokens_in":10441,"tokens_out":2651,"would_cite":false,"duration_ms":26838,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hierarchical panoptic segmentation model for sugar-beet fields improves segmentation of leaves and weeds—and with it leaf-count accuracy—by adding focal and boundary losses to Mask2Former, reaching $PQ^\\dagger$ 81.89 on PhenoBench.","keywords":["hierarchical panoptic segmentation","precision agriculture","leaf counting","boundary loss","focal loss","Mask2Former","PhenoBench","crop and weed segmentation"],"falsifier":"Retrain the ResNet-50 base and the ResNet-50 focal-plus-boundary variant several times with different random seeds on the standard 1407-image split and compare the distributions of $PQ^\\dagger$ and leaf RMSE; if the intervals overlap or the ordering flips, the claimed benefit is not reproducible. A simpler check is whether the same recipe changes $PQ^\\dagger$ on the held-out test set in the same direction.","tokens_in":9502,"feed_emoji":"🌱","tokens_out":6001,"duration_ms":56044,"temperature":0.7,"pith_summary":"This paper tries to show that a hierarchical panoptic segmentation model for farm images can be made better at the small, easy-to-miss objects—individual leaves and weeds—without slowing it down or enlarging it. Starting from Mask2Former, a transformer-based segmentation architecture, the authors add a second transformer decoder so plants and leaves are segmented at the same time, then swap cross-entropy for focal loss and add a boundary loss originally designed for highly unbalanced medical images. On the PhenoBench sugar-beet dataset, the ResNet-50 version improves the combined panoptic metric $PQ^\\dagger$ from 81.48 to 81.89 and lowers leaf-counting error (RMSE over ground-truth plants from 2.35 to 2.30). The underlying motivation is practical: better leaf and weed masks mean more accurate leaf counts for growth monitoring and more targeted herbicide application.","feed_headline":"Boundary loss lifts leaf and weed segmentation to 81.89 PQ+","feed_subtitle":"Focal and boundary losses sharpen small-instance masks and leaf counts in a Mask2Former crop model.","key_machinery":"The load-bearing mechanism is the loss function on binary masks. Focal loss ($L_{\\text{focal}} = -\\alpha_t (1-p_t)^\\gamma \\log(p_t)$, with $\\gamma=2.0$, $\\alpha=0.25$) down-weights easy background pixels so that scarce foreground pixels from leaves and weeds contribute more. The boundary loss of Kervadec et al. replaces region integrals with an integral over the interface between the predicted and ground-truth regions, using a precomputed level-set distance map $\\phi_G$; its weight starts at 0.01 and grows by 0.0006 per epoch. These losses are applied to masks produced by two separate transformer decoders—one for plants, one for leaves—that share the same learnable queries, with deep supervision at every transformer layer. The loss recipe is what carries the argument: it is what the ablation varies, and it changes segmentation quality at no cost to inference speed.","core_discovery":"The paper's central claim is that on the PhenoBench benchmark, the combination of focal loss and boundary loss, added to an adapted Mask2Former with a separate transformer decoder for leaves, yields better segmentation of small instances and better leaf counts than the unmodified architecture. In the ablation on the standard 1407-image training set with a ResNet-50 backbone, adding both losses raises $PQ^\\dagger$ from 81.48 (base) to 81.81, and to 81.89 with test-time augmentation on leaf masks, while the focal-loss-only variant stays essentially flat at 81.46. Leaf-counting error measured as RMSE over all ground-truth plants falls from 2.35 to 2.30 with both losses. With a SwinL backbone the same recipe reaches $PQ^\\dagger$ of 83.62 without and 83.70 with test-time augmentation, and it outperforms the published second-place competition submission (81.33) on the same standard split, although it trails the first-place entry that trained on additional validation images.","pith_inferences":["I infer that the reported $PQ^\\dagger$ and leaf-RMSE differences, each below about half a point, should be checked against run-to-run variance; single training runs are not enough to guarantee the focal-plus-boundary recipe is the cause of the improvement.","I infer the boundary-loss idea, designed for highly unbalanced medical segmentations, is likely to transfer to other agricultural or remote-sensing tasks where small objects like weeds, fruits, or seedlings dominate the error.","I infer that leaf-count accuracy may need a counting-specific head or post-processing, since even the paper's best segmentation leaves a nontrivial RMSE and segmentation gains translate into only modest counting gains.","I infer that combining the loss recipe with a stronger backbone (for instance, the BEiT backbone used by the first-place entry) might close the remaining gap to 82.62, but this is an extrapolation from the paper's backbone comparison."],"forward_implications":["A lightweight ResNet-50 model can reach panoptic quality competitive with much heavier published systems once focal and boundary losses are added, so small-instance accuracy need not require a large model.","Leaf-count error drops with improved leaf segmentation: RMSE over ground-truth plants falls from 2.35 to 2.30, and over predicted plants from 2.11 to 2.05 for the ResNet-50 model with both losses.","Separating plant and leaf decoders lets one forward pass output both plant-level and leaf-level masks, and the leaf branch can be improved with test-time augmentation without hurting the plant branch.","Because the loss changes do not affect inference speed, the same recipe can be applied to faster architectures for field deployment."],"supporting_citations":[{"why":"Base architecture being adapted; supplies the Mask2Former model, pretrained COCO weights, point sampling, and the original mask-loss baseline.","marker":"[7]"},{"why":"Supplies the boundary loss and the level-set distance-map approximation that the paper adds to improve small-instance segmentation.","marker":"[12]"},{"why":"Supplies the focal loss that replaces cross-entropy in the mask loss to focus on hard-to-classify foreground pixels.","marker":"[15]"},{"why":"Provides the PhenoBench dataset with plant, leaf, and weed annotations used for training and evaluation.","marker":"[25]"},{"why":"Defines the crop and leaf panoptic-quality evaluation style and serves as an earlier in-field phenotyping baseline.","marker":"[26]"},{"why":"Presents the HAPT hierarchical architecture with separate plant and leaf decoders, used as a comparison baseline.","marker":"[20]"},{"why":"Provides the CVPPA competition benchmark and the first-, second-, and third-place results that the paper compares against.","marker":"[1]"},{"why":"Supplies the ResNet-50 backbone for the lightweight, inference-speed-focused variant.","marker":"[11]"},{"why":"Supplies the SwinL backbone for the high-performance variant used to reach the best reported numbers.","marker":"[16]"}],"fun_headline_variants":["Focal + boundary loss lift leaf/weed PQ to 81.89","Dual losses improve small-instance segmentation and leaf counting","Boundary loss sharpens leaf and weed masks, cutting count error","Boundary loss boosts PQ and leaf count accuracy in crop model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported improvements rest on the assumption that differences of a few tenths of a point in $PQ^\\dagger$ and 0.05 in leaf RMSE, measured from single training runs without error bars, reflect the loss functions rather than random variation.","fun_headline_variants_meta":{"raw":{"variants":["Focal + boundary loss lift leaf/weed PQ to 81.89","Dual losses improve small-instance segmentation and leaf counting","Boundary loss sharpens leaf and weed masks, cutting count error","Boundary loss boosts PQ and leaf count accuracy in crop model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001202,"raw_usage":{"total_tokens":4944,"prompt_tokens":925,"completion_tokens":4019,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":3945}},"tokens_in":541,"tokens_out":4019,"duration_ms":26653,"temperature":1.0,"reasoning_tokens":3945,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:47:53.144615+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the ResNet-50 base and the ResNet-50 focal-plus-boundary variant several times with different random seeds on the standard 1407-image split and compare the distributions of $PQ^\\dagger$ and leaf RMSE; if the intervals overlap or the ordering flips, the claimed benefit is not reproducible. A simpler check is whether the same recipe changes $PQ^\\dagger$ on the held-out test set in the same direction.","supporting_citations":[{"cited_title":"In: International conference on medical imaging with deep learning","cited_arxiv_id":null,"evidence_quote":"Supplies the boundary loss and the level-set distance-map approximation that the paper adds to improve small-instance segmentation."},{"cited_title":"arXiv preprint (2023)","cited_arxiv_id":null,"evidence_quote":"Provides the PhenoBench dataset with plant, leaf, and weed annotations used for training and evaluation."},{"cited_title":"In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","cited_arxiv_id":null,"evidence_quote":"Defines the crop and leaf panoptic-quality evaluation style and serves as an earlier in-field phenotyping baseline."},{"cited_title":"In: 2023 IEEE International Conference on Robotics and Automation (ICRA)","cited_arxiv_id":null,"evidence_quote":"Presents the HAPT hierarchical architecture with separate plant and leaf decoders, used as a comparison baseline."},{"cited_title":"https://cvppa2023.github","cited_arxiv_id":null,"evidence_quote":"Provides the CVPPA competition benchmark and the first-, second-, and third-place results that the paper compares against."},{"cited_title":"In:Proceedings of the IEEE/CVF international conference on computer vision","cited_arxiv_id":null,"evidence_quote":"Supplies the SwinL backbone for the high-performance variant used to reach the best reported numbers."}],"review_version":1}