{"id":"c7ae3e25-a4ee-4465-be88-6723d1c09753","arxiv_id":"2601.00940","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LQDS provides a 5,000-image, 14-class real-world liquid benchmark, and LQDM, a boundary-attention model, sets the current baseline with 59.28 mIoU.","lead":"The paper introduces LQDS, a 5,000-image benchmark grouping real-world liquids into 14 classes, and LQDM, a boundary-attention segmentation model that reports 59.3 mean IoU on its test set. It gives robots a first common benchmark for a messy visual task: recognizing liquids they should avoid or pour.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set hyperparameter selection (ω and backbone) likely inflates LQDM's reported 59.28 mIoU; without a validation split the SOTA margin is not trustworthy.","rationale":"The reader's weakest assumption identified the possibility that LQDM's choices (e.g., SigLIP2 backbone, ω=200) were tuned on the test set and that baselines may be undertuned. My concern zooms in on the first part: the manuscript itself shows test-set performance for different ω values in Table 5, making the test-set tuning visible rather than hypothetical. This directly undermines the reliability of the reported 59.28 mIoU and the 'outperforms SOTA' claim. The paper provides no validation set, so there is no evidence that the selected hyperparameters would generalize. This is a protocol flaw that any reviewer should flag, and it strengthens the need for the conditions the reader already imposed (releasing code, documenting validation-based selection, etc.). I therefore see no reason to change the verdict from CONDITIONAL; the concern supports the reader's assessment rather than overturning it. I also note the component analysis suggests the backbone contributes most of the gain, which further tempers the architectural claim, but the test-set tuning is the more decisive issue.","tokens_in":11571,"tokens_out":3992,"duration_ms":37444,"concrete_test":"Ask the authors to rerun the ω sweep (Table 5) and the backbone comparison (Table 4) using a held-out validation split (e.g., 10% of the training set, stratified by class) or 5-fold cross-validation on the 4200 training images. Fix ω and backbone based on validation performance, then retrain on the full training set and report test mIoU. If the test mIoU drops by more than ~1 point from 59.28, or if a different ω/backbone is selected, the reported SOTA margin is partially an artifact of test-set tuning. Additionally, retrain EoMT with the SigLIP2 backbone under the same training schedule and data augmentation as LQDM; if its mIoU approaches or exceeds 56.50 (the BL+SL component), the architectural contribution over a same-backbone baseline is small.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that LQDM outperforms state-of-the-art methods on LQDS rests on the reported test-set mIoU (59.28 vs. 50.12 for EoMT). However, the evaluation protocol is compromised by apparent test-set tuning. Section 5.3 and Table 5 present an ablation over the boundary-loss weight ω, reporting mIoU for ω ∈ {1, 10, 100, 200} and selecting ω=200 because it yields the highest mIoU. No validation set is described anywhere in the manuscript; the only held-out split mentioned is the 800-image testing set. Thus ω was effectively chosen by maximizing performance on the test set. Similarly, the backbone choice (SigLIP2 vs. DINOv2) in Table 4 appears to have been selected based on test-set performance, since the paper does not report validation results for this choice either. This makes LQDM's reported metrics in-sample estimates: the model has effectively seen the test set through hyperparameter selection. In contrast, the 15 SOTA baselines were retrained 'using their publicly available implementations and parameter settings' (Section 5.2), i.e., without any test-set-driven tuning. The comparison is therefore not apples-to-apples; the reported 9.16 mIoU lead may be partly an artifact of tuning protocol rather than architectural superiority. Moreover, the component analysis shows that simply swapping the backbone (BL→BL+SL) accounts for +6.38 mIoU, while the boundary branch (+1.41) and cross-attention (+1.37) contribute only ~2.8 mIoU combined. If EoMT were also given the SigLIP2 backbone and tuned, the remaining margin could be much smaller. This is a load-bearing concern because the paper's primary contribution—a new architecture that beats SOTA—depends on the validity of this comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces LQDS, a manually annotated real-world dataset of 5,000 images covering 14 liquid classes, and proposes LQDM, a dual-branch transformer architecture with a mask branch and a boundary branch coupled through cross-attention. The authors report that LQDM outperforms 15 existing segmentation methods on the LQDS test set, reaching 59.28 mean IoU versus 50.12 for the best competitor EoMT, and also report competitive performance on ADE20K. The dataset and code are released. The central claim is that LQDM establishes a strong baseline for liquid segmentation and that its boundary-guided cross-attention mechanism is responsible for the improvement.","tokens_in":12000,"tokens_out":3388,"duration_ms":36843,"significance":"If the empirical claims are taken at face value, LQDS is potentially the first multi-class, real-world liquid segmentation dataset, and the proposed architecture is a reasonable baseline for future work. The dataset construction, the inclusion of 15 baselines, and the external ADE20K generalization check are concrete contributions. However, the quantitative SOTA claim is currently supported only by a single test-set evaluation, with model selection performed on the test set and baselines run with default settings. Because the reported lead of 9.16 mIoU is comparable to the gain from simply changing the backbone (+6.38 mIoU in Table 4), the architectural contribution is not yet isolated. The paper's contribution is therefore better viewed as a dataset-and-baseline paper whose performance claims need a sounder evaluation protocol before they can be accepted as stated.","major_comments":[{"comment":"The SOTA claim rests on test-set hyperparameter selection. The only held-out split described anywhere is the 800-image test set; no validation split is mentioned. Table 5 reports mIoU on the test set for ω ∈ {1,10,100,200} and selects ω=200, and Table 4 selects the SigLIP2 backbone based on the same test-set metrics. Since LQDM's reported 59.28 mIoU is the result of maximizing performance on the test set, the comparison against baselines with default settings is not an apples-to-apples evaluation. Please introduce a validation split, perform hyperparameter and backbone selection on it, and report test results only after selection. Also report results across multiple seeds with error bars.","section":"§5.2, §5.3, Table 5"},{"comment":"The component analysis shows that replacing the DINOv2 backbone with SigLIP2 in the baseline (BL→BL+SL) improves mIoU from 50.12 to 56.50, an increase of 6.38; the boundary branch adds only 1.41 and the cross-attention adds only 1.37. The best competitor in Table 1, EoMT, is evaluated with its default DINOv2 backbone (§5.3 states 'For our baseline, we use EoMT, pretrained with a DINOv2 backbone as per the original implementation'). Thus the reported 9.16 mIoU advantage over EoMT may be largely a backbone advantage rather than an architectural one. To support the claim that LQDM's cross-attention design is responsible for the SOTA result, please ablate EoMT with the same SigLIP2 backbone and/or report the proposed architecture with the DINOv2 backbone.","section":"Table 4 and Table 1"},{"comment":"There is no evidence that the improvements are statistically reliable. Each configuration is reported as a single run with no variance, and several per-class IoU values are highly variable (e.g., medical 59.36 for LQDM vs 29.84 for EoMT, tea 16.09 vs 21.52). Given the small test set (800 images) and the class imbalance documented in Figure 3, the 1-2 mIoU differences attributed to the boundary branch and cross-attention may be within run-to-run noise. Please provide at least three independent runs for the key configurations and report mean and standard deviation, or, at minimum, state whether all numbers come from a single run and discuss the implications.","section":"§5.3, Table 4"},{"comment":"The comparison protocol for baselines is asymmetric. Baselines are 'retrain[ed] ... using their publicly available implementations and parameter settings' — i.e., default hyperparameters designed for other datasets — while LQDM benefits from LQDS-specific choices of ω, backbone, and training scheme. Even if a validation split were introduced, the paper should tune the baselines on that same validation split, or clearly state that the comparison is against off-the-shelf baselines, which weakens the word 'state-of-the-art' in the abstract. Without this, the reported margin in Table 1 cannot be attributed to the proposed method.","section":"§5.2"}],"minor_comments":[{"comment":"In Table 3 and Table 6 the row labeling the proposed method reads 'LQDS' instead of 'LQDM'. This is confusing because LQDS is the dataset name.","section":"Table 3, Table 6"},{"comment":"The baseline list includes both GCNet and CCNet, but the reference list only contains a CCNet entry [6]. GCNet is not cited. Please either add the correct reference or remove the method.","section":"Table 1"},{"comment":"The sentence 'Values beyond 200 is when additional performance benefits level out' is unclear and contains a grammatical error. Since Table 5 only tests up to 200, the claim about values beyond 200 is not supported by the reported experiments.","section":"§5.3"},{"comment":"The annotation section reports that images are labeled by 'students who work on related fields' but provides no inter-annotator agreement or quality metrics. For a dataset contribution, reporting at least a small agreement study (e.g., IoU between annotators on a subset) would strengthen the benchmark.","section":"§3"},{"comment":"The boundary ground-truth thickness is set to '1% of the image diagonal length' without justification or sensitivity analysis. Since the boundary branch is central to the method and the thickness is a free parameter, a brief rationale or a robustness check would be appropriate.","section":"§4.4"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about test-set hyperparameter selection is confirmed by the manuscript: no validation split is described, and Table 5 and Table 4 use test-set metrics to select ω and the backbone. This is the main obstacle to accepting the performance claim. The dataset and code release are valuable, and the paper can be made publishable by adding a proper validation protocol and isolating the architectural contribution from the backbone change. I do not see the issue as unfixable within the scope of the manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the useful part: LQDS is the first real multi-class general-liquid dataset, 5000 manually annotated images across 14 classes, with a sensible stratified split and an external ADE20K sanity check. That alone is a solid contribution for robotics and transparent/reflective segmentation research. The LQDM architecture—a mask transformer with a boundary branch and cross-attention—is a modest but legitimate extension; the ablations show both the boundary branch and cross-attention add something on top of the backbone.\n\nThe soft spot is the evaluation protocol. The paper never describes a validation split. Table 5 selects the boundary-loss weight ω by picking the value with the highest mIoU on what appears to be the test set, and the backbone choice (SigLIP2 over DINOv2) is likewise justified by 'experimentation' with no validation numbers. The 15 baselines are retrained with their default settings. So LQDM benefits from test-set-informed choices while the baselines don't. Given that the backbone swap alone accounts for +6.38 mIoU (Table 4) and the boundary branch and cross-attention together add ~2.8, the reported 9.16-point lead over EoMT is very likely inflated. The fair claim would be 'with a better pretrained backbone and tuned loss weight, this architecture family does well,' not 'our architecture beats SOTA.'\n\nAlso, no error bars, no annotation quality metric, and the code/data are 'to be released' rather than available now. Those are verification gaps, not fatal flaws.\n\nThe ADE20K result (59.6 mIoU) is a useful external check, though the parameter count is higher than the comparators and the gain over EoMT-DinoV3 is a tenth of a point.\n\nNet: the dataset deserves to be published and used. The architectural claim needs re-running with a proper validation split and better-tuned baselines before it carries weight. I'd encourage the authors to rerun with a held-out validation set and report that protocol; the dataset alone is worth a paper.\n\nFor peer review: definitely send out. The topic is timely, the dataset is reusable, and the flaws are fixable.","headline":"The LQDS dataset is a genuine gap-filler for liquid segmentation, but the SOTA claim for LQDM rests on an asymmetric tuning protocol that needs fixing before the margin is trusted.","tokens_in":12448,"tokens_out":1985,"would_cite":true,"duration_ms":20724,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dual-branch network that lets mask predictions attend to liquid-edge features sets a benchmark for segmenting liquids in real-world images.","keywords":["liquid segmentation","semantic segmentation","boundary detection","cross-attention","vision transformer","real-world dataset","transparent objects","reflective objects"],"falsifier":"Retrain the strongest baseline with the same pretrained encoder and the same boundary-supervised, high-weight schedule on LQDS and compare on the official test split; if the mIoU gap shrinks from 9.16 to near zero, the claimed advantage is tuning and backbone, not cross-attention. A complementary check: run LQDM on a set of thin, low-contrast spills where boundary discontinuities are weak; if it performs no better than the mask-only baseline, the boundary-detection premise is false.","tokens_in":11519,"feed_emoji":"💧","tokens_out":7950,"duration_ms":67032,"temperature":0.7,"pith_summary":"Liquids are hard for standard segmentation models because they deform, come in many colors, and can be transparent or reflective, so the pixels they occupy often look like whatever is behind them. The paper tries to establish two things: that general liquid segmentation deserves its own benchmark, and that boundary information is the key to getting it right. To show this, it introduces LQDS, a 5,000-image, 14-class, manually annotated collection of real-world liquids, and LQDM, a network with a mask branch and a boundary branch that use cross-attention so the mask branch can borrow features learned at liquid edges. On the LQDS test set, LQDM reports 59.28 mean IoU against 50.12 for the strongest of 15 compared methods, and it shows a gain on a general 150-class scene benchmark as evidence that boundary-guided attention is not liquid-specific.","feed_headline":"Edge-attention model beats 15 rivals at liquid segmentation","feed_subtitle":"A new 5,000-image dataset and boundary cross-attention lift mean IoU to 59.28 percent.","key_machinery":"The load-bearing mechanism is cross-attention between two sets of learnable queries. At block i the mask-branch queries Q_m are updated by attending to the boundary-branch queries Q_b at the same block, Q'_m = Q_m + CrossAttnBlock(Q_m, Q_b), letting mask predictions refine against boundary-specific features. The two branches share transformer encoder blocks, which the paper argues makes the backbone learn boundary features more deeply and saves parameters. Supervision is a weighted sum of a mask loss and a boundary binary-cross-entropy loss, with the boundary mask derived from ground truth at 1% of the image diagonal. The component analysis shows each addition—pretrained encoder weights, the","core_discovery":"The central claim is that liquid segmentation in the wild is tractable once a model is pointed at the one place where liquids disagree with their surroundings: the boundary. Liquid boundaries show sharper discontinuities in semantics and low-level texture than interiors, so LQDM trains a dedicated boundary branch and lets mask-branch queries attend to boundary-branch queries at every one of several transformer blocks. Using a recent pretrained vision encoder and a boundary-loss weight of 200, the full model reaches 59.28 mean IoU and 71.61 mean pixel accuracy on the LQDS test set, beating all 15 compared networks on both metrics and on 11 of 14 classes. The paper also positions LQDS as the f","pith_inferences":["Editorial inference: since the paper's best configuration uses a newer pretrained encoder and a boundary-loss weight selected by experiment, a fair comparison would give the same encoder and schedule to the baseline models; otherwise part of the reported lead could be representation, not the cross-attention mechanism.","Editorial inference: LQDS is dominated by liquids inside containers near the image center, so the benchmark says little about thin spills, puddles on textured ground, or extreme viewing angles; those are the natural next test splits.","Editorial inference: the boundary thickness (1% of the diagonal) and the loss weight are both knobs; varying them could raise the ceiling or reveal that the boundary branch helps mainly as an auxiliary loss rather than through cross-attention itself.","A direct test: train LQDM with cross-attention ablated but the boundary branch kept, and compare across several seeds; the paper's single ablation suggests the attention adds roughly 1.37 mIoU, but multi-seed runs would show how stable that increment is."],"forward_implications":["If the reported numbers hold, robot manipulation and navigation systems get a concrete starting point for detecting cups, spills, puddles, and containers in daily scenes.","LQDS gives the community a common 5,000-image, 14-class test bed, so future liquid-segmentation methods can be compared on the same real-world distribution instead of single-liquid or synthetic sets.","The finding that boundary-guided attention also improves masks for ordinary objects implies the mechanism may transfer to other segmentation tasks where edges are ambiguous.","Per-class results jump most for transparent or reflective classes such as wine, medical liquids, milk, and honey, suggesting boundary features matter most where the interior is unreliable.","The gap of 9.16 mean IoU over the strongest baseline is the paper's quantitative claim for the method."],"fun_headline_variants":["Boundary-aware AI sees liquids in the wild","Liquid segmentation solved with edge attention","New dataset and model help robots avoid spills","How to make robots see water, wine, and medicine","Cross-attention on boundaries boosts liquid segmentation"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that liquid boundaries reliably show detectable discontinuities in semantics and low-level features; if transparent or thin liquids erase those discontinuities, the boundary branch has nothing to attend to, and separately, that the 15 comparison methods were retrained as fairly as LQDM was tuned, since LQDM's encoder and boundary-loss weight were selected after experiments.","fun_headline_variants_meta":{"raw":{"variants":["Boundary-aware AI sees liquids in the wild","Liquid segmentation solved with edge attention","New dataset and model help robots avoid spills","How to make robots see water, wine, and medicine","Cross-attention on boundaries boosts liquid segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000149,"raw_usage":{"total_tokens":1018,"prompt_tokens":722,"completion_tokens":296,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":466,"completion_tokens_details":{"reasoning_tokens":227}},"tokens_in":466,"tokens_out":296,"duration_ms":3845,"temperature":1.0,"reasoning_tokens":227,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T12:58:36.296701+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the strongest baseline with the same pretrained encoder and the same boundary-supervised, high-weight schedule on LQDS and compare on the official test split; if the mIoU gap shrinks from 9.16 to near zero, the claimed advantage is tuning and backbone, not cross-attention. A complementary check: run LQDM on a set of thin, low-contrast spills where boundary discontinuities are weak; if it performs no better than the mask-only baseline, the boundary-detection premise is false.","supporting_citations":[],"review_version":1}