{"id":"9ab98f66-1fe5-4987-ae64-4d74ed39a585","arxiv_id":"2412.15637","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"An incremental unsupervised domain adaptation method with domain-specific adapter layers improves crack segmentation mIoU by 0.65 on source and 2.7 on target over the FADA baseline.","lead":"CrackUDA is a deep network that adapts a crack segmentation model to new image domains without requiring labels on the new images. It combines domain-invariant and domain-specific layers with adversarial training and reports small gains over one baseline, plus a new drone-collected crack dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 5.2's checkpoint selection using target-domain mIoU invalidates the reported unsupervised target improvement in Table 3.","rationale":"The reader's weakest assumption identifies exactly the load-bearing flaw: Section 5.2's checkpoint-saving criterion uses target mIoU, which requires target labels, and the reported target number is selected by that criterion. This makes the headline unsupervised target improvement invalid. I considered whether an internal typo in Eq. 8 (M2 called with phi_s1 instead of phi_s2) was more fundamental, but the checkpoint-selection leak is the decisive issue because it directly affects the reported numbers and the comparison to FADA. The paper does provide a useful dataset and a plausible architecture, but the core UDA claim cannot be accepted without fixing or reframing the evaluation protocol. Hence the reader's REJECT verdict stands unchanged.","tokens_in":11480,"tokens_out":3318,"duration_ms":28242,"concrete_test":"Retrain Step 2 under a strict UDA protocol: use target images only in the unlabeled adversarial loss; select the checkpoint using only source-domain validation mIoU (or a fixed epoch schedule); then evaluate on a held-out target test split that was never used for checkpoint selection or adaptation. If the resulting target mIoU is not above FADA's 60.73, the headline claim fails. As a secondary check, record target mIoU for all checkpoints and report mean and standard deviation to quantify the selection bias introduced by the current oracle-based checkpoint rule.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that CrackUDA improves unsupervised crack segmentation on the target domain by 2.7 mIoU over FADA. Section 5.2 states: 'For Step 2, The model checkpoints are saved only if there is an increase in mIoU scores for both the source and target domains.' Computing target-domain mIoU during training requires target ground-truth labels. Section 6.1 further states that all 358 BuildCrack images are used for training and validation, and Table 3 reports target mIoU on all 358 images. Therefore the reported 63.43 is not an unsupervised result: it is selected by oracle access to target labels over the training trajectory. FADA's 60.73 is a single run or standard checkpoint, so the comparison is not apples-to-apples. This protocol flaw directly undermines the abstract's claim of 'improvement of 0.65 and 2.7 mIoU on source and target domains respectively' as a UDA result. The method may still have value if reframed as a study with supervised model selection or a fixed-checkpoint UDA protocol, but as written the target mIoU is not a valid unsupervised domain adaptation outcome.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CrackUDA, a two-step incremental unsupervised domain adaptation (UDA) method for binary crack segmentation. The architecture combines an encoder with shared domain-invariant parameters and domain-specific residual adapters, two domain-specific decoders, a gradient reversal layer, and an adversarial discriminator. A KL-divergence loss between the old and new decoder outputs is used to reduce catastrophic forgetting. The authors also introduce BuildCrack, a 358-image drone-captured building crack dataset. Experiments on CrackSeg9K sub-datasets and BuildCrack report a source-domain mIoU of 79.83 and a target-domain mIoU of 63.43, compared with FADA's 79.18 and 60.73, and the paper claims improvements of 0.65 and 2.7 mIoU on source and target domains respectively.","tokens_in":11693,"tokens_out":2565,"duration_ms":22443,"significance":"If the reported gains were valid, the paper would offer a useful incremental-UDA recipe for crack segmentation and a new public dataset for the community. The dataset contribution and the architectural idea of combining domain-invariant encoder weights with domain-specific adapters and KL-based consolidation are potentially valuable, and the ablation study is clearly presented. However, the central unsupervised target-domain claim is undermined by the target-label-based checkpoint selection protocol described in Section 5.2. Because the headline 2.7 mIoU target improvement is obtained by selecting the checkpoint that maximizes target-domain mIoU, it is not a valid UDA result as reported. The baseline comparison is also weakened by the exclusion of five non-converged baselines. These issues are load-bearing, so the manuscript in its current form cannot support the stated conclusions.","major_comments":[{"comment":"The checkpoint selection protocol invalidates the reported unsupervised target result. Section 5.2 states that in Step 2 'model checkpoints are saved only if there is an increase in mIoU scores for both the source and target domains.' Computing target-domain mIoU during training requires access to target ground-truth labels, which violates the UDA assumption that the target domain is unlabeled. Since Section 6.1 states that all 358 BuildCrack images are used for training and validation, the reported target mIoU of 63.43 in Table 3 is selected by oracle access to target labels over the training trajectory. This makes the comparison with FADA (60.73) not apples-to-apples, and the headline '2.7 mIoU improvement on target' is not a valid unsupervised result. The authors should re-run the experiments with a protocol that does not use target labels for model selection, for example by saving checkpoints based on source validation mIoU only or by fixing the checkpoint at a predetermined epoch, and report the resulting target mIoU.","section":"Section 5.2 and Table 3"},{"comment":"The 'state-of-the-art' comparison is not statistically or methodologically solid. Five of the nine listed baselines (AdaptSegnet, ADVENT, IAST, DAFormer, CBST) are marked as not converging and are effectively excluded from the comparison, leaving FADA, MaxSquare, DACS, and ProDA. The claim of surpassing state-of-the-art therefore rests almost entirely on a single baseline (FADA), and the source-domain gain over FADA is only 0.65 mIoU, which is within the range of run-to-run variability that the paper does not quantify. The authors should either provide a serious convergence analysis for the failed baselines, report their best obtained results with the same checkpoint-selection rule, or substantially soften the 'surpasses SOTA' claim.","section":"Table 3"},{"comment":"The ablation study is affected by the same target-oracle checkpoint-selection issue. If the '2 Step' and '2 Step w/o KLD' rows are produced using the Section 5.2 rule of saving checkpoints only when both source and target mIoU increase, then the reported 9.93 mIoU drop on BuildCrack when removing LKLD may be confounded with the checkpoint-selection mechanism rather than reflecting the loss's genuine contribution. The paper should clarify whether the same checkpoint rule was used in ablations and, if so, re-run the ablations under a valid UDA selection protocol.","section":"Section 6.4, Table 4"}],"minor_comments":[{"comment":"There is a typographical error: 'without a significantdropinaccuracy' should read 'without a significant drop in accuracy'.","section":"Abstract"},{"comment":"The formula for the GRL scaling uses the symbol lambda on both sides of the equation ('lambda = 2/(1+e^{-lambda p}) - 1'), which is ambiguous; a distinct symbol such as lambda_p for the epoch-dependent value would be clearer.","section":"Equation (4)"},{"comment":"The statement that all 358 BuildCrack images are used for training and validation is confusing in a UDA setting, since validation with target labels is standard for evaluation but the paper should explicitly state that target labels are used only for final evaluation and never for checkpoint selection or adaptation; as written, it is contradicted by Section 5.2.","section":"Section 6.1"},{"comment":"SDNET2018 is listed as containing 1411 images with 0% cracks, yet the text says CrackSeg9K aggregates crack datasets with consistent labeling; this deserves a brief explanation to avoid the impression that a non-crack dataset was used as a segmentation source.","section":"Table 1"}],"recommendation":"reject","confidential_remarks":"The core problem is an evaluation-protocol flaw that is stated explicitly in the manuscript, so it cannot be dismissed as reviewer misinterpretation. The paper would need a full re-run of experiments with a valid UDA checkpoint-selection rule and a more robust baseline comparison before the claimed contribution can be assessed. Given that the authors have already released a project website and promise code, a revised version with corrected experiments could be resubmitted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate incremental-UDA design for crack segmentation and a useful new building-crack dataset, but the paper overstates its main result. Section 5.2 says checkpoints are saved only if mIoU increases on both source and target domains. Computing target mIoU requires target ground truth. Section 6.1 says all 358 BuildCrack images are used for training and validation, and Table 3 reports target mIoU on all 358. So the 63.43 target figure is a best-checkpoint number chosen with oracle access to target labels, while FADA's 60.73 is presumably a normal checkpoint. That comparison is not apples-to-apples, and the abstract's \"unsupervised\" claim does not hold as stated. The stress-test note is right.\n\nWhat is genuinely there: the two-step adapter architecture (frozen shared encoder, new domain-specific residual adapters, second decoder, KL regularization, GRL) is a real combination, and the ablation shows both LKLD and GRL contribute. BuildCrack fills a gap: 358 drone images of building facades with low contrast, shadows, and occlusions; it is comparable to CrackSeg9K subsets. Table 2's leave-one-subset-out experiments are a decent stress test, even though they inherit the same checkpoint-selection issue. Citation pattern is fine: adapter machinery follows Garg et al., and the UDA losses follow standard adversarial alignment.\n\nSoft spots beyond the protocol flaw: five of nine baselines did not converge (AdaptSegNet, ADVENT, IAST, DAFormer, CBST) and are set aside without analysis, so the \"surpasses SOTA\" claim rests mostly on a single converged baseline, FADA. The reported gains are small (0.65 and 2.7 mIoU), and there are no confidence intervals or multiple seeds. Table 2's per-subset improvements are also best-checkpoint selected. The paper does not include code or model weights, only a project page; that limits reproducibility, though the dataset is promised to be public.\n\nWho it's for: people working on crack segmentation and incremental UDA will want BuildCrack. The method is worth a second look if the checkpoint rule is corrected to a fixed schedule or source-only selection, or if the paper is reframed as a supervised-model-selection study. As written, the main claim needs revision.\n\nRecommendation: send to peer review, but with a strong prompt to the authors to fix the checkpoint protocol before acceptance. The dataset alone justifies referee time.","headline":"The adaptation machinery is plausible and the new dataset is useful, but the headline target gain is not a valid UDA number because target labels were used for checkpoint selection.","tokens_in":12260,"tokens_out":2878,"would_cite":true,"duration_ms":22237,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CrackUDA lifts crack segmentation on unlabeled building imagery by adapting an encoder-decoder with adversarial alignment, reporting 79.83 source and 63.43 target mIoU.","keywords":["crack segmentation","unsupervised domain adaptation","incremental learning","adversarial training","domain-invariant features","structural health monitoring","BuildCrack dataset","encoder-decoder network"],"falsifier":"Retrain CrackUDA with checkpoints selected solely on source validation mIoU, then evaluate once on BuildCrack and compare against FADA; if the target mIoU drops to or below FADA's 60.73, the claimed 2.7 mIoU advantage comes from label-based selection rather than the adaptation mechanism.","tokens_in":11308,"feed_emoji":"🏗️","tokens_out":2990,"duration_ms":25245,"temperature":0.7,"pith_summary":"The paper proposes CrackUDA, a two-step unsupervised domain adaptation method for crack segmentation in civil structures. The authors claim that splitting the network into domain-invariant and domain-specific parameters, adding a second decoder, and aligning features adversarially lets a model trained on labeled source images adapt to an unlabeled target domain while largely retaining source accuracy. They report 79.83 mIoU on the CrackSeg9K validation set and 63.43 mIoU on BuildCrack, a new drone-collected building crack dataset, outperforming the best baseline FADA by 0.65 and 2.7 mIoU respectively. The paper also introduces BuildCrack and shows the incremental scheme helps on held-out CrackSeg9K sub-datasets.","feed_headline":"CrackUDA adapts crack segmentation to new domains without target labels","feed_subtitle":"A two-step adversarial network with domain-specific decoders adds 2.7 mIoU on drone-captured building cracks.","key_machinery":"The central object is the encoder-decoder with a residual-adapter design: the encoder contains shared domain-invariant convolutional weights and domain-specific parallel residual adapter layers (DS-RAP) plus domain-specific batch normalization (DS-BN), while two separate decoders produce domain-specific predictions. During the second step, a KL-divergence loss between the frozen first decoder's outputs and the second decoder's outputs on source images regularizes the shared weights, and a discriminator with a gradient reversal layer forces the encoder's features to be domain-indistinguishable. This combination is what the paper credits for adapting to the target while avoiding catastrophic forgetting on the source.","core_discovery":"The central claim is that an incremental unsupervised domain adaptation framework can improve crack segmentation on a target domain without using any target labels, provided the network architecture separates domain-invariant from domain-specific parameters. In step one, a standard encoder-decoder (ERFNet backbone) is trained on labeled source images. In step two, new domain-specific adapter parameters and a second decoder are added, the first decoder is frozen, and alternating segmentation and adversarial training align source and target feature distributions through a gradient reversal layer. The authors report that this raises target mIoU on BuildCrack to 63.43 (2.7 higher than FADA) while keeping source mIoU at 79.83 (0.65 higher than FADA). They also show that removing either the KL-divergence loss on shared parameters or the adversarial alignment degrades target performance, supporting the architecture's role.","pith_inferences":["The reported target gain may not be purely unsupervised: the training protocol selects checkpoints using both source and target mIoU, which uses target labels for model selection; a strictly unsupervised version would hide target labels during selection.","The same encoder-decoder splitting could generalize to other thin-structure segmentation tasks, such as road markings or power-line wires, where domain shift is driven by surface texture and lighting.","If the checkpoint-selection leak is removed, the true unsupervised advantage might be smaller than 2.7 mIoU, though the architecture's ablations suggest the alignment losses still contribute.","The method's reliance on a fixed source dataset and one target at a time leaves open whether adapters can be stacked for multiple targets while preserving performance on all previous ones."],"forward_implications":["If the approach holds, crack segmentation models can be adapted to new image domains without annotating target images, reducing the cost of structural health surveys.","UDA methods that previously worked on driving scenes can be repurposed for thin, low-contrast structures like cracks by separating domain-invariant and domain-specific parameters.","The new BuildCrack dataset provides a publicly releasable benchmark for building crack segmentation under drone-imagery domain shift.","The incremental design suggests a path for sequentially adapting to multiple target domains without retraining from scratch, since each new domain adds its own adapter set.","The reported mIoU improvements, though modest, come on top of an already strong baseline, so even a few points matter in safety-critical infrastructure inspection."],"supporting_citations":[{"why":"Supplies the ERFNet backbone used as the encoder-decoder base for segmentation.","marker":"[38]"},{"why":"FADA is the strongest UDA baseline the paper compares against and the reference for the headline mIoU gains.","marker":"[47]"},{"why":"CrackSeg9K provides the source dataset and validation split, and its sub-datasets are used for leave-one-out experiments.","marker":"[23]"},{"why":"Provides the residual-adapter block design with domain-specific parameters that the paper adapts for its incremental encoder.","marker":"[14]"},{"why":"Founds the domain-adversarial training framework and gradient reversal layer that CrackUDA uses for feature alignment.","marker":"[13]"}],"fun_headline_variants":["CrackUDA: UDA for crack segmentation without target labels","CrackUDA: Learn crack features across domains with zero target labels","Unsupervised domain adaptation boosts crack segmentation by 2.7 mIoU","CrackUDA: Adversarial UDA improves crack segmentation on unseen domains"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The unsupervised framing assumes no target labels are used anywhere in training or model selection, but the protocol in Section 5.2 saves checkpoints only when mIoU increases on both source and target domains, which uses target ground truth to pick the final model.","fun_headline_variants_meta":{"raw":{"variants":["CrackUDA: UDA for crack segmentation without target labels","CrackUDA: Learn crack features across domains with zero target labels","Unsupervised domain adaptation boosts crack segmentation by 2.7 mIoU","CrackUDA: Adversarial UDA improves crack segmentation on unseen domains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000908,"raw_usage":{"total_tokens":3909,"prompt_tokens":954,"completion_tokens":2955,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":2873}},"tokens_in":570,"tokens_out":2955,"duration_ms":18866,"temperature":1.0,"reasoning_tokens":2873,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:13:48.898715+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain CrackUDA with checkpoints selected solely on source validation mIoU, then evaluate once on BuildCrack and compare against FADA; if the target mIoU drops to or below FADA's 60.73, the claimed 2.7 mIoU advantage comes from label-based selection rather than the adaptation mechanism.","supporting_citations":[{"cited_title":"IEEE Transactions on Intelligent Transportation Systems PP, 1–10 (10 2017)","cited_arxiv_id":null,"evidence_quote":"Supplies the ERFNet backbone used as the encoder-decoder base for segmentation."},{"cited_title":"In: The European Conference on Computer Vision (ECCV) (August 2020)","cited_arxiv_id":null,"evidence_quote":"FADA is the strongest UDA baseline the paper compares against and the reference for the headline mIoU gains."},{"cited_title":"In: Computer Vision–ECCV 2022 Workshops: Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VII","cited_arxiv_id":null,"evidence_quote":"CrackSeg9K provides the source dataset and validation split, and its sub-datasets are used for leave-one-out experiments."},{"cited_title":"In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","cited_arxiv_id":null,"evidence_quote":"Provides the residual-adapter block design with domain-specific parameters that the paper adapts for its incremental encoder."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Founds the domain-adversarial training framework and gradient reversal layer that CrackUDA uses for feature alignment."}],"review_version":1}