{"id":"6961529c-b69d-4e2b-885c-09cd6c33423a","arxiv_id":"1909.01955","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A deep network called DexiNed, trained on a new BIPED edge dataset, reports improved F-measures on edge detection benchmarks and generalization to other datasets without fine-tuning.","lead":"DexiNed, a new deep neural network for detecting edges in images, is trained from scratch on a new dataset called BIPED and produces thin, human-like edge maps. The paper claims it beats previous methods on edge detection benchmarks and generalizes to new images without retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed SOTA margins are not statistically supported because the model is selected on the test set and no error bars are reported; the MDBD generalization claim rests on a 0.002 ODS difference over RCF on 20 test images.","rationale":"The paper contributes a new architecture, a new dataset, and released code, and those are concrete contributions. The central empirical claim, however, is that DexiNed trained on BIPED outperforms established edge detectors and generalizes without fine-tuning. That claim depends entirely on the reported F-measure numbers, and the protocol does not yet support the strength of the conclusion. The reader's weakest assumption identifies the same core issue: a single random 200/50 BIPED split with design choices made on the test set. I agree, and I add the sharpest manifestation: the MDBD generalization result, which is the only cross-dataset edge-detection win, is a 0.002 ODS difference over RCF on 20 images. That difference is far too small to support 'best performance' without a significance test. The proposed check would settle whether the margins are reproducible or within noise. If the check confirms the margins, the conditional verdict could be upgraded; if not, the claims should be softened. Since the concern does not identify an internal contradiction or a definitively wrong result, it does not move the verdict, so UNCHANGED is appropriate.","tokens_in":11099,"tokens_out":6056,"duration_ms":59370,"concrete_test":"Using the released code, train DexiNed and RCF on BIPED under a protocol with a held-out validation set: use 20 of the 200 training images solely for selecting the upsampling mode and hyperparameters, then evaluate the frozen model on the untouched 50-image test set. Repeat over at least 5 random 200/50 splits and report mean ± std of ODS/OIS for DexiNed-a and RCF/BDCN, plus a paired bootstrap 95% confidence interval for the DexiNed-a minus RCF difference on each split. Additionally, on the 20 MDBD test images, compute a paired bootstrap CI for DexiNed-a minus RCF. If the BIPED CI includes 0 or the MDBD CI includes 0, the central 'best results' and generalization claims are not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 splits BIPED into 200 training/validation and 50 test images, and Section 5 says the architecture was 'carefully tuned till reach its final version.' The upsampling mode DexiNed-dc was chosen by comparing P/R curves on the test set (Fig. 6(a)). Thus the final DexiNed numbers are selected, not predicted, on this test set. Table 1(b) shows DexiNed-a at .859 ODS vs RCF .843 and BDCN .839; without repeated splits or paired intervals a .016-.020 margin on 50 correlated images may be noise. The cross-dataset claim is even thinner: Table 2 reports DexiNed-a .859 vs RCF .857 on MDBD, a 0.002 ODS difference, computed on the 20-image MDBD test subset, with no significance test. Since the paper's central claim is that a single BIPED-trained network generalizes to other edge detection datasets, this margin must be shown to be stable. The paper itself flags no limitation on this; the discrepancy between text 'DexiNed obtains the best performance in MDBD' and the table's DexiNed-f .837/.837 further complicates interpretation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DexiNed, a convolutional neural network for edge detection built from dense extreme inception blocks and trained from scratch on a newly introduced dataset, BIPED (Barcelona Images for Perceptual Edge Detection). The architecture produces six side outputs at different scales through learned upsampling blocks, and these outputs are combined into fused and averaged edge maps. Training uses a weighted cross-entropy loss (Eq. 1) with per-scale weights (Eq. 2). The authors evaluate DexiNed on BIPED, MDBD, CID, BSDS300, BSDS500, NYUD, and PASCAL, comparing against HED, RCF, CED, BDCN, and others. They claim superior ODS/OIS/AP performance on edge-detection-oriented benchmarks and that the model generalizes to other edge detection datasets without fine-tuning, while acknowledging that it underperforms on contour/boundary benchmarks where competing methods were trained on those datasets.","tokens_in":11446,"tokens_out":3766,"duration_ms":37017,"significance":"If the empirical claims are accepted, the paper makes two useful contributions: a new from-scratch architecture that avoids ImageNet pretraining and produces thin edge maps, and a new carefully annotated dataset, BIPED, with released code and data. Those strengths are real and should be credited. However, the central performance claims are empirical architecture claims, not derivations, and they currently rest on a single random split of BIPED, on design choices made using the test set, and on a very small cross-dataset margin on MDBD. The paper would be significantly strengthened by repeated-split evaluation, confidence intervals, and clearer separation between model selection and evaluation. With those additions, the contribution would be solid; without them, the reported margins over RCF and BDCN are not yet established.","major_comments":[{"comment":"The reported state-of-the-art margins on BIPED are not statistically supported because the model is selected using the test set. Section 5 states that the architecture was 'carefully tuned till reach its final version,' and Section 5.1 chooses the upsampling strategy by comparing precision/recall curves on the same 50-image test set (Fig. 6(a)). Since the final DexiNed-a and DexiNed-f results are therefore selected, not predicted, on this test set, the ODS differences in Table 1(b) (DexiNed-a .859 vs. RCF .843, BDCN .839, HED .829, CED .795) may be optimistically biased. The authors should use a held-out validation split for all design choices or report results over multiple random BIPED splits with paired confidence intervals; without this, the claim that DexiNed systematically outperforms prior methods on this benchmark is not established.","section":"§4.1, §5.1, Table 1(b)"},{"comment":"The cross-dataset generalization claim rests on a difference that is within noise. In Table 2, DexiNed-a achieves ODS .859 on MDBD versus RCF .857 on the 20-image MDBD test subset, a 0.002 margin, and the text states that 'DexiNed obtains the best performance in the MDBD dataset' even though the table reports DexiNed-f at .837/.837. With no significance test or confidence interval over the 20 test images, and with internally inconsistent numbers for DexiNed-f, the claimed improvement over RCF/HED on MDBD is not demonstrated. The authors should report per-image paired differences, bootstrap intervals, or repeated evaluation protocols, and should reconcile the DexiNed-f entries.","section":"§5.1, Table 2 (MDBD)"},{"comment":"The abstract's claim that evaluations on 'different benchmarks' show improvements is overbroad. Table 2 shows that DexiNed substantially underperforms methods trained on the corresponding data on BSDS500 (ODS .728 vs. RCF .806), NYUD (.602 vs. HED .720), and PASCAL (.475 vs. CED .726); the authors themselves attribute this to task mismatch. The improvement claim should be restricted to the edge-detection-oriented settings (BIPED and MDBD), and the paper should explicitly state that on contour/boundary benchmarks the proposed method is not competitive.","section":"Abstract, §5.1, Table 2"}],"minor_comments":[{"comment":"In Eq. (2), the notation is inconsistent: $L(W,w)$ and $w_n$ are used without clearly defining the subscript on $w$; the summand in Eq. (1) uses the unusual symbol $\\wr_n$ instead of a standard loss notation. Please clarify the variables.","section":"§3.3, Eq. (2)"},{"comment":"The augmentation description says the process 'resulted in 288 images per each 200 images,' which is unclear; please clarify whether 288 is the total augmented set per original image or per training subset, and correct the sentence.","section":"§4.1"},{"comment":"The dataset name is written as 'MDBM' in Section 4.1 and as 'MDBD' elsewhere; please use the acronym consistently throughout.","section":"§4.1, §4.2"},{"comment":"The fusion-layer initialization is described as $1/(N-1)$ without specifying the number of outputs $N$ or the rationale; please give the exact initialization and verify the denominator.","section":"§4.4"},{"comment":"There are several typos and inconsistencies: 'CCNs' in the Introduction, 'Dixe' in Section 4.4, and mixed capitalization of 'xception'/'Xception'. A careful proofreading pass is recommended.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere is my read of DexiNed (arXiv:1909.01955). The real contribution is the BIPED dataset and a cleanly implemented, from-scratch edge detector that is competitive with HED, RCF, CED, and BDCN. Retraining the baselines on BIPED is the right way to compare, and the cross-dataset evaluation on MDBD is a good instinct. If you work on edge detection, BIPED alone is worth having, and the code is public.\n\nThe soft spots are real and they sit right at the center of the paper. The model is selected on the test set: Section 5 says DexiNed was “carefully tuned till reach its final version,” and the upsampling strategy is chosen by looking at P/R curves on the 50-image BIPED test set (Fig. 6(a)). There is no separate validation split. Then the headline numbers come from one random 200/50 split with no error bars or repeated trials. On MDBD, the claimed cross-dataset win is a 0.002 ODS gap over RCF (0.859 vs 0.857), computed on 20 test images. That margin is not statistically supported. The text also says DexiNed “obtains the best performance in the MDBD dataset,” but DexiNed-f actually scores 0.837 in the same table, so only the averaged variant wins, narrowly. The abstract says the method “can be used in any edge detection task without previous training or fine tuning process,” but on BSDS500, NYUD, and PASCAL it clearly underperforms; the authors do acknowledge in the body that these are contour/boundary datasets, yet the abstract does not carry that caveat. The “first thin edge-maps” claim is not established either—nothing measures thinness, and HED/RCF also produce visually thin edges.\n\nI would not call this a takedown. The architecture is a sensible combination of dense connections, Xception-style blocks, and multi-scale supervision; training from scratch without ImageNet weights is a nice property; and the dataset is a real service to the community. The flaws are methodological, not fatal, and they are fixable: report multiple splits or bootstrapped confidence intervals, describe the tuning protocol with a proper validation strategy, and tone down the generalization claims.\n\nWho is this for? Anyone building on edge detection benchmarks or looking for an off-the-shelf edge model trained without pretraining. It deserves serious peer review—conditional acceptance at most, with the experimental protocol tightened. I would send it to a careful referee rather than desk-reject it.\n\nBest,\n[Your name]","headline":"DexiNed contributes a genuinely useful dataset and a from-scratch edge detector, but the SOTA claims rest on a single test split with no error bars and some test-set tuning, so the abstract overstates what is shown.","tokens_in":11917,"tokens_out":2184,"would_cite":true,"duration_ms":24232,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single convolutional network trained from scratch on a new edge-detection dataset, BIPED, produces thin human-like edge maps and outperforms HED, RCF, CED, and BDCN on that benchmark.","keywords":["edge detection","convolutional neural network","deep learning","dense inception network","BIPED dataset","multi-scale learning","boundary detection","ODS/OIS F-measure"],"falsifier":"Run several independent random splits of BIPED into training and test sets, and hold out a separate validation set for all design choices, especially the upsampling method. If DexiNed's ODS/OIS advantage over RCF or BDCN disappears when re-tested on untouched splits, the reported generalization claim is not robust.","tokens_in":10937,"feed_emoji":"🖼️","tokens_out":4192,"duration_ms":39239,"temperature":0.7,"pith_summary":"This paper proposes DexiNed, a deep convolutional network that predicts thin, human-plausible edge maps from color images. The authors claim that, trained only once from scratch on their newly introduced BIPED dataset, DexiNed beats four established deep edge detectors (HED, RCF, CED, BDCN) on BIPED under ODS/OIS F-measure and average precision, and generalizes to other edge-oriented datasets without fine-tuning. They also introduce BIPED, 250 carefully annotated outdoor images, as a public benchmark. The claim matters because it suggests a single, non-pretrained architecture can serve general edge detection, making the task simpler and more reproducible.","feed_headline":"One edge detector, trained from scratch, beats HED and RCF.","feed_subtitle":"DexiNed learns thin, human-like edges on a new dataset and generalizes without retraining.","key_machinery":"The load-bearing design is DexiNed's encoder plus upsampling blocks. The encoder uses six main blocks inspired by Xception, with 3x3 convolutions, batch normalization and ReLU, plus two kinds of skip connections: main connections between blocks and edge connections that average sub-block outputs to preserve edge features in deep layers. Each block's features go through an upsampling block that gradually doubles resolution using transposed convolutions until the map matches the ground-truth size, producing eight intermediate edge-maps; these are fused by a learned layer. The network is deeply supervised with HED's weighted cross-entropy loss. This combination is what the paper credits for thin edges and for avoiding the edge-feature loss that DeepEdge identified in deep layers.","core_discovery":"The central discovery claimed is that a densely connected, inception-style encoder with per-scale upsampling and deep supervision can learn edge detection from scratch, without ImageNet pretraining, provided the training set is carefully annotated. On their own BIPED test split, the averaged prediction (DexiNed-a) reaches ODS 0.859, OIS 0.867, and AP 0.905, above HED, RCF, CED, and BDCN trained on the same data. DexiNed also reports the best performance on MDBD, the other edge-oriented public dataset, while being trained only on BIPED. The qualitative claim that its edge maps are thin and visually coherent is supported by examples shown in the paper.","pith_inferences":["If the BIPED benchmark is adopted, the field should also specify multiple random splits and a validation protocol, since the paper's single split may overstate small F-measure differences.","DexiNed's weaker results on BSDS500, NYUD, and PASCAL suggest the claims are specific to edge-detection datasets; users of boundary/contour benchmarks should not expect the same gains.","A quick test would compare DexiNed against RCF and BDCN when all are trained on BIPED and evaluated on an untouched random split and on a fourth edge dataset.","The no-fine-tuning generalization might depend on annotation density of the target dataset; test on sparsely annotated data with per-pixel precision/recall."],"forward_implications":["A single DexiNed trained once on BIPED can be dropped into edge-detection applications without per-dataset training or fine-tuning, at least for edge-oriented data.","The BIPED dataset gives the community a comparable benchmark where edge detectors can be trained from scratch and evaluated on ODS/OIS/AP.","Thin, visually plausible edge maps could improve downstream tasks like photo sketching, segmentation, or image-to-image translation.","Training from scratch removes the dependency on ImageNet-pretrained backbones for edge detection, enabling more compact or task-specific training pipelines."],"supporting_citations":[{"why":"Supplies the holistically-nested multi-scale architecture and weighted cross-entropy loss that DexiNed adapts.","marker":"[36]"},{"why":"Inspires the six-main-block encoder structure with separable-convolution style blocks.","marker":"[7]"},{"why":"State-of-the-art baseline trained on BIPED for comparison; also a VGG-based multi-scale edge detector.","marker":"[20]"},{"why":"Bidirectional cascade network used as a comparison baseline trained on BIPED.","marker":"[14]"},{"why":"Deep crisp boundaries baseline trained and evaluated on BIPED.","marker":"[34]"},{"why":"MDBD is the pre-existing edge-detection dataset used for cross-dataset evaluation.","marker":"[23]"},{"why":"Defines BSDS datasets and the ODS/OIS F-measure evaluation methodology used throughout.","marker":"[3]"},{"why":"Provides the observation that deep layers lose edge features, motivating DexiNed's skip and edge connections.","marker":"[4]"}],"fun_headline_variants":["DexiNed edges: thin, from scratch, no fine-tuning","No pretraining, no fine-tuning: DexiNed wins edge detection","New BIPED dataset helps edge detector generalize","Thin edge maps without ImageNet: DexiNed leads","From scratch edge detection beats pretrained nets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The performance advantage rests on the assumption that the single random 200/50 split of BIPED, with no separate validation set, gives stable, unbiased F-measure estimates; the paper selected the upsampling strategy using the same test set, so the final numbers may be optimistically chosen.","fun_headline_variants_meta":{"raw":{"variants":["DexiNed edges: thin, from scratch, no fine-tuning","No pretraining, no fine-tuning: DexiNed wins edge detection","New BIPED dataset helps edge detector generalize","Thin edge maps without ImageNet: DexiNed leads","From scratch edge detection beats pretrained nets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1387,"prompt_tokens":798,"completion_tokens":589,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":414,"completion_tokens_details":{"reasoning_tokens":513}},"tokens_in":414,"tokens_out":589,"duration_ms":6139,"temperature":1.0,"reasoning_tokens":513,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:03:39.734524+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run several independent random splits of BIPED into training and test sets, and hold out a separate validation set for all design choices, especially the upsampling method. If DexiNed's ODS/OIS advantage over RCF or BDCN disappears when re-tested on untouched splits, the reported generalization claim is not robust.","supporting_citations":[{"cited_title":"Xie and Z","cited_arxiv_id":null,"evidence_quote":"Supplies the holistically-nested multi-scale architecture and weighted cross-entropy loss that DexiNed adapts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Inspires the six-main-block encoder structure with separable-convolution style blocks."},{"cited_title":"Liu, M.-M","cited_arxiv_id":null,"evidence_quote":"State-of-the-art baseline trained on BIPED for comparison; also a VGG-based multi-scale edge detector."},{"cited_title":"Bi-Directional Cascade Network for Perceptual Edge Detection","cited_arxiv_id":"1902.10903","evidence_quote":"Bidirectional cascade network used as a comparison baseline trained on BIPED."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Deep crisp boundaries baseline trained and evaluated on BIPED."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MDBD is the pre-existing edge-detection dataset used for cross-dataset evaluation."},{"cited_title":"Arbelaez, M","cited_arxiv_id":null,"evidence_quote":"Defines BSDS datasets and the ODS/OIS F-measure evaluation methodology used throughout."},{"cited_title":"Bertasius, J","cited_arxiv_id":null,"evidence_quote":"Provides the observation that deep layers lose edge features, motivating DexiNed's skip and edge connections."}],"review_version":1}