{"id":"1aa760a0-54b0-446b-be30-2c8ac89ec82e","arxiv_id":"1908.07957","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"DISCo combines segment-wise temporal correlations, a U-Net that predicts pixel affinities, and signed-graph partitioning to achieve top average F1 scores on the Neurofinder calcium imaging benchmark.","lead":"This paper presents DISCo, a deep learning pipeline that finds individual cells in calcium imaging videos by combining temporal correlations between pixel signals with shape information from an average image. It reports top average F1 scores on the Neurofinder public benchmark for calcium imaging cell segmentation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Universal 'outperforms all' claim rests on incomplete leaderboard: the paper excludes Mask R-CNN and human-label submissions without verifying their Neurofinder training status, and the 0.01 margin over 3dCNN lacks variance or significance testing.","rationale":"Good-faith reading: the paper contributes a sound pipeline; the lesion study and public code are real evidence. The central claim to stress is the universal benchmark claim, not the internal architecture. The weakest point is not internal correctness but the completeness of the comparison. The paper explicitly excludes two leaderboard entries (Section 2 footnote) because their training procedures are unknown. For a claim of 'outperforms all other methods trained on these datasets,' an unknown entry cannot be excluded: the burden is to show it is not a Neurofinder-trained method. The 0.01 margin over 3dCNN compounds this: with no seeds or error bars, and per-video differences that are not significant under a paired test, the word 'outperforms' is stronger than the evidence supports. The abstract's attribution to single-model DISCo (0.63) rather than DISCos (0.67) is an additional overstatement, though the full text is clearer. The reader's weakest assumption identified leaderboard comparability generally; I agree, and the exclusion of Mask R-CNN is the concrete, checkable gap that directly threatens the universal claim. A metadata check on the Neurofinder leaderboard would settle it. If the check fails, the central claim should be revised to 'DISCos matches or slightly exceeds the best documented Neurofinder-trained methods,' which is still a useful result. Therefore the verdict should remain CONDITIONAL; no adjustment to the reader's verdict is needed.","tokens_in":14479,"tokens_out":7169,"duration_ms":62912,"concrete_test":"Query the archived Neurofinder leaderboard and its submission metadata (via the CodeNeuro repository or the challenge organizers) to determine whether the excluded entries Mask R-CNN and human-label were trained on Neurofinder training data, and record their average F1 scores. If either is Neurofinder-trained and has average F1 >= 0.67, the central claim is refuted; if they are not Neurofinder-trained or score below 0.67, the comparison set is complete and the claim is provisionally intact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim (Section 1 and Table 3) is that DISCos outperforms all other methods trained on the Neurofinder datasets. This is a universal claim over the leaderboard, but the paper's own Section 2 footnote excludes two leaderboard submissions, Mask R-CNN and human-label, because 'we have no information on the used models and training procedures.' Since the claim is explicitly about all methods trained on the Neurofinder datasets, an entry whose training data are unknown cannot be assumed away. If Mask R-CNN was trained on Neurofinder and scores at or above 0.67 average F1, the headline claim is false. Even among the included methods, the margin over 3dCNN is 0.01 (0.67 vs 0.66) from single runs with no variance or significance test; a paired Wilcoxon test on the per-video F1 scores in Table 3 gives p > 0.05, so 'outperforms' is not statistically supported. The abstract further attributes the result to single-model DISCo, which actually scores 0.63 (Table 3), below 3dCNN's 0.66. The load-bearing gap is therefore that the comparison set is not demonstrated to be complete, and the reported advantage is within run-to-run noise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents DISCo, a calcium-imaging cell segmentation method that computes segment-wise Pearson correlations between pixels, feeds these together with a mean intensity projection into a two-stage CNN that predicts pairwise affinities and a foreground/background map, and extracts instances by partitioning the resulting signed graph with the GASP algorithm. The method is evaluated on the Neurofinder public benchmark and compared with leaderboard entries, with additional lesion studies on input modalities and temporal aggregation. The main reported results are an average F1 of 0.63 for the single-model configuration DISCo and 0.67 for the per-series configuration DISCos, with the paper claiming that DISCos outperforms all other methods trained on the Neurofinder datasets.","tokens_in":14763,"tokens_out":7636,"duration_ms":69552,"significance":"If supported, the contribution would be valuable: DISCo offers a computationally cheaper alternative to 3D-CNN-on-video approaches while retaining temporal information, and it integrates instance segmentation directly. The code is publicly available, the evaluation is on an external public benchmark, and the lesion studies in Tables 4 and 5 support the main design choices. However, the central 'outperforms all' claim is not established as stated: it is misattributed in the abstract, it rests on an incomplete leaderboard comparison, and the 0.01 margin over 3dCNN is within the noise of single-run evaluations. With those claims corrected or tempered, the paper's contribution is solid and reproducible.","major_comments":[{"comment":"The abstract states that 'DISCo outperforms all existing models trained on these datasets,' but Table 3 shows the single-model DISCo at average F1 0.63, below 3dCNN's 0.66; the 0.67 average that tops the listed Neurofinder-trained methods belongs to DISCos, the per-series configuration. Section 1 itself states the claim correctly for DISCos. The abstract and any unqualified excerpts should be revised so the top result is attributed to DISCos, and the Table 3 caption's statement about a 'single model' should be limited to comparisons among single-model methods.","section":"Abstract and Section 1"},{"comment":"The claim that DISCos 'outperforms all other methods trained on the Neurofinder datasets' is a universal claim over the leaderboard, but the comparison set is not demonstrated to be complete. The footnote in Section 2 excludes the Mask R-CNN and human-label submissions because no information is available on their models and training procedures; if either of these entries was trained only on Neurofinder training data and scored at least 0.67, the stated claim would be false. Since the leaderboard snapshot was taken on 2019-11-15, the claim should be restricted to the listed methods with verified training procedures and to that snapshot, or the training status of the excluded entries should be documented.","section":"Section 2 (footnote) and Table 3"},{"comment":"The reported advantage over the closest competitor 3dCNN is 0.01 in average F1 (0.67 vs. 0.66), based on single runs with no variance estimate and no significance test. A paired Wilcoxon signed-rank test on the nine per-video F1 scores in Table 3 yields p > 0.05, so the word 'outperforms' is not statistically supported for this comparison. The authors should either provide variance across multiple training runs with a significance test, or replace 'outperforms' by 'is competitive with' for comparisons involving 3dCNN. This is load-bearing because the headline claim rests on this margin.","section":"Section 4, Table 3"}],"minor_comments":[{"comment":"The word 'segmentaiton' in the second paragraph should be 'segmentation'.","section":"Section 1"},{"comment":"In the DISCos/max row, '0,78' uses a comma as decimal separator; use '0.78' for consistency.","section":"Supplementary Table 7"},{"comment":"The figure shows C=3 neighbors while the text uses C=15; add a note that the figure is illustrative.","section":"Figure 2 and Section 3.1"},{"comment":"The criterion for background exclusion, that the background prediction is higher than the foreground prediction, is equivalent to a 0.5 threshold on the foreground channel; state this explicitly.","section":"Section 3.3"},{"comment":"The exact p-values for the two-sided Wilcoxon tests against UNet2DS and Conv2D are not reported; give them in the text or in the table.","section":"Section 4"},{"comment":"Because the Neurofinder leaderboard is time-varying, the access date should be repeated in the caption of Table 3 rather than only in the body text.","section":"Table 3"},{"comment":"Indicate that the missing standard deviations for some rows reflect series with only one training video, or explain why they are absent.","section":"Supplementary Table 8"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's components are technically sound, and the requested revision is within scope: the main issues are claim calibration, comparison completeness, and significance testing rather than correctness of the method. The editor may also wish to ask the authors to provide a version-stamped list of leaderboard entries considered, so that the 'outperforms all' claim is reproducible in the future."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nThe key thing to know: this is a solid, clearly written methods paper with a genuinely new pipeline, and the lesion study earns its keep. But the headline claim in the abstract is slightly ahead of the evidence. The single-model method called DISCo scores 0.63 average F1; the 0.67 that \"outperforms all\" comes from DISCos, the per-series model ensemble. That distinction is in the body, but the abstract blurs it.\n\nWhat's actually new: the combination of segment-wise temporal correlations, aggregated by a small 3D CNN, merged with a mean projection, fed to a U-Net that predicts pixel affinities and foreground, then partitioned with GASP. Each ingredient has lineage, but the integration is new and the ablation shows it matters. The two CNNs are trained end-to-end, the code is public, and the single-video training experiment is an honest probe of practical utility. The lesion study also shows CNN aggregation beats hand-crafted statistics like scaled correlation.\n\nThe soft spots are about the comparison, not the method. First, the abstract attributes the win to DISCo, while the table shows the per-series DISCos is the model that reaches 0.67. Second, the claim is universal over the Neurofinder leaderboard, but the paper excludes Mask R-CNN and human-label entries because their training is unknown. That is a real gap: the claim covers all methods trained on Neurofinder, and an excluded entry could invalidate it. Third, the margin over 3dCNN is 0.01 (0.67 vs 0.66) from single runs, with no error bars or significance test. A quick paired Wilcoxon on the per-video F1 scores gives p>0.05, so \"outperforms\" is not statistically supported. I'd like to see the claim softened to \"competitive with\" or repeated training runs to estimate variance.\n\nNone of this sinks the paper. The core engineering is sound, the evaluation is external, and the write-up is honest about limitations. The paper is useful for anyone building cell segmentation tools for calcium imaging and for benchmarking against Neurofinder. It deserves a serious referee; the revisions I'd ask for are tightening the abstract, narrowing the comparison claim, and either adding error bars or rephrasing the 0.01 edge.\n\nRecommendation: send it to peer review with a request for those revisions.","headline":"Solid methods paper with a real pipeline advance, but the 'outperforms all' claim is slightly ahead of the evidence.","tokens_in":15289,"tokens_out":4113,"would_cite":true,"duration_ms":36250,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that DISCo, by combining segment-wise pixel correlations with a summary image and partitioning a signed graph of learned affinities, outperforms all other methods trained on the Neurofinder calcium-imaging datasets.","keywords":["calcium imaging","cell segmentation","instance segmentation","pixel affinities","correlation clustering","deep learning","signed graph partitioning","Neurofinder benchmark"],"falsifier":"Rerun the released DISCo code on the Neurofinder training set, train per-series models with several random seeds, and submit to the Neurofinder test set; the claim fails if the average F1 does not reliably stay above 0.66, the score reported for 3dCNN.","tokens_in":14273,"feed_emoji":"🧠","tokens_out":8292,"duration_ms":67350,"temperature":0.7,"pith_summary":"The paper aims to establish that a cell-segmentation method for calcium-imaging videos can outperform existing models on the public Neurofinder benchmark by combining two complementary cues: temporal correlations between pixels and a static shape-based summary image. Rather than classifying foreground pixels, DISCo predicts pairwise affinities between nearby pixels and extracts cells by partitioning a signed graph built from those affinities. The main numerical claim is that per-dataset-series training (DISCos) achieves the highest average F1 (0.67) among methods trained only on the Neurofinder training data, slightly ahead of the 3dCNN baseline (0.66). Lesion studies support the core design: neither correlations alone (0.50) nor summary images alone (0.49) match the combination, and segment-wise correlations with a learned aggregator beat whole-video correlations and fixed statistics.","feed_headline":"Correlation-based network tops Neurofinder cell segmentation","feed_subtitle":"Using segment-wise pixel correlations plus a summary image, DISCo's per-series model hits a 0.67 average F1.","key_machinery":"The load-bearing object is the predicted pixel-affinity field combined with the GASP signed-graph partitioner. Affinities $a_{ij}\\in[0,1]$ are converted to edge weights $w_{ij}=a_{ij}-0.5$, so positive weights attract pixels into the same cluster and negative weights separate them; GASP then merges clusters while average linkage is positive. This turns instance segmentation into correlation clustering, solved approximately by a greedy agglomerative algorithm. The other essential component is the input representation: segment-wise Pearson correlations ($N=10$ segments, $C=15$ neighbor offsets) aggregated by a 3D CNN, concatenated with a mean-intensity projection.","core_discovery":"DISCo treats cell segmentation in calcium-imaging videos as a graph-partitioning problem on learned pixel affinities. The video is split into ten temporal segments; for each pixel, Pearson correlations are computed against fifteen neighboring pixels in each segment, producing a stack that a small 3D convolutional network aggregates. This aggregated correlation map is concatenated with the mean-intensity projection over the whole video and passed to a U-Net that predicts, for each pixel, a foreground/background score and affinities to nearby pixels. Affinities are converted to signed edge weights ($w_{ij}=a_{ij}-0.5$) and background edges are removed before the GASP algorithm partitions the graph into cell instances. On the Neurofinder test set, the paper reports that the per-series-trained variant (DISCos) achieves an average F1 of 0.67, outperforming all other methods trained only on the Neurofinder datasets, with 3dCNN at 0.66; the single-model variant (DISCo) reaches 0.63.","pith_inferences":["None of the comparisons reports variance across training runs; given the 0.01 margin over 3dCNN, an obvious extension is to retrain DISCos with multiple seeds and report the distribution, which would tell whether the outperformance is systematic or within noise.","The affinity-plus-graph-partitioning recipe is agnostic to the imaging modality; the same signed-graph clustering of learned affinities has been used for other instance-segmentation tasks, so feeding temporal correlation features into that pipeline could transfer to video instance segmentation of other dynamic objects.","The authors do not analyze per-dataset precision/recall trade-offs; a testable extension would be to calibrate the background-edge exclusion threshold per dataset series, since the paper reports that recall limits series 00 and precision limits series 01.","Because the summary image carries the inactive cells, DISCo is positioned for all-cell segmentation; a natural next experiment is to evaluate it on datasets labeled specifically for active cells to see whether the correlation pathway can be dropped without loss."],"forward_implications":["A single DISCo model, trained once on all five Neurofinder dataset series, attains a 0.63 average F1 on the test set, showing that one network can handle heterogeneous recordings without per-series tuning.","Per-series training (DISCos) raises the average F1 to 0.67 and edges out the previous best Neurofinder-trained method (3dCNN at 0.66), supporting the claim that the correlation-plus-shape design is competitive with spatiotemporal deep learning.","The single-video training experiment, with a 0.58 average F1 on test videos from the same series, implies that a neuroscience lab could label one video for a new recording setup and get usable segmentation without retraining on large data.","The lesion study's ordering of inputs implies that future systems should treat temporal correlations and shape summaries as complements rather than alternatives; dropping either cue costs roughly 0.13-0.14 in average F1."],"supporting_citations":[{"why":"Supplies the Neurofinder public benchmark and leaderboard that define the evaluation and the comparison scores.","marker":"[5]"},{"why":"Provides the correlation-space baseline HNCcorr and the idea of using pixel correlations, along with the combined HNCcorr+Conv2D baseline.","marker":"[6]"},{"why":"STNeuroNet/3dCNN is the strongest competitor trained on the Neurofinder datasets, the baseline that DISCos must beat.","marker":"[26]"},{"why":"Defines segment-wise (scaled) correlations, the temporal features that DISCo uses as input.","marker":"[30]"},{"why":"GASP, the signed-graph partitioning algorithm that produces the final cell instances from predicted affinities.","marker":"[39]"},{"why":"U-Net architecture used as the second network that maps aggregated correlations and summary image to affinities.","marker":"[35]"},{"why":"Source of the Dice-loss affinity learning precedent and the linkage-criterion discussion used in training.","marker":"[38]"}],"fun_headline_variants":["DISCo: correlation graph partitioning tops Neurofinder segmentation","DISCo: correlation-based graph cuts beat Neurofinder bench","DISCo: learned pixel correlations map cells, top Neurofinder","Correlation graph segmentation tops calcium imaging benchmark","DISCo: correlation-driven partition beats Neurofinder F1 0.67"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central outperformance claim depends on the Neurofinder leaderboard scores for other methods being current, accurate, and directly comparable; with a winning margin of only 0.01 average F1 over the runner-up 3dCNN, stale leaderboard entries or slight differences in training data would erase the claimed advantage.","fun_headline_variants_meta":{"raw":{"variants":["DISCo: correlation graph partitioning tops Neurofinder segmentation","DISCo: correlation-based graph cuts beat Neurofinder bench","DISCo: learned pixel correlations map cells, top Neurofinder","Correlation graph segmentation tops calcium imaging benchmark","DISCo: correlation-driven partition beats Neurofinder F1 0.67"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001041,"raw_usage":{"total_tokens":4361,"prompt_tokens":913,"completion_tokens":3448,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":3380}},"tokens_in":529,"tokens_out":3448,"duration_ms":24140,"temperature":1.0,"reasoning_tokens":3380,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:53:49.618342+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the released DISCo code on the Neurofinder training set, train per-series models with several random seeds, and submit to the Neurofinder test set; the claim fails if the average F1 does not reliably stay above 0.66, the score reported for 3dCNN.","supporting_citations":[{"cited_title":"Neurofinder public benchmark","cited_arxiv_id":null,"evidence_quote":"Supplies the Neurofinder public benchmark and leaderboard that define the evaluation and the comparison scores."},{"cited_title":"Chet- tih, Matthias Minderer, Christopher Harvey, and Dorit S","cited_arxiv_id":null,"evidence_quote":"Provides the correlation-space baseline HNCcorr and the idea of using pixel correlations, along with the combined HNCcorr+Conv2D baseline."},{"cited_title":"Fast and robust active neu- ron segmentation in two-photon calcium imaging using spa- tiotemporal deep learning","cited_arxiv_id":null,"evidence_quote":"STNeuroNet/3dCNN is the strongest competitor trained on the Neurofinder datasets, the baseline that DISCos must beat."},{"cited_title":"Scaled correlation analysis: a better way to compute a cross-correlogram","cited_arxiv_id":null,"evidence_quote":"Defines segment-wise (scaled) correlations, the temporal features that DISCo uses as input."},{"cited_title":"U-net: Convolutional networks for biomedical image segmentation","cited_arxiv_id":null,"evidence_quote":"U-Net architecture used as the second network that maps aggregated correlations and summary image to affinities."},{"cited_title":"Ham- precht","cited_arxiv_id":null,"evidence_quote":"Source of the Dice-loss affinity learning precedent and the linkage-criterion discussion used in training."}],"review_version":1}