{"id":"c91a075b-e3b2-4c22-b97a-23c242190d73","arxiv_id":"2505.23637","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Concatenating per-barcode feature vectors outperforms aggregating persistence barcodes before vectorization for TDA-based medical image classification in most tested settings.","lead":"To combine topological features from multiple slices or views of a medical image, practitioners can merge the raw persistence barcodes first or merge the feature vectors afterwards. This paper tests both strategies on CT, ultrasound, and mammography data and finds that concatenating the feature vectors usually gives higher classification accuracy than aggregating the barcodes.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Table 8 contradicts the 'almost all metrics' conclusion: for KiTS19 with VR filtration, barcode aggregation wins 3 of 5 vectorizations, and single-split best-of selection makes the preference claim premature.","rationale":"I read the paper as a practical empirical benchmark: given multiple persistence barcodes per image, should one aggregate barcodes before vectorization or concatenate per-barcode vectors? The central claim is the preference for concatenation. The strongest support would be consistent, statistically robust wins across datasets, filtrations, and vectorizations. That support is not present. The results section itself concedes one instance of no difference, but supplementary Table 8 reveals several configuration-level wins for aggregation, so the summary overstates consistency. The single-split, best-of-selection design compounds this: with roughly 40 test patients and no repeated splits, a 5-10 point gap is within sampling noise. The reader's CONDITIONAL verdict already captures the need for repeated splits and confidence intervals. My review agrees with that direction but identifies an even more concrete defect: the paper's own tables contradict the 'almost all metrics' claim. Therefore I recommend the verdict stays CONDITIONAL; the authors should add repeated splits and revise the conclusion to a configuration-dependent claim. I found no issue with the mathematical definitions or the reproducibility of the TDA pipeline; the code is listed and the methods are standard. The concern is about evidence strength and internal consistency, not the underlying method.","tokens_in":13413,"tokens_out":7027,"duration_ms":71532,"concrete_test":"Run a script that parses Tables 5-10 and counts, for every dataset-filtration-vectorization row, whether accuracy and F1 favor aggregation or concatenation. This will verify the 'except one instance' statement directly. Then rerun the full pipeline with 100 random stratified 80/20 splits (fixed seeds, same LASSO feature selection and PyCaret/Optuna protocol) and record paired per-cell accuracy differences. Compute 95% bootstrap confidence intervals for the mean difference and a sign test on the per-cell win rate. If the win rate is not significantly above 0.5 or the intervals include zero overall, the conclusion should be that aggregation and concatenation are comparable, with concatenation preferred only in specific dataset-filtration-vectorization settings.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that the reported test-set comparisons support the general conclusion that concatenation is preferred. Two problems combine. First, Section 3 states that concatenation outperformed aggregation across 'almost all performance metrics except in one instance,' but Table 8 shows three clear aggregation wins for KiTS19 with landmark-based VR filtration: Betti curve accuracy 0.775 vs 0.750, entropy summary 0.825 vs 0.750, and persistent landscapes 0.800 vs 0.725, with the best overall accuracy tied at 0.825. Those differences are 2.5-7.5 percentage points on a test set of roughly 40 patients, i.e., 1-3 patients. Second, the headline comparisons in Tables 3-4 are maxima over five vectorizations and up to 19 classifiers evaluated on a single 80/20 split; no confidence intervals, bootstrap, or repeated splits are reported. Selecting the best configuration post hoc on one split can manufacture apparent advantages. Therefore the central 'preferred approach' claim is not statistically established and is internally contradicted by the paper's own supplementary results.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports an empirical comparison of two ways to turn multiple persistence barcodes from a medical image into a single input for classifiers: (i) aggregate all barcodes and vectorize once, or (ii) vectorize each barcode and concatenate the resulting feature vectors. It applies this comparison to four medical imaging datasets, two filtration constructions, five barcode vectorizations, and 19 classifiers, and reports test-set accuracy, AUC, recall, precision, and F1 for the best configuration of each strategy. The authors conclude that feature concatenation is preferable and recommend it for similar experiments. The paper includes per-vectorization supplementary tables and a link to the experiment code.","tokens_in":13690,"tokens_out":5688,"duration_ms":51878,"significance":"If the headline conclusion were statistically supported, the paper would offer a practical rule for TDA practitioners. The study has real strengths: multiple datasets spanning CT, ultrasound, and mammography; two distinct topological constructions; five vectorizations; and public code. However, the central claim is currently overreaching: the comparison is built on a single train/test split with post-hoc selection of the best vectorization and classifier, no uncertainty quantification is provided, and the supplementary tables contain a clear counterexample to the \"almost all metrics\" statement. With the test sets for the 3D datasets containing roughly 40 patients each, the reported effect sizes are often within the range of a few patients. This is a useful benchmarking study that needs a more careful statistical analysis and a more conditional conclusion.","major_comments":[{"comment":"The headline comparison selects the best vectorization and classifier for each strategy on the same 80% training split and evaluates once on the reserved 20%. No confidence intervals, bootstrap, repeated splits, or multiplicity control are reported. Since the 3D test sets contain roughly 40 patients, a 2.5–7.5 percentage point difference corresponds to one to three patients, so the magnitudes in Tables 3 and 4 cannot be distinguished from noise without a paired test (e.g., McNemar for accuracy or DeLong for AUC). The conclusion that concatenation is \"preferred\" is not statistically established by the present design.","section":"Section 2.6.2, Tables 3–4"},{"comment":"The text states that feature concatenation outperformed aggregation on almost all metrics \"except in one instance,\" but Table 8 contradicts this. For KiTS19 with landmark-based VR filtration, barcode aggregation achieves higher accuracy than concatenation for four of the five vectorizations: Betti curve 0.775 vs 0.750, entropy summary 0.825 vs 0.750, persistent statistics 0.825 vs 0.800, and persistent landscapes 0.800 vs 0.725. The per-vectorization comparison therefore does not support a blanket preference, and the conclusion should be conditional on dataset, filtration, and vectorization.","section":"Section 3 vs. Table 8"},{"comment":"The phrase \"metrics show no statistically significant differences\" appears without any statistical test. No p-value, confidence interval, or test name is given. Either supply appropriate paired tests (McNemar for accuracy, DeLong for AUC) or remove the significance language; otherwise the sentence is an unsupported claim.","section":"Section 3"}],"minor_comments":[{"comment":"The BUSI dataset is described as a balanced subset of 376 patients in the text, but Table 1 reports 362 images; please reconcile the numbers.","section":"Section 2.5 and Table 1"},{"comment":"The Persistent Tropical Coordinates method uses a parameter r, but the manuscript does not state the value used for r in the experiments; please specify it, as the Betti grid size gamma is given but the tropical r is not.","section":"Section 2.4"},{"comment":"These tables report only the best classifier for each vectorization; including the number of test samples and the class balance per dataset would help readers assess the precision of the reported percentages.","section":"Tables 5–10"},{"comment":"Several typographical errors should be corrected: the title has \"THEEFFECTS\", the table headers contain \"Barcod Agg.\", and \"V ectorization\" appears in the Section 2.4 heading.","section":"Throughout"},{"comment":"The paper reports that the top three classifiers were selected by 5-fold CV and then fine-tuned with Optuna, but the hyperparameter search spaces and the exact criterion used to pick the final model are not specified; this limits reproducibility despite the code link.","section":"Section 2.6.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's core weakness is statistical, not methodological. I would encourage the editor to require a revision in which the authors re-analyze with repeated CV or bootstrap (or at least paired tests), address the multiplicity induced by selecting among 5 vectorizations and 19 classifiers, and revise the abstract and conclusion to be conditional on dataset, filtration, and vectorization. The supplementary tables are informative and should be preserved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a useful benchmark, but the headline conclusion overreaches its own evidence. The paper compares two ways of combining multiple persistence barcodes from a single medical image: aggregate the barcodes then featurize, or featurize each barcode then concatenate. As far as I know, this is the first systematic comparison of these two strategies in medical imaging, and the scope is genuinely solid: four datasets, two filtration types, five vectorizations, 19 classifiers, and code on GitHub. If you work on TDA pipelines for medical images, this is worth having on the shelf.\n\nThe problem is the conclusion. The abstract and conclusion say feature concatenation \"yields better classification performance and is therefore a preferred approach.\" The paper's own Table 8 contradicts that for KiTS19 with VR filtration: aggregation wins on four of five vectorizations by accuracy (Betti 0.775 vs 0.750, entropy 0.825 vs 0.750, persistence statistics 0.825 vs 0.800, landscapes 0.800 vs 0.725), with concatenation only taking PTC. The text says the two are \"analogous\" there, which is honest, but the abstract still tells practitioners to prefer concatenation. The broader pattern is also mixed: on liver-cubical the win is real, but on liver-VR and KiTS19 it is a near tie, with aggregation winning several individual configurations.\n\nThe evaluation design can't support a strong preference claim either. Everything rests on a single 80/20 split, and the headline numbers are maxima over five vectorizations and up to 19 classifiers after a 5-fold CV selection step. No confidence intervals or repeated splits are reported. The differences are often a few patients — the KiTS19 test set is about 40 patients — so the margin could easily flip.\n\nWhat's genuinely valuable here is the careful description of the two streams and the reproducible code. The paper deserves a serious referee because the question matters and the data collection is substantial. But the referee should push for repeated splits and confidence intervals, or for a conclusion that acknowledges the winner depends on dataset, filtration, and vectorization. As it stands, the evidence supports \"try both; the choice matters less than other pipeline decisions\" rather than \"concatenate.\"","headline":"Useful systematic benchmark, but the headline preference for concatenation over aggregation is not supported by the paper's own supplementary tables.","tokens_in":14129,"tokens_out":3264,"would_cite":true,"duration_ms":28989,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"When a medical image yields multiple persistence barcodes, featurizing each barcode individually and concatenating the vectors outperforms aggregating the barcodes first.","keywords":["Topological data analysis","Persistent homology","Persistence barcodes","Barcode aggregation","Feature concatenation","Medical image classification","Barcode vectorization"],"falsifier":"Run the same pipelines on the same datasets with repeated stratified cross-validation, for example 10 repeats of 5-fold CV, and count how often aggregation beats concatenation; if the win rate approaches 50 percent across folds, the claimed advantage is a selection artifact rather than a property of the combination strategy.","tokens_in":13259,"feed_emoji":"🩻","tokens_out":5088,"duration_ms":44316,"temperature":0.7,"pith_summary":"This paper asks how to combine multiple persistence barcodes computed from one medical image before classification: merge all barcodes into one summary and featurize that, or featurize each barcode and concatenate the resulting vectors. Across four datasets — liver CT, kidney CT, breast ultrasound, and mammograms — and two barcode-construction methods, the paper reports that concatenation consistently gave higher accuracy, AUC, and F1 scores than aggregation. The finding matters because many TDA pipelines that analyze 3D scans slice by slice, or use multiple landmark patterns, aggregate barcodes as a default and may be discarding per-slice information. The paper offers concatenation as a practical guideline for TDA-enabled medical image classification.","feed_headline":"Concatenate barcode features, don't aggregate them","feed_subtitle":"On CT, ultrasound, and mammogram datasets, per-barcode vectors improved classification in almost every setup.","key_machinery":"The comparison turns on two ways of combining multiple persistence barcodes into one feature vector. Barcode aggregation takes the multiset union of all intervals from the per-slice or per-landmark barcodes, keeping duplicates and original order, and applies a vectorization method once; feature concatenation applies the same vectorization to each barcode independently and joins the resulting vectors. Aggregation erases which slice or landmark produced each bar before featurization, whereas concatenation keeps each barcode's contribution as a separate block of the feature vector, which the paper credits with preserving detailed topological information.","core_discovery":"The paper argues that when a single medical image gives rise to several persistence barcodes, the final topological feature vector should be built by vectorizing each barcode separately and concatenating the vectors, rather than by aggregating the barcodes into one multiset of intervals and then vectorizing the aggregate. Aggregation collapses the individual barcodes into a single summary and loses the detailed topological and geometric information that per-barcode featurization preserves. This claim is supported by experiments on liver tumour CT, kidney tumour CT, breast ultrasound, and mammography datasets, using cubical-complex and landmark-based Vietoris–Rips filtrations together with five vectorization methods: Betti curve, persistent statistics, entropy summary, persistent landscapes, and persistent tropical coordinates. Across the reported configurations, feature concatenation matched or beat aggregation on the main metrics in almost every case, with the only tie occurring on the kidney dataset with landmark-based filtration.","pith_inferences":["A natural extension is to test learned combinations of per-barcode vectors, such as attention pooling, since plain concatenation keeps all information but grows feature dimension linearly with barcode count.","The proposed mechanism predicts that concatenation's advantage should increase as the number of barcodes per image grows; this could be checked by varying the number of slices on the liver CT dataset.","If the finding transfers, it may also guide non-medical TDA pipelines where multiple persistence diagrams are computed from one object, such as multi-view or time-varying data.","Replicating the comparison with repeated cross-validation would show whether the preference for concatenation holds in expectation rather than on a single split."],"forward_implications":["TDA pipelines that produce multiple barcodes per image should vectorize each barcode and concatenate the vectors when the goal is classification accuracy.","The benefit of concatenation appears across 2D and 3D modalities, cubical and landmark-based filtrations, and five different vectorizations, so it is not tied to one feature type.","Aggregation may still be preferred when feature dimension or memory is a hard constraint, since concatenation lengthens the feature vector in proportion to the number of barcodes.","Because the comparison uses a single held-out 80/20 split, the direction of the advantage is clearer than its exact magnitude in other settings."],"supporting_citations":[{"why":"Supplies the uniform local binary pattern geometries used to select landmark pixels for point-cloud construction.","marker":"[16]"},{"why":"Provides the cubical complex filtration method used to build persistence barcodes directly from image grids.","marker":"[19]"},{"why":"Supplies the persistent statistics vectorization, including bar counts and entropy features.","marker":"[24]"},{"why":"Defines the entropy summary function used as one of the five vectorization methods.","marker":"[25]"},{"why":"Introduces persistence landscapes, the vectorization method used in the comparison.","marker":"[26]"},{"why":"Defines persistent tropical coordinates, the vectorization method that produced the best 2D result.","marker":"[27]"},{"why":"Supplies the KiTS19 kidney tumour CT dataset used as a 3D benchmark.","marker":"[29]"},{"why":"Supplies the breast ultrasound images (BUSI) dataset used as a 2D benchmark.","marker":"[30]"},{"why":"Supplies the mammography images (DDSM) dataset used as a 2D benchmark.","marker":"[31]"}],"fun_headline_variants":["Per-barcode features beat aggregation in medical imaging","Concatenate persistence barcodes, not aggregate them","Medical imaging: concatenate topological features per barcode","Feature concatenation wins over barcode aggregation","Why per-barcode feature vectors outperform aggregated ones"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that choosing the best classifier and vectorization for each strategy on the same validation procedure, and comparing them on a single 80/20 train-test split, is a fair measure of aggregation versus concatenation rather than an artifact of model selection or one lucky split.","fun_headline_variants_meta":{"raw":{"variants":["Per-barcode features beat aggregation in medical imaging","Concatenate persistence barcodes, not aggregate them","Medical imaging: concatenate topological features per barcode","Feature concatenation wins over barcode aggregation","Why per-barcode feature vectors outperform aggregated ones"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000217,"raw_usage":{"total_tokens":1428,"prompt_tokens":929,"completion_tokens":499,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":426}},"tokens_in":545,"tokens_out":499,"duration_ms":5391,"temperature":1.0,"reasoning_tokens":426,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:40:54.307098+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same pipelines on the same datasets with repeated stratified cross-validation, for example 10 repeats of 5-fold CV, and count how often aggregation beats concatenation; if the win rate approaches 50 percent across folds, the claimed advantage is a selection artifact rather than a property of the combination strategy.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the uniform local binary pattern geometries used to select landmark pixels for point-cloud construction."},{"cited_title":"On the stability of persistent entropy and new summary functions for topological data analysis.Pattern Recognition, 107:107509, 2020","cited_arxiv_id":null,"evidence_quote":"Defines the entropy summary function used as one of the five vectorization methods."},{"cited_title":"Tropical coordinates on the space of persistence barcodes.Foundations of Computational Mathematics, 19(1):101–129, 2019","cited_arxiv_id":null,"evidence_quote":"Defines persistent tropical coordinates, the vectorization method that produced the best 2D result."},{"cited_title":"Current status of the digital database for screening mammography","cited_arxiv_id":null,"evidence_quote":"Supplies the mammography images (DDSM) dataset used as a 2D benchmark."}],"review_version":1}