{"id":"48accabd-46fa-425c-b0f8-7dd6cf903c41","arxiv_id":"2501.12522","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Out-of-distribution images show longer average H0 persistence lifetimes in a ResNet18 embedding layer than in-distribution training and test images.","lead":"This paper measures how out-of-distribution images differ inside a neural network using topology. It finds that unfamiliar images leave longer-lived traces in the network's internal representation, which could become a signal for detecting when a model faces inputs it was not trained on.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"H0 average lifetime is a single-linkage merge-distance scale statistic; the reported OOD gap may reflect embedding spread rather than failure of topological simplification, so a normalization control is needed before the central claim is accepted.","rationale":"The reader's weakest-assumption analysis already identifies the core concern: average H0 lifetime conflates topological simplification with the overall scale or variance of embedding points. My stress-test confirms and sharpens this: because H0 births are all at zero, average H0 lifetime is by definition the average merge distance of single-linkage clustering. This is a purely geometric quantity. The paper's interpretation in Section 6 that OOD data is not 'trivialized' therefore requires a control that the authors do not provide. The concern is not that the empirical distributions are wrong; with 50,000 bootstrap iterations and non-overlapping confidence intervals, the reported ordering is likely replicable. The concern is that the ordering may be explained by OOD embeddings being more spread out or having larger norms, which is a much weaker claim than failure of topological simplification. I do not see a second concern of comparable weight: the H1 analysis is inconclusive by the authors' own admission, but the H0 confound alone is sufficient to condition the verdict. The proposed test is concrete, cheap, and would settle the issue: normalize each subsample by its own scale and see if the separation persists. If it does, the central claim survives in a strengthened form; if not, the paper's headline should be revised to describe a scale difference rather than a topological one. The verdict remains CONDITIONAL because the paper can address this with additional experiments; no change to the reader's conditional recommendation is needed.","tokens_in":12051,"tokens_out":2131,"duration_ms":25519,"concrete_test":"For each bootstrap subsample of Train, Test, and OOD embeddings, divide all pairwise distances by that subsample's mean pairwise distance (or, alternatively, by its median pairwise distance) before computing the H0 persistence diagram, and then recompute the average H0 lifetime distributions and 95% confidence intervals. If the Train/Test/OOD distributions overlap after this normalization, the reported gap in Section 6 is a scale artifact rather than a topological difference. As a complementary check, compute the mean pairwise distance alone as a non-topological baseline and compare its separation power against H0 average lifetime; if the baseline separates OOD from ID as cleanly as the topological summary, the topological interpretation is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim is that well-trained DNNs do not topologically simplify OOD embeddings, evidenced by longer average H0 lifetime for OOD subsamples (Section 6, Tables 1-2). The load-bearing issue is that, as the paper itself notes in Section 5.2, every H0 feature is born at filtration parameter 0, so average H0 lifetime equals average death time, which is exactly the average of the merge distances of single-linkage clustering. This is a scale/spread statistic, not a topological invariant: any OOD-induced increase in embedding norms or pairwise distances mechanically increases average lifetime, with no change in the shape or connectivity structure of the point cloud. The paper performs no normalization for overall scale, variance, or pairwise distance. Thus the observation 'OOD average lifetime is longer' may be true but does not by itself support the narrative that topological simplification 'does not hold for OOD examples.' The absence of a significant effect for maximum H0 lifetime and overlapping H1 lifetimes (Figures 3, 5) makes the H0 average-lifetime result the sole piece of evidence for the central claim, so the confound is decisive if it lands. The fix is a standard control: normalize each bootstrap subsample before computing persistence and check whether the separation survives.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the persistent homology of latent embeddings from a ResNet18 classifier, comparing bootstrap subsamples of training, test, and out-of-distribution (OOD) data. For two settings (MNIST vs. EMNIST and CIFAR-10 vs. CIFAR-100), it computes H0 and H1 summary statistics, primarily average and maximum lifetimes, over 50,000 bootstrap iterations. The central empirical claim is that the average H0 lifetime of OOD embeddings is longer than that of in-distribution training or test embeddings, which the authors interpret as evidence that well-trained DNNs do not induce the same topological simplification on OOD inputs that they induce on in-distribution data.","tokens_in":12287,"tokens_out":6065,"duration_ms":63597,"significance":"If the central claim survives scrutiny, the paper would extend the topological-simplification observations of Naitzat et al. (2020) to realistic architectures and datasets, and would identify a simple persistent-homology summary as a potentially useful signal for OOD detection. The computational pipeline, using bootstrap subsampling and GPU-accelerated Ripser, is a practical strength, and the comparison across four benchmark datasets is a reasonable first step. However, the current evidence is not yet convincing: the headline statistic is scale-dependent, the paper lacks a formal statistical test, and the connection between the measured quantity and the stated topological conjecture is not established.","major_comments":[{"comment":"The headline result rests on the H0 average lifetime, but this quantity is not a topological invariant in the sense needed by the paper's argument. Because every H0 feature is born at filtration value zero, the average lifetime equals the average death time, which is exactly the average merge distance of the single-linkage clustering of the bootstrap subsample. A change in the overall scale or spread of the embedding points therefore changes the average lifetime even if the connectivity structure is unchanged. The paper reports no normalization of embedding norms, pairwise distances, or per-dimension scales, so the observed OOD gap in Tables 1 and 2 may simply reflect larger-norm or more dispersed OOD embeddings rather than a failure of topological simplification. I ask the authors to repeat the analysis after normalizing each bootstrap subsample (e.g., by the median pairwise distance or by per-dimension standard deviation) and to report whether the separation survives; a comparison with the trivial baseline of mean pairwise distance would additionally show what TDA adds.","section":"§5.2 and §6"},{"comment":"The abstract and Section 6 claim the OOD average lifetime is \"statistically longer,\" but the paper never performs a statistical test. The 95% bootstrap confidence intervals for the n=150 samples do not overlap, which is suggestive, but the text does not report a p-value, an effect size, or any correction for the multiple sample sizes shown in Appendix A. Moreover, only one trained model per dataset is examined, so no statement can be made about variability across training runs. I request a formal two-sample test (bootstrap or permutation test) on the lifetime distributions, results for all sample sizes, and at least a few independently trained models to establish that the gap is not specific to one training run.","section":"§6, Tables 1–2"},{"comment":"The paper's conceptual claim is about topological simplification, operationalized in Section 3 as the trivialization of each class subset to β0=1 and the multi-class conjecture β0(f(X))=n. However, the reported summary statistics never directly measure Betti numbers or the number of connected components; H0 average lifetime is a summary of merge distances, not a count of components. To connect the measurements to the stated conjecture, the authors should report, for the same bootstrap subsamples, the number of H0 features (or the persistence diagram rank) and the class-wise behavior, and verify that ID subsamples actually exhibit approximately ten connected components for CIFAR-10/MNIST before claiming that OOD data \"does not hold\" the trivialization property.","section":"§3 and §6"}],"minor_comments":[{"comment":"The text uses \"CIF AR-10\" and \"CIF AR-100\" with a space; the standard notation is CIFAR-10 and CIFAR-100, and the inconsistent spacing should be fixed throughout.","section":"Abstract and §5.1"},{"comment":"The word \"complimentary\" in the paragraph on activation graphs should be \"complementary.\"","section":"§2"},{"comment":"The sentence \"converges (asymptotically) to the the ground truth\" contains a duplicated article and should read \"to the ground truth.\"","section":"§4.3"},{"comment":"The sentence beginning \"Rabadán and Blumberg [2019] present a simplified explanation of approach developed by Fasy et al. [2013], Chazal et al. [2013] demonstrates...\" is ungrammatical and should be revised for clarity.","section":"§4.3"},{"comment":"Training details for the ResNet18 models (epochs, optimizer, learning rate, and test accuracy) are not reported, which makes it difficult to assess how \"well-trained\" the models are; these details should be added or referenced.","section":"§5.1"},{"comment":"The text asserts that the trivialization effect \"holds for all classes and both training and test splits,\" but no class-wise results are shown; please either add class-wise figures/tables or soften the claim.","section":"§6.1"},{"comment":"The captions describe H1 features as \"connected components,\" but H1 features are loops or holes; the captions should be corrected.","section":"Figures 3 and 5"},{"comment":"The term \"landmarks\" is introduced in the abstract and contributions but is not defined or used again; either define it or remove it.","section":"§1.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible empirical exploration, but the central claim is not yet established because the H0 average lifetime is a scale-dependent single-linkage merge statistic. The normalization control and a formal statistical comparison are essential. If the authors can provide those and show the separation survives, I would view the paper as a useful contribution; otherwise, the manuscript is better framed as reporting a scale phenomenon rather than a topological one."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the two things you should know. The paper is a descriptive TDA study of ResNet18 embeddings, and its central observation is that bootstrap subsamples of OOD images have larger average H0 lifetimes than training/test subsamples, with non-overlapping 95% intervals on MNIST/EMNIST and CIFAR-10/CIFAR-100. The authors read this as evidence that topological simplification holds for in-distribution data but fails for OOD. That reading is not yet supported, because H0 average lifetime is a single-linkage merge scale, not a topological invariant. The paper itself notes that all H0 features are born at zero, so lifetime equals death time; the death times are just the merge distances in the dendrogram. If OOD embeddings have larger norms or wider spread, you get longer lifetimes for free, with no change in shape. There is no normalization control for scale or variance. So the result may be true but does not establish the topological claim.\n\nWhat is genuinely new: extending the Naitzat et al. simplification result to test data and to a realistic architecture, and showing a reproducible, apparent separation between ID and OOD embeddings. The experimental setup is careful—Ripser, bootstrap with several sample sizes, H0 and H1, two dataset pairs—and the authors are honest about H1's lack of separation and about the computation cost. That is real work.\n\nWhere it falls short, in order: (1) the scale confound is load-bearing and easy to fix; normalize each subsample (e.g., to unit variance or unit mean pairwise distance) and see if the separation survives. (2) There is no formal statistical test, just non-overlapping CIs, which is fine for a descriptive study but not for the strength of the claim. (3) Only one model per dataset, no repeated training runs, so we don't know if this is model-specific. (4) No code or data, which matters for a purely empirical paper. (5) The headline statistic is chosen post hoc; maximum lifetime and H1 do not separate. That makes the story fragile.\n\nThe paper deserves a serious referee: the observation is interesting and the method is sound enough to reproduce, and the fix is straightforward. I would send it to review with a request for the normalization control and for code/data. If the separation survives normalization, it becomes a solid contribution to TDA-based OOD detection; if not, the paper should be reframed as a cautionary study.","headline":"OOD examples show longer average H0 persistence in ResNet18 embeddings, but the 'no topological simplification' reading is undercut by that statistic being a scale measure; still deserves peer review with a normalization control.","tokens_in":12809,"tokens_out":2825,"would_cite":false,"duration_ms":29572,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["55N31","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A well-trained ResNet18 does not topologically simplify out-of-distribution embeddings, leaving their average persistence statistically longer than for training or test examples.","keywords":["out-of-distribution detection","persistent homology","topological simplification","deep neural networks","latent embeddings","bootstrap","ResNet18","Vietoris-Rips filtration"],"falsifier":"Recompute the H0 average lifetime on embeddings after normalizing each point to unit norm. If the OOD versus in-distribution gap vanishes under this normalization, the reported effect is a scale artifact, not a topological property.","tokens_in":11845,"feed_emoji":"🧠","tokens_out":7425,"duration_ms":62971,"temperature":0.7,"pith_summary":"This paper sets out to show that out-of-distribution (OOD) examples are topologically different from in-distribution data inside a deep neural network. Using persistent homology on the embedding layer of ResNet18, the authors find that the average lifetime of connected components (H0 features) is statistically longer for OOD samples than for training or test samples. They interpret this as evidence that a well-trained network fails to impose the same topological simplification on unfamiliar inputs that it imposes on familiar ones. The finding holds across two benchmark pairs, MNIST/EMNIST and CIFAR-10/CIFAR-100, and suggests that a simple topological summary statistic could support OOD detection.","feed_headline":"OOD data resists a network's topological simplification","feed_subtitle":"Average persistence of connected components in a ResNet18 embedding layer separates OOD from in-distribution samples.","key_machinery":"The central object is the 0-dimensional persistent homology (H0) of the Vietoris-Rips filtration on bootstrap subsamples of the embedding layer. Each H0 feature is born at filtration radius zero when a point appears and dies when it merges into another component; the lifetime is that death radius. The summary statistic is the average lifetime over all H0 features in a subsample. The authors use Ripser to compute persistence diagrams and bootstrap resampling (50,000 iterations, subsample sizes 25–150) to obtain distributions and 95% confidence intervals that separate OOD from in-distribution data.","core_discovery":"The central claim is that the average lifetime (persistence) of OOD examples is statistically longer than that of training or test examples. The authors measure this by computing persistent homology of bootstrap subsamples of 512-dimensional embeddings from the penultimate layer of ResNet18. For H0, average lifetime separates the OOD distribution from the in-distribution distributions with non-overlapping 95% confidence intervals, while the training and test distributions nearly coincide. For H1, the differences are much weaker, indicating that the signal lives primarily in the connected-component structure. The authors argue that this is a 'topological landmark' for OOD inputs: the network trivializes in-distribution embeddings by merging clusters, but cannot do the same for unfamiliar data.","pith_inferences":["If the gap is driven by embedding scale, normalizing embeddings would shrink it; if it survives normalization, the topological interpretation would be strengthened, and this is a testable next step.","Average H0 lifetime could be combined with existing OOD scores (like softmax confidence or energy) to improve detection without retraining the network.","The near-overlap of H1 distributions across OOD and in-distribution data suggests that higher-dimensional homology contributes little for these datasets, so a practical detector could ignore H1 entirely.","The method's sensitivity to subsample size (tested up to 150) is unexplored; larger subsamples might sharpen the separation but increase computation, so an efficiency–accuracy tradeoff remains open."],"forward_implications":["A detector could flag inputs as OOD when a bootstrap subsample of their embeddings has an average H0 lifetime above a threshold calibrated on in-distribution data.","Topological simplification appears to be a property of well-trained networks that holds for test data, not just for the training set.","The distance between average-lifetime distributions might serve as a quantitative measure of how far a new dataset is from a model's learned manifold.","Because only H0 is needed, the approach is computationally feasible for realistic architectures and could scale to deployment."],"supporting_citations":[{"why":"Establishes that well-trained networks topologically simplify their training data, the premise this paper extends to test and OOD data.","marker":"Naitzat et al. [2020]"},{"why":"Provides the Ripser.py library used for all persistent homology computations in the experiments.","marker":"Tralie et al., 2018"},{"why":"Presents the Ripser algorithm that makes computing Vietoris-Rips barcodes efficient enough for this pipeline.","marker":"Bauer, 2021"},{"why":"Introduces the bootstrap-based statistical inference for persistence diagrams that underlies the confidence intervals.","marker":"Fasy et al. [2013]"},{"why":"Shows that bootstrap estimates of persistent homology converge to the ground truth, justifying the subsampling approach.","marker":"Chazal et al. [2013]"},{"why":"Provides the stability theorem that supports the noise-robustness of persistence diagrams, a background assumption for the method.","marker":"Cohen-Steiner et al., 2005"}],"fun_headline_variants":["Topology reveals why OOD data eludes DNNs","OOD examples keep their topological complexity","Persistence of OOD data defies network simplification","Unfamiliar inputs resist the network's topology","OOD data's longer persistence signals a topological difference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that average H0 lifetime measures topological simplification rather than the overall scale or spread of the embedding points, because larger or more variable embeddings would mechanically lengthen lifetimes.","fun_headline_variants_meta":{"raw":{"variants":["Topology reveals why OOD data eludes DNNs","OOD examples keep their topological complexity","Persistence of OOD data defies network simplification","Unfamiliar inputs resist the network's topology","OOD data's longer persistence signals a topological difference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000319,"raw_usage":{"total_tokens":1780,"prompt_tokens":902,"completion_tokens":878,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":804}},"tokens_in":518,"tokens_out":878,"duration_ms":6944,"temperature":1.0,"reasoning_tokens":804,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:06:15.271533+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the H0 average lifetime on embeddings after normalizing each point to unit norm. If the OOD versus in-distribution gap vanishes under this normalization, the reported effect is a scale artifact, not a topological property.","supporting_citations":[],"review_version":1}