REVIEW 4 major objections 6 minor 56 references
Persistent Homology of Topic Networks for the Prediction of Reader Curiosity
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Topological cavities in a story's topic network—disconnected components, loops, and voids—raise explained deviance in reader-curiosity prediction from 29.7% to 72.9% over a control model.
desk verdict A promising proof-of-concept whose headline claim of significant prediction is not supported by its own permutation test. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a dynamic topic network: vertices are topics obtained by embedding text chunks, reducing dimension, and density-based clustering; edges connect topics that appear in consecutive chunks and are weighted by the cosine similarity of their embeddings. The network is accumulated chapter by chapter, and a Vietoris–Rips filtration on shortest-path distances produces persistence diagrams; from these, the pipeline extracts Betti numbers $\beta_0$, $\beta_1$, $\beta_2$ (components, loops, voids) and the Bottleneck and Wasserstein distances between consecutive chapters' diagrams. These features serve as the independent variables, alongside controls, in a generalized additive model of mean reader curiosity.
What would settle it
Randomly shuffle the chapter order used to build the cumulative topic network while keeping each chapter's curiosity rating in place; if the topological features still explain chapter-level curiosity as well, they are tracking something other than narrative information gaps (such as chapter length or topic count).
Extended reading notes
Core claim
The central discovery is that topological cavities in a cumulative topic network are predictive of reader curiosity. Across 27 chapters, Betti numbers ($\beta_0$, $\beta_1$, $\beta_2$) and the Bottleneck and Wasserstein distances between consecutive persistence diagrams improve a generalized additive model of mean chapter curiosity over a null model containing only chapter index and number of novel topics (72.9% vs. 29.7% explained deviance; $\chi^2 = 11.25$, df = 4.7, $p < .001$). A sensitivity analysis across three embedding models and several window/overlap sizes largely reproduces the improvement, which the authors read as evidence that the effect is not an artifact of one parameter choice.
Load-bearing premise
The load-bearing premise is that the topological cavities found in the topic network correspond to the information gaps that actually make readers curious, rather than being incidental geometric features of the clustering.
Editorial extensions
If this is right
- Chapter-level curiosity can be predicted from the geometry of a text alone, without collecting new reader ratings, once a model is fitted.
- The pipeline gives a quantitative handle on "plot holes"—loops and voids in the topic network—linking narrative structure to a psychological theory of curiosity.
- The method is not tied to one embedding model or chunking scheme: the sensitivity analysis shows the predictive gain across several configurations.
- Because the network is built from any sequence of text chunks with periodic engagement measurements, the same machinery transfers to other sequential content such as news articles or video.
Reading between the lines
- A direct test the authors do not run: compare curiosity prediction on the intact novel against prediction on a chapter-permuted version; if topological features retain their explanatory power after permutation, the "gap" interpretation would be in doubt.
- The same pipeline could be pointed at other self-report dimensions—suspense, interest, boredom—to see whether the same cavities drive different engagement states, which would help separate curiosity-specific effects from general narrative arousal.
- If the information-gap mapping is valid, editors and writers could use the method to locate chapters where curiosity dips and revise pacing, a practical use the paper does not discuss.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript introduces a pipeline that segments a narrative into chunks, constructs a dynamic topic network via embedding/UMAP/HDBSCAN, applies persistent homology to derive Betti numbers and bottleneck/Wasserstein distances between consecutive networks, and uses these as predictors in a GAM for chapter-level mean reader curiosity ratings (n=27 chapters, 49 raters) from The Hunger Games. The paper reports that the full model explains 72.9% of deviance versus 29.7% for a control model and states that topological features significantly improve curiosity prediction, and it includes a sensitivity analysis across embedding models and window sizes.
Significance. The interdisciplinary combination of TDA and curiosity theory is novel, and the authors share code and data, which is a strength. However, the central empirical claim is not supported by the reported statistics: the main permutation test is marginal (p<.06), the analysis is in-sample, and the sensitivity analysis is inconsistent. The paper's value is currently exploratory, not confirmatory, and the abstract and conclusion overstate the strength of the evidence.
major comments (4)
- [Section 5.4 and Abstract] The central claim that topological features 'significantly improve' curiosity prediction is not secured by the reported tests. The permutation test for the full model gives p<0.06, which is not significant at the conventional 0.05 level, and the likelihood-ratio test (chi2=11.25, df=4.7, p<0.001) is an unreliable basis for comparing penalized GAMs with n=27 because chi-square approximations are anti-conservative in small samples. A permutation test of the deviance difference between the full and null models should be reported; this is the direct test of the improvement claim.
- [Section 4.4 and Section 5.4] The term 'prediction' is used throughout (Abstract, Section 5.4, Conclusion), but all reported statistics are in-sample goodness-of-fit measures on the same 27 chapter-level observations used to fit the model. With eleven smooth terms each having basis dimension k=4, high in-sample deviance is expected even if the predictors have no out-of-sample value. To support predictive claims, the authors need a cross-validation procedure (e.g., leave-one-chapter-out) or must substantially weaken the language to 'in-sample association'.
- [Appendix A, Table 3] The sensitivity analysis does not support the robustness claim. At the 0.05 threshold, only six of the twelve configurations are significant; the primary Voyage 5/2 configuration is p<0.06, and two configurations fail badly (Voyage 10/3 p=0.288; E5 7/2 p=0.418). The statement in A.1 that improvement is significant 'across nearly all' configurations is inaccurate and should be corrected.
- [Section 3.3 and Appendix B.3] The mapping from topological cavities (beta0, beta1, beta2 and their distances) to Loewenstein's information gaps is asserted without independent validation. The statistical association, if established, would not by itself validate this psychological interpretation; the paper should either provide construct-validity evidence or present the mapping as an interpretive hypothesis.
minor comments (6)
- [Section 1] The text says 'Our main contributions are threefold' but then lists four contributions; please fix the numbering or the count.
- [Appendix C] The citation 'Shrount and Fleiss (1979)' should be 'Shrout and Fleiss (1979)'.
- [Appendix A.1] The phrase 'explanation of deviation' should be 'explained deviance', and '72 9%' should be '72.9%'.
- [Section 4.1.1] The dataset section reports n=76 participants but later restricts the analysis to n=49; please clarify this early in the methods.
- [Section 1 and Section 4.1] The contribution item says 'The Hunger Game' but should be 'The Hunger Games'.
- [Table 3] The abbreviation 'W/O' is used in the table without definition in the text; please define it as Window/Overlap.
Circularity Check
Central 'prediction' claim is supported only by in-sample deviance: the model is fitted to the same 27 chapter ratings it is said to predict, so the reported improvement is a goodness-of-fit result, not a predictive validation.
-
fitted input called prediction
[Abstract; Section 5.4 (Generalized Additive Model)]
"We then use the topological features from our pipeline as independent variables to predict these ratings, and experimentally show that they significantly improve curiosity prediction compared to a baseline model (73% vs. 30% explained deviance) ... Full Model ... explains 65.7% of the variance (permutation test, p < .05) and 72.9% of the deviance (permutation test, p < .06)."
The GAM is estimated on all 27 chapter-level mean curiosity ratings (Section 4.4), and the reported 72.9% explained deviance is the in-sample fit of that same model to those same ratings. Because the smoothing parameters and coefficients are chosen to maximize fit to these exact observations, the 'predicted' values are the fitted values; the full-versus-null deviance gap is a goodness-of-fit comparison on training data, not a prediction on held-out chapters. Thus the central quantitative claim—that topological features significantly improve curiosity prediction—reduces by construction to measuring the fit of the model to the data that generated it, with no out-of-sample step.
full rationale
The derivation of the topological predictors is self-contained: Betti numbers, Bottleneck distances, and Wasserstein distances are computed from text embeddings and topic networks without using the curiosity ratings, and the GAM is fit with standard penalized estimation. There is no load-bearing self-citation chain, no imported uniqueness theorem, and no definitional equivalence between the topological features and the outcome. The circularity is confined to the paper's central validation wording: the 'prediction' of reader curiosity is evaluated by explained deviance on the same 27 chapter ratings used to fit the model. In-sample deviance is a goodness-of-fit quantity, so the reported 72.9% versus 29.7% gap does not demonstrate predictive validity; it is a fit to the training data. This warrants a 6 under the fitted-input-called-prediction pattern, not higher, because the features themselves are independent of the target and the pipeline could in principle be evaluated on held-out data. The mapping of topological cavities to Loewenstein information gaps is an unvalidated psychological assumption, but it is not a circular step in the derivation chain.
Assumptions & free parameters
free parameters (6)
- Sliding window size (sentences, overlap) =
5, 2
- UMAP embedding dimension =
32
- UMAP n_neighbors =
15
- HDBSCAN min_cluster_size =
3
- GAM basis dimension k =
4
- Winsorization percentiles =
2.5 and 97.5
assumptions (5)
- domain assumption Loewenstein's Information Gap Theory describes reader curiosity
- ad hoc to paper Topological cavities are valid proxies for information gaps
- ad hoc to paper The cumulative dynamic topic network represents a reader's evolving knowledge state
- domain assumption Shortest-path distances in the topic graph are appropriate for the Vietoris-Rips filtration
- domain assumption Mean curiosity across 49 readers is a reliable chapter-level measure
invented entities (1)
-
Topological cavities as information gaps
Cite this review
Pith. "Pith review of Persistent Homology of Topic Networks for the Prediction of Reader Curiosity." pith.science (2026). https://pith.science/paper/G62LDGPZ
@misc{pith2026250611095,
author = {Pith},
title = {Pith review of: Persistent Homology of Topic Networks for the Prediction of Reader Curiosity},
year = {2026},
howpublished = {\url{https://pith.science/paper/G62LDGPZ}},
note = {Machine review of arXiv:2506.11095}
}
read the original abstract
Reader curiosity, the drive to seek information, is crucial for textual engagement, yet remains relatively underexplored in NLP. Building on Loewenstein's Information Gap Theory, we introduce a framework that models reader curiosity by quantifying semantic information gaps within a text's semantic structure. Our approach leverages BERTopic-inspired topic modeling and persistent homology to analyze the evolving topology (connected components, cycles, voids) of a dynamic semantic network derived from text segments, treating these features as proxies for information gaps. To empirically evaluate this pipeline, we collect reader curiosity ratings from participants (n = 49) as they read S. Collins's ''The Hunger Games'' novel. We then use the topological features from our pipeline as independent variables to predict these ratings, and experimentally show that they significantly improve curiosity prediction compared to a baseline model (73% vs. 30% explained deviance), validating our approach. This pipeline offers a new computational method for analyzing text structure and its relation to reader engagement.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
B. Abu-Salih and S. Alotaibi. 2024. https://doi.org/10.1016/j.heliyon.2024.e25383 A systematic literature review of knowledge graph construction and application in education . Heliyon, 10(3):e25383
-
[4]
M. Allaoui, M. L. Kherfi, and A. Cheriet. 2020. https://doi.org/10.1007/978-3-030-51935-3_34 Considerably improving clustering algorithms using UMAP dimensionality reduction technique: A comparative study . In International Conference on Image and Signal Processing, volume 12119 of Lecture Notes in Computer Science, pages 317--325. Springer
-
[5]
M. S. Asyaky and R. Mandala. 2021. https://doi.org/10.1109/ICAICTA53211.2021.9640285 Improving the performance of hdbscan on short text clustering by using word embedding and UMAP . In 8th International Conference on Advanced Informatics: Concepts, Theory and Applications, pages 1--6, Bandung, Indonesia. IEEE
-
[6]
J. Berger, W. W. Moe, and D. A. Schweidel. 2023. https://doi.org/10.1177/00222429231152880 What holds attention? linguistic drivers of engagement . Journal of Marketing, 87(5):793--809
-
[7]
R. J. G. B. Campello, D. Moulavi, and J. Sander. 2013. https://doi.org/10.1007/978-3-642-37456-2_14 Density-based clustering based on hierarchical density estimates . In Pacific-Asia Conference on Knowledge Discovery and Data Mining, volume 7819 of Lecture Notes in Computer Science, pages 160--172, Berlin, DE. Springer
- [8]
Show all 56 references
-
[9]
Gunnar Carlsson. 2020. https://doi.org/10.1038/s42254-020-00249-3 Topological methods for data modelling . Nature Reviews Physics, 2(12):697--708
2020 doi
-
[10]
Frédéric Chazal and Bertrand Michel. 2021. https://doi.org/10.3389/frai.2021.667963 An Introduction to Topological Data Analysis : Fundamental and Practical Aspects for Data Scientists . Frontiers in Artificial Intelligence, 4
2021
-
[11]
Chesnokova, S
A. Chesnokova, S. Zyngier, V. Viana, J. Jandre, A. Rumbesht, and F. Ribeiro. 2017. https://doi.org/10.5325/complitstudies.54.4.0824 Cross-cultural reader response to original and translated poetry: An empirical study in four languages . Comparative Literature Studies, 54(4):824--849
2017 doi
-
[12]
N. H. Christianson, A. Sizemore Blevins, and D. S. Bassett. 2020. https://doi.org/10.1098/rspa.2019.0741 Architecture and evolution of semantic networks in mathematics texts . Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 476(2239):20190741
2020
-
[13]
S. Collins. 2011. https://shop.scholastic.co.uk/the-hunger-games The Hunger Games . Scholastic, London, UK
2011
-
[14]
Csardi and T
G. Csardi and T. Nepusz. 2006. https://igraph.org The igraph software package for complex network research . InterJournal, Complex Systems:1695
2006
- [15]
- [16]
-
[17]
Epskamp, A
S. Epskamp, A. O. J. Cramer, L. J. Waldorp, V. D. Schmittmann, and D. Borsboom. 2012. https://doi.org/10.18637/jss.v048.i04 qgraph : Network visualizations of relationships in psychometric data . Journal of Statistical Software, 48(4):1--18
2012 doi
-
[18]
B. T. Fasy, J. Kim, F. Lecci, C. Maria, D. L. Millman, and V.. Rouvreau. 2024. https://doi.org/10.32614/CRAN.package.TDA TDA : Statistical tools for topological data analysis . Accessed 2025-02-12
2024 doi
-
[19]
Gerlach, T
M. Gerlach, T. P. Peixoto, and E. G. Altmann. 2018. https://doi.org/10.1126/sciadv.aaq1360 A network approach to topic models . Science Advances, 4(7):eaaq1360
2018 doi
- [20]
-
[21]
Z. Guo, L. Xia, Y. Yu, T. Ao, and C. Huang. 2024. https://openreview.net/forum?id=bbVH40jy7f Light RAG : Simple and fast retrieval-augmented generation . Technical report, Beijing University of Posts and Telecommunications
2024
- [22]
-
[23]
M. D. Humphries and K. Gurney. 2008. https://doi.org/10.1371/journal.pone.0002051 Network `small-world-ness': A quantitative method for determining canonical network equivalence . PLoS ONE, 3(4):e0002051
2008 doi
-
[24]
J. Liu, J. Wang, Q. Zheng, W. Zhang, and L. Jiang. 2012. https://doi.org/10.1016/j.knosys.2012.07.011 Topological analysis of knowledge maps . Knowledge-Based Systems, 36:260--267
2012 doi
-
[25]
Loewenstein
G. Loewenstein. 1994. https://doi.org/10.1037/0033-2909.116.1.75 The psychology of curiosity: A review and reinterpretation . Psychological Bulletin, 116(1):75--98
1994 doi
-
[26]
M. M. Maslej, R. A. Mar, and V. Kuperman. 2021. https://doi.org/10.1037/aca0000282 The textual features of fiction that appeal to readers: Emotion and abstractness. Psychology of Aesthetics, Creativity, and the Arts, 15(2):272--283
2021 doi
-
[27]
McInnes, J
L. McInnes, J. Healy, and S. Astels. 2016. https://hdbscan.readthedocs.io/ The hdbscan clustering library . Accessed 2025-02-12
2016
- [28]
-
[29]
Melville
J. Melville. 2024. https://doi.org/10.32614/CRAN.package.uwot uwot: The uniform manifold approximation and projection (umap) method for dimensionality reduction . Accessed 2025-02-11
2024 doi
-
[30]
Mones, L
E. Mones, L. Vicsek, and T. Vicsek. 2012. https://doi.org/10.1371/journal.pone.0033799 Hierarchy measure for complex networks . PLoS ONE, 7(3):e33799
2012 doi
-
[31]
Moroni and M
D. Moroni and M. A. Pascali. 2021. https://doi.org/10.1134/S1054661821030184 Learning topology: Bridging computational topology and machine learning . Pattern Recognition and Image Analysis, 31(3):443--453
2021 doi
-
[32]
Morris, V
J. Morris, V. Kuleshov, V. Shmatikov, and A. Rush. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.765 Text embeddings reveal (almost) as much as text . In Conference on Empirical Methods in Natural Language Processing, pages 12448--12460, Singapore. Association for Computat...
2023 doi
- [33]
-
[34]
E. Munch. 2017. https://doi.org/10.18608/jla.2017.42.6 A user's guide to topological data analysis . Journal of Learning Analytics, 4(2)
2017 doi
-
[35]
P Patankar, D
S. P Patankar, D. Zhou, C. W. Lynn, J. Z. Kim, M. Ouellet, H. Ju, P. Zurn, D. M. Lydon-Staley, and D. S. Bassett. 2023. https://doi.org/10.1177/26339137231207633 Curiosity as filling, compressing, and reconfiguring knowledge networks . Collective Intelligence, 2(4):1--18
2023 doi
-
[36]
Pons and M
P. Pons and M. Latapy. 2006. https://doi.org/10.7155/jgaa.00124 Computing communities in large networks using random walks . Journal of Graph Algorithms and Applications, 10(2):191--218
2006 doi
-
[37]
R Core Team . 2024. https://www.R-project.org/ R: A language and environment for statistical computing . Accessed 2025-02-12
2024
-
[38]
Schiefele
U. Schiefele. 1999. https://doi.org/10.1207/s1532799xssr0303_4 Interest and learning from text . Scientific Studies of Reading, 3(3):257--279
1999 doi
-
[39]
scikit-learn developers . 2025. https://scikit-learn.org/stable/auto_examples/cluster/plot_cluster_comparison.html Comparing different clustering algorithms on toy datasets . Accessed 2025-02-12
2025
-
[40]
D. R. Sheehy. 2012. https://doi.org/10.1145/2261250.2261286 Linear-size approximations to the V ietoris-- R ips filtration . In 28th Annual Symposium on Computational Geometry, pages 239--248. ACM
2012
-
[41]
P. E. Shrount and J. L. Fleiss. 1979. https://doi.org/10.1037/0033-2909.86.2.420 Intraclass correlations: Uses in assessing rater reliability . Psychological Bulletin, 86(2):420--428
1979 doi
-
[42]
Sotirakou, D
C. Sotirakou, D. Trilling, P. Germanakos, D. A. Sinis, and C. Mourlas. 2021. https://doi.org/10.3233/WEB-210457 Understanding the link between audience engagement metrics and the perceived quality of online news using machine learning . Web Intelligence, 19(1-2):63--86
2021 doi
-
[43]
Spearman
C. Spearman. 1904. https://doi.org/10.2307/1412159 The proof and measurement of association between two things . American Journal of Psychology, 15(1):72--101
1904 doi
-
[44]
Tymochko, J
S. Tymochko, J. Chaput, T. Doster, E. Purvine, J. Warley, and T. Emerson. 2021. https://doi.org/10.1109/ICMLA52953.2021.00069 Con connections: Detecting fraud from abstracts using topological data analysis . In 20th IEEE International Conference on Machine Learning and Applica...
2021
- [45]
-
[46]
Voyage AI . 2025. https://blog.voyageai.com/2025/01/07/voyage-3-large/ v oyage-3-large: the new state-of-the-art general-purpose embedding model . Accessed 2025-02-11
2025
-
[47]
R. R. Wadhwa, D. F. K. Williamson, A. Dhawan, and J. G. Scott. 2018. https://doi.org/10.21105/joss.00860 TDA stats: R pipeline for computing persistent homology in topological data analysis . Journal of Open Source Software, 3(28):860
2018 doi
- [48]
-
[49]
Ward and D
A. Ward and D. Litman. 2008. https://doi.org/10.1007/978-3-540-69132-7_49 Semantic cohesion and learning . In International Conference on Intelligent Tutoring Systems, volume 5091 of Lecture Notes in Computer Science, pages 459--469, Berlin, DE. Springer
2008 doi
-
[50]
S. N. Wood. 2011. https://doi.org/10.1111/j.1467-9868.2010.00749.x Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models . Journal of the Royal Statistical Society Series B: Statistical Methodology, 73(1):3--36
2011
-
[51]
S. N. Wood. 2017. https://doi.org/10.1201/9781315370279 Generalized Additive Models: An Introduction with R , 2 edition. Chapman and Hall/CRC
2017 doi
-
[52]
Z. Yang, R. Algesheimer, and C. A. Tessone. 2016. https://doi.org/10.1038/srep30750 A comparative analysis of community detection algorithms on artificial networks . Scientific Reports, 6:30750
2016 doi
-
[53]
J. Yu. 2024. https://doi.org/10.31219/osf.io/4qk5j Text embeddings to measure text topics . Open Science Framework
2024 doi
- [54]
-
[55]
D. Zhou, S. Patankar, D. M. Lydon-Staley, P. Zurn, M. Gerlach, and D. S. Bassett. 2024. https://doi.org/10.1126/sciadv.adn3268 Architectural styles of curiosity in global Wikipedia mobile app readership . Science Advances, 10(43):eadn3268
2024 doi
-
[56]
X. Zhu. 2013. https://www.ijcai.org/Abstract/13/288 Persistent homology: An introduction and a new text representation for natural language processing . In International Joint Conference on Artificial Intelligence, pages 1953--1959
2013
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.