REVIEW 4 major objections 5 minor 43 references
Interactive Duplicate Search in Software Documentation
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A pattern-based near-duplicate search algorithm for software documentation is proven complete: any true near duplicate of a chosen pattern always shares a guaranteed minimum overlap with some fragment in the output.
desk verdict A useful formalization and interactive process, but the completeness proof has a false lemma and an omitted proof, so the central guarantee is unproven. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The paper's central object is the near-duplicate group (Definition 2): an ordered set of non-overlapping text fragments that all contain the same ordered collection of substrings, the archetype, whose total length is at least a fraction $k$ of each fragment. The algorithm works in three phases: a sliding window of length $|p|/k$ sweeps the document and keeps windows whose longest-common-subsequence edit distance to $p$ falls below a computed threshold (Lemmas 2 and 3 ensure every true near duplicate sits inside some kept window); a shrinking phase narrows each window to the subfragment closest to $p$; and a filtering phase removes duplicate and contained fragments. The completeness criterion is expressed through $O_{\min}(k) = \frac{|p|}{2}\left(3k - \frac{1}{k}\right)$, which increases with $k$ and becomes positive exactly when $k > 1/\sqrt{3}$. The proof runs through Lemmas 1–5; Lemma 4, the claim that shrinking preserves the criterion, is stated with its proof omitted.
What would settle it
Construct a small document containing a known near-duplicate group, run the algorithm, and check whether every $g$ in the group has some returned $w$ with $|g \cap w| \geq \frac{|p|}{2}\left(3k - \frac{1}{k}\right)$; a missed group falsifies Theorem 1. Since Lemma 4 is the unproved hinge, the decisive check is an exhaustive search over short strings for a case where shrinking discards the only fragment covering a near duplicate. A second, independent check compares the implemented edit-distance calculation with the true LCS distance on adversarial string pairs: any pair where the tool rejects a fragment that LCS distance would accept falsifies completeness for the implemented version.
Extended reading notes
Core claim
On the paper's own terms, the discovery is a formal theorem: for any document $D$, pattern $p \in D$, similarity $k \in (1/\sqrt{3},\, 1]$, and any near-duplicate group $G$ of $p$ with similarity $k$, the three-phase pattern-based near-duplicate search algorithm returns a set $R$ such that every $g \in G$ has some $w \in R$ with $|g \cap w| \geq \frac{|p|}{2}\left(3k - \frac{1}{k}\right)$. This completeness criterion is a no-miss guarantee: the user will always find each true near duplicate in the output, possibly shifted or resized, because it must share a fixed minimum overlap with one of the returned fragments. The paper also claims the guarantee survives four of its five optimizations, and reports experiments on 19 industrial documents where the algorithm runs interactively and most outputs contain under 100 elements.
Load-bearing premise
The completeness theorem rests on Lemma 4, which asserts that the shrinking phase never destroys the overlap required by the criterion, and the paper explicitly omits that proof; a second load-bearing premise is that the edit-distance measure in the implemented tool matches the longest-common-subsequence distance used in the proofs.
Editorial extensions
If this is right
- Users can trust that a near-duplicate search returns every true near duplicate for the chosen pattern and similarity, so manual review of the output is complete rather than a gamble.
- Because $O_{\min}(k)$ grows with $k$, demanding stricter similarity ($k \geq 0.77$) guarantees each output/duplicate overlap exceeds half the pattern length, making results easier to recognize and validate by eye.
- The preserved completeness under optimizations 1, 2, 4, and 5 means the performance improvements do not weaken the no-miss guarantee in practice; optimization 3 can occasionally break it, though rarely.
- The complexity bound and experiments (90% of runs under two minutes, 84% of outputs under 100 elements) indicate that the interactive loop—choose pattern, search, edit results—is sustainable on documents up to a few megabytes.
- A direct corollary of Lemma 1 is that all near duplicates of a pattern have lengths within a factor of $k$, so capping windows at $|p|/k$ is not ad hoc but a provable bound.
Reading between the lines
- The proof skeleton—window containment via the triangle inequality plus a length-ratio bound—should generalize to other metric edit distances and to token-level alphabets, so the completeness argument may be portable beyond character-level LCS distance.
- The experiments' observed overlaps exceed the pessimistic $O_{\min}(k)$ bound, so in practice the tool may miss far less than the theorem guarantees; quantifying that excess overlap on more document types would refine the bound.
- Replacing the user's manual pattern pick with an automatic selection from the heat map's maximum-temperature region would test whether the completeness guarantee still yields meaningful groups without human guidance.
- The proven guarantee covers false negatives, not false positives; measuring precision on the industrial corpus—how many returned fragments are meaningful near duplicates—is the natural next step the authors leave open.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents an interactive process for detecting near-duplicate fragments in software documentation. The workflow uses a heat map of exact duplicates to guide a user in selecting a meaningful pattern, then runs a three-phase pattern-based search algorithm whose output the user can edit. The main formal contribution is a definition of a near-duplicate group (Definition 2), a completeness criterion (Equation (2)), and Theorem 1, which claims that for every near duplicate g of pattern p with similarity k, the algorithm's output R contains a fragment w with |g ∩ w| ≥ O_min(k). The paper also describes five optimizations, gives complexity estimates, and reports runtime and output-size experiments on 19 industrial documents.
Significance. If the completeness theorem were valid, the paper would make a useful contribution: it gives a precise formal notion of near duplicate for documentation and a matching algorithm with a worst-case coverage guarantee, which is rare in this area. The interactive design and the use of a duplicate heat map are sensible, and the experimental corpus of real documents is a strength. However, the central proof is not currently sound: Lemma 2 is false as stated, Lemma 4 is explicitly unproved, and the gap between the proven algorithm and the evaluated implementation is not accounted for. These issues are load-bearing for the main claim, so the paper needs substantial revision before the contribution can be accepted.
major comments (4)
- [Section 6.2, Lemma 2] The proof of Lemma 2 uses the inequality |g| ≤ k|p|, attributing it to Lemma 1. Lemma 1 states k ≤ |g1|/|g2| ≤ 1/k, so with g2 = p it gives |g| ≥ k|p|, not |g| ≤ k|p|. The error is not cosmetic: the lemma itself is false. For k = 0.6, p = 0^60, and g = 1^24 0^36, the archetype 0^36 satisfies Definition 2 for p and g, but d(g,p) = 48, while (1 − k^2)|p| = 38.4. Since Lemma 3 uses Lemma 2 and Theorem 1 relies on Lemmas 3–5, the completeness theorem is not established by the submitted proof.
- [Section 6.2, Lemma 4] The proof of Lemma 4 is omitted, with the text stating 'We omit a formal proof due to its large size.' This lemma is load-bearing: phase 2 replaces each window w1 by a single shrunk fragment w2, and it is not obvious that the selected w2 preserves a sufficiently large intersection with every near duplicate contained in the original window. The main idea described in the paper (considering worst-case shrinkage to k|p| and corner positions) is not a proof. The completeness claim cannot be considered proven without a complete argument for this step.
- [Section 6.3 and Section 8] Optimization 3 is admitted to break completeness in Note 1, yet the experimental section does not state which optimizations were enabled during the reported runs. If Optimization 3 was active, the evaluated implementation is not the algorithm whose completeness is claimed in Theorem 1. Moreover, the experiments report only runtime and output size; there is no ground-truth evaluation of precision or recall. Consequently, the paper does not provide empirical evidence that near duplicates are not missed or that the output is usable beyond being small enough for manual inspection.
- [Section 3.1] The paper says the longest common subsequence distance is used in the proofs and in the algorithm, but the implementation relies on Python's difflib library. difflib's sequence matching is a heuristic and is not guaranteed to compute the exact longest common subsequence distance. This creates a gap between the formal model and the evaluated implementation; the completeness theorem applies only if the distance used in the implementation is exactly the one defined in Section 3.1.
minor comments (5)
- [Abstract] The phrase 'a pattern-based, and the proof of its completeness' is grammatically incomplete; the contribution list needs rewording.
- [Introduction] There is a duplicated article in 'we present an an approach for interactive detection of near duplicates'.
- [Section 6.2, Lemma 2 proof] The proof refers to 'according to (2)' when deriving |A|/|p| ≥ k; the relevant statement is Definition 2, not the completeness criterion in Equation (2).
- [Section 7, Theorem 3] The complexity estimate in Theorem 3 is stated without proof ('We omit the proof due to its large volume'). If this is intended as a contribution, the proof should be supplied or the statement should be marked as conjectural.
- [Section 8] The text 'convertation' should be 'conversion'. Also, the conclusion that most near-duplicate groups are small follows only indirectly from the experiments, since the experiments measure output size rather than the size of user-validated near-duplicate groups.
Circularity Check
No significant circularity: algorithm thresholds are derived from the formal definition, not fitted to the conclusion.
full rationale
The paper's derivation chain is self-contained. The completeness criterion (Eq. 2) is defined independently of the algorithm output, and the proof of Theorem 1 proceeds through auxiliary Lemmas 1-5, which are stated and proven (or at least outlined) within the paper. The threshold k_di in Eq. (1) is not fitted from data or from the desired conclusion; it is obtained by applying the triangle inequality to the bound stated in Lemma 2, so that any window containing a near duplicate is guaranteed to pass the phase-1 test. The formal definition of a near duplicate (Def. 2) does not mention the algorithm or its output, so the completeness claim is not true by definition. The paper's self-citations (e.g., to its earlier near-duplicate definition in [16,37]) are contextual and not load-bearing: the definition is fully restated in the paper. Concerns raised by the skeptic (e.g., the apparent reversal of an inequality in the proof of Lemma 2 and the omitted proof of Lemma 4) are matters of mathematical correctness, not circularity; they do not indicate that any result was assumed through self-citation or that any prediction was constructed from its own inputs. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (2)
- similarity measure k =
user-selected in (1/sqrt(3), 1]
- minimum exact-duplicate group length (four tokens) =
4 tokens
assumptions (3)
- standard math The longest common subsequence distance is a metric and satisfies the triangle inequality.
- domain assumption The edit distance computed by the difflib library equals the theoretical LCS distance used in the proofs.
- domain assumption Real near duplicates in software documentation conform to Definition 2, i.e., share a common archetype covering at least fraction k of each fragment.
Cite this review
Pith. "Pith review of Interactive Duplicate Search in Software Documentation." pith.science (2026). https://pith.science/paper/NDYO3BNH
@misc{pith2026190808266,
author = {Pith},
title = {Pith review of: Interactive Duplicate Search in Software Documentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/NDYO3BNH}},
note = {Machine review of arXiv:1908.08266}
}
read the original abstract
Various software features such as classes, methods, requirements, and tests often have similar functionality. This can lead to emergence of duplicates in their descriptive documentation. Uncontrolled duplicates created via copy/paste hinder the process of documentation maintenance. Therefore, the task of duplicate detection in software documentation is of importance. Solving it makes planned reuse possible, as well as creating and using templates for unification and automatic generation of documentation. In this paper, we present an interactive process for duplicate detection that involves the user in order to conduct meaningful search. It includes a new formal definition of a near duplicate, a pattern-based, and the proof of its completeness. Moreover, we demonstrate the results of experimenting on a collection of documents of several industrial projects.
Figures
Reference graph
Works this paper leans on
-
[1]
Brooks, F. P. The Mythical Man-Month: Es- says on Software Engineering / F. P. Brooks. — Addison-Wesley, 1975
work page 1975
-
[2]
Parnas, D. L. Precise Documentation: The Key to Better Software / D. L. Parnas // The Future of Software Engineering. — Berlin, Hei- delberg: Springer Berlin Heidelberg, 2011. — P. 125–148
work page 2011
-
[3]
Bassett, P. G. Framing Software Reuse: Lessons from the Real World / P. G. Bassett. — Upper Saddle River, NJ, USA: Prentice- Hall, Inc., 1997
work page 1997
-
[4]
Research journey towards indus- trial application of reuse technique / S
Jarzabek, S. Research journey towards indus- trial application of reuse technique / S. Jarz- abek, U. Pettersson // ICSE. — 2006. — P. 608–611
work page 2006
-
[5]
A systematic literature review of software requirements reuse approaches / M
Irshad, M. A systematic literature review of software requirements reuse approaches / M. Irshad, K. Petersen, S. M. Poulding // In- formation & Software Technology. — 2018. — Vol. 93. — P. 223–245
work page 2018
-
[6]
Tool Support for Crosscutting Con- cerns of API Documentation / M
Horie, M. Tool Support for Crosscutting Con- cerns of API Documentation / M. Horie, S. Chiba // Proceedings of the 9th Interna- tional Conference on Aspect-Oriented Software D.V. Luciv, D.V. Koznov, A.A. Shelikhovskii et al. Development. — New York, NY, USA: ACM,
-
[7]
Oumaziz, M. A. Documentation Reuse: Hot or Not? An Empirical Study / M. A. Oumaziz, A. Charpentier, J.-R. Falleri, X. Blanc // Mastering Scale and Complexity in Software Reuse: 16th International Conference on Soft- ware Reuse. — Springer International Publish- ing, 2017. — P. 12–27
work page 2017
-
[8]
Koznov, D. V. DocLine: A method for software product lines documentation development / D. V. Koznov, K. Yu. Romanovsky // Pro- gramming and Computer Software. — 2008. — Vol. 34, no. 4. — P. 216–224
work page 2008
Show all 43 references
-
[9]
Refactoring the Documen- tation of Software Product Lines / K
Romanovsky, K. Refactoring the Documen- tation of Software Product Lines / K. Ro- manovsky, D. Koznov, L. Minchin // Lecture Notes in Computer Science. — Berlin, Heidel- berg: Springer-Verlag, 2011. — Vol. 4980. — P. 158–170
2011
-
[10]
Documentation Reuse: Managing Similar Documents / S
Jarzabek, S. Documentation Reuse: Managing Similar Documents / S. Jarzabek, D. Dan // FedCSIS. — 2017. — P. 1325–1334
2017
-
[11]
Can clone detection support quality assessments of requirements speci- fications? / E
Juergens, E. Can clone detection support quality assessments of requirements speci- fications? / E. Juergens, F. Deissenboeck, M. Feilkas, B. Hummel, B. Schaetz, S. Wag- ner, C. Domann, J. Streit // Proceedings of ACM/IEEE 32nd International Conference on Software Engineering....
2010
-
[12]
Reusable software documenta- tion with phrase annotations / M
Nosál’, M. Reusable software documenta- tion with phrase annotations / M. Nosál’, J. Porubän // Central European Journal of Computer Science. — 2014. — Vol. 4, no. 4. — P. 242–258
2014
-
[13]
Preliminary report on empirical study of repeated fragments in internal doc- umentation / M
Nosál’, M. Preliminary report on empirical study of repeated fragments in internal doc- umentation / M. Nosál’, J. Porubän // Pro- ceedings of Federated Conference on Computer Science and Information Systems. — 2016. — P. 1573–1576
2016
-
[14]
Analysis and visualization of information quality of technical documenta- tion / A
Wingkvist, A. Analysis and visualization of information quality of technical documenta- tion / A. Wingkvist, W. Lowe, M. Ericsson, R. Lincke // Proceedings of the 4th European Conference on Information Management and Evaluation. — 2010. — P. 388–396
2010
-
[15]
Clone Detection in Reuse of Soft- ware Technical Documentation / D
Koznov, D. Clone Detection in Reuse of Soft- ware Technical Documentation / D. Koznov, D. Luciv, H. A. Basit, O. E. Lieh, M. Smirnov // International Andrei Ershov Memorial Conference on Perspectives of Sys- tem Informatics (2015). — Springer Nature,
2015
-
[16]
Luciv, D. V. Detecting Near Duplicates In Software Documentation / D. V. Lu- civ, D. V. Koznov, G. A. Chernishev, A. N. Terekhov, K. Yu. Romanovsky, D. A. Grigoriev // Programming and Computer Software. — 2018. — Vol. 44, no. 5
2018
-
[17]
Koznov, D. V. Duplicate management in software documentation maintenance / D. V. Koznov, D. V. Luciv, G. A. Cherni- shev // Proceedings of V International conference Actual problems of system and software engineering. — Vol. 1989. — CEUR Workshop Proceedings, 2017. — P. 195–201
1989
-
[18]
— URL: http: //www.math.spbu.ru/user/kromanovsky/ docline/index.html
Duplicate Finder. — URL: http: //www.math.spbu.ru/user/kromanovsky/ docline/index.html
-
[19]
Luciv, D. V. Poster: Duplicate Finder Toolkit / D. V. Luciv, D. V. Koznov, G. A. Chernishev, H. A. Basit, K. Yu. Ro- manovsky, A. N. Terekhov // Proceedings of the International Conference on Software En- gineering (ICSE 2018). — 2018. — P. 171–172
2018
-
[20]
Basit, H. A. Efficient Token Based Clone De- tection with Flexible Tokenization / H. A. Ba- sit, S. J. Puglisi, W. F. Smyth, A. Turpin, S. Jarzabek // Proceedings of the 6th Joint Meeting on European Software Engineering Conference and the ACM SIGSOFT Sympo- sium on the Founda...
2007
-
[21]
Identifying duplicate functionality in textual use cases by aligning semantic actions / A
Rago, A. Identifying duplicate functionality in textual use cases by aligning semantic actions / A. Rago, C. Marcos, J. A. Diaz-Pace // Soft- ware & Systems Modeling. — 2016. — Vol. 15, no. 2. — P. 579–603
2016
-
[22]
Finding approximate patterns in strings / E
Ukkonen, E. Finding approximate patterns in strings / E. Ukkonen // Journal of Algorithms. — 1985. — Vol. 6, no. 1. — P. 132–137. Interactive Duplicate Search in Software Documentation
1985
-
[23]
Broder, A. Z. On the resemblance and contain- ment of documents / A. Z. Broder // Compres- sion and Complexity of Sequences 1997. Pro- ceedings. — IEEE, 1997. — P. 21–29
1997
-
[24]
Fast Text Searching: Allowing Errors / S
Wu, S. Fast Text Searching: Allowing Errors / S. Wu, U. Manber // Commun. ACM. — 1992. — Vol. 35, no. 10. — P. 83–91
1992
-
[25]
Landau, G. M. Fast string matching with k dif- ferences/G.M.Landau, U.Vishkin//Journal of Computer and System Sciences. — 1988. — Vol. 37, no. 1. — P. 63–78
1988
-
[26]
A Fast Bit-vector Algorithm for Ap- proximate String Matching Based on Dynamic Programming / G
Myers, G. A Fast Bit-vector Algorithm for Ap- proximate String Matching Based on Dynamic Programming / G. Myers // J. ACM. — 1999. — Vol. 46, no. 3. — P. 395–415
1999
-
[27]
Binary codes capable of cor- recting spurious insertions and deletions of ones / V
Levenshtein, V. Binary codes capable of cor- recting spurious insertions and deletions of ones / V. Levenshtein // Problems of Infor- mation Transmission. — 1965. — Vol. 1. — P. 8–17
1965
-
[28]
Computing Patterns in Strings / W.Smyth.— Addison-Wesley, 2003.— P.423
Smyth, W. Computing Patterns in Strings / W.Smyth.— Addison-Wesley, 2003.— P.423
2003
-
[29]
A survey of longest common sub- sequence algorithms / L
Bergroth, L. A survey of longest common sub- sequence algorithms / L. Bergroth, H. Hako- nen, T. Raita // String Processing and Infor- mation Retrieval, 2000. SPIRE 2000. Proceed- ings. Seventh International Symposium on. —
2000
-
[30]
Mining of massive datasets / J
Leskovec, J. Mining of massive datasets / J. Leskovec, A. Rajaraman, J. D. Ullman. — Cambridge university press, 2014
2014
-
[31]
Algorithms on Strings, Trees, and Sequences / D
Gusfield, D. Algorithms on Strings, Trees, and Sequences / D. Gusfield. — Cambridge Uni- versity Press, 1997
1997
-
[32]
Ratcliff, J. W. Pattern Matching: The Gestalt Approach / J. W. Ratcliff, D. E. Metzener // Dr. Dobb’s Journal. — 1988. — Vol. 13, no. 7. — P. 46–72
1988
-
[33]
Ab- boud, A
Abboud,A.TighthardnessresultsforLCSand other sequence similarity measures / A. Ab- boud, A. Backurs, V. V. Williams // Founda- tions of Computer Science (FOCS), 2015 IEEE 56th Annual Symposium on. — 2015. — P. 59– 78
2015
-
[34]
— URL: https:// docs.python.org/3/library/difflib.html
Python DiffLib module. — URL: https:// docs.python.org/3/library/difflib.html
-
[35]
Abouelhoda, M. I. Replacing Suffix Trees with Enhanced Suffix Arrays / M. I. Abouelhoda, S. Kurtz, E. Ohlebusch // J. of Discrete Algo- rithms. — 2004. — Vol. 2, no. 1. — P. 53–86
2004
-
[36]
Visualization of eye gaze data using heat maps / O
Špakov, O. Visualization of eye gaze data using heat maps / O. Špakov, D. Miniotas // Elek- tronikairelektrotechnika.— 2007.— P.55–58
2007
-
[37]
Luciv,D.V.DetectingNearDuplicatesinSoft- ware Documentation. — 2017. — arXiv : 1711.04705
2017 arXiv
-
[38]
A fast string searching algo- rithm / Robert S
Boyer, Robert S. A fast string searching algo- rithm / Robert S. Boyer, J. Strother Moore // Communications of the ACM. — 1977. — Vol. 20, no. 10. — P. 762–772
1977
-
[39]
— URL: https://pandoc.org/
Pandoc: a universal document converter. — URL: https://pandoc.org/
-
[40]
Drobintsev, P. D. A formal approach to test scenarios generation based on guides / P. D. Drobintsev, V. P. Kotl- yarov, A. A. Letichevsky // Automatic Control and Computer Sciences. — 2014. — Dec. — Vol. 48, no. 7. — P. 415–423
2014
-
[41]
Pakulin, N. V. Model-based testing of Inter- net Mail Protocols / N. V. Pakulin, A. N. Tu- gaenko // Proceedings of the Institute for Sys- tem Programming. — 2011. — Vol. 20. — P. 125–141
2011
-
[42]
To a Method of Evaluating ontolo- gies / V.A.Gorovoy E.S.Bolotnikova, T.A.Gavrilova // Journal of Computer and Systems Sciences International
E.S.Bolotnikova, T.A.Gavrilova V.A.Gorovoy. To a Method of Evaluating ontolo- gies / V.A.Gorovoy E.S.Bolotnikova, T.A.Gavrilova // Journal of Computer and Systems Sciences International. — 2011. — Vol. 50, no. 3. — P. 448–461
2011
-
[2016]
9609 ofLecture Notes in Com- puter Science
— Vol. 9609 ofLecture Notes in Com- puter Science. — P. 170–185
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.