REVIEW 5 major objections 6 minor 21 references
Efficient Online String Matching Based on Characters Distance Text Sampling
T0 review · 5 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper presents a character-distance sampling method for online string matching that, when every consecutive pivot gap is below block size $k$, is lossless: searching the sampled distance sequence finds every occurrence while using…
desk verdict A genuinely novel distance-sampling idea, but the current preprint has load-bearing off-by-one bugs in position reconstruction and a broken Search-0 routine; not correct as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the pair $(\dot{y}, \tau)$: $\dot{y}$ stores each pivot occurrence's position modulo $k$ (one byte when $k=256$), and $\tau$ stores, for each block of $k$ text characters, the index of the last pivot occurrence in that block. Lemma 2 and Corollary 2 supply the reconstruction formula (6), which recovers every consecutive-pivot distance from two consecutive $\dot{y}$ values exactly when that distance is $< k$. This identity is what makes the sampled text a lossless filter rather than a heuristic: it guarantees that a matching distance sequence in the sampled text corresponds to a genuine pattern alignment, and it gives the constant-time position lookup used during verification.
What would settle it
Set $k=3$, choose 'a' as the pivot, and run the algorithm on pattern $x=\texttt{abbba}$ (a at positions 1 and 5, true gap 4) and text $y=\texttt{abbbab}$ (a at positions 1 and 5). The pattern's sampled distance is 4, while relation (6) reconstructs the text gap as $(5 \bmod 3) - (1 \bmod 3) = 1$, so the search reports no candidate and misses the occurrence at position 1; observing that miss on this input shows the bounded-gap condition is essential to the lossless-filter claim.
Extended reading notes
Core claim
The paper's central object is the characters-distance sampled sequence: if the pivot character $c$ occurs at positions $\delta(1), \delta(2), \ldots, \delta(n_c)$, the sampled text is the sequence of gaps $\Delta(i) = \delta(i+1) - \delta(i)$. Since storing every full position would be costly, the algorithm stores only the $k$-bounded positions $\dot{y}[i] = \delta(i) \bmod k$ together with a block table $\tau$, and reconstructs each gap on the fly by the identity $\Delta(i) = \dot{y}[i+1] - \dot{y}[i]$ when $\dot{y}[i+1] > \dot{y}[i]$, and $\Delta(i) = \dot{y}[i+1] + k - \dot{y}[i]$ otherwise. The paper proves this reconstruction is exact whenever every consecutive pivot gap in the text is strictly less than $k$, and that the resulting filter never discards a true occurrence. It then searches the reconstructed distance sequence for the pattern's distance sequence and verifies each candidate in the original text, claiming that the whole procedure inherits linear worst-case and optimal average-case bounds from the underlying string-matching routine.
Load-bearing premise
The distance filter is exact only when every distance between two consecutive pivot characters in the text is strictly less than the block size $k$; if a pivot gap reaches or exceeds $k$, relation (6) wraps modulo $k$ and can reconstruct the wrong distance, causing valid matches to be missed.
Editorial extensions
If this is right
- Choosing the most frequent character as pivot gives the shortest sampled text (one byte per pivot occurrence), about 11% of text size; choosing a rank-20 pivot lowers this to about 2.8%.
- If the underlying search routine is linear in the worst case, the sampled search inherits the same $O(n)$ bound; if the routine is optimal on average, the sampled search inherits $O(n \log_\sigma m / m)$ for patterns up to a few hundred characters when $k=256$.
- On English text the measured search times are 32% below plain Horspool for 2-character patterns and 91% below for 256-character patterns, with preprocessing faster than the earlier occurrence-sampling approach.
- Preprocessing time is lower than the earlier occurrence-sampling method by 15–50%, mainly because the new sampled structure is smaller.
Reading between the lines
- Editorial inference: the same $(\dot{y}, \tau)$ structure gives near-constant recovery of pivot positions, so it could be promoted to a lightweight sparse index that counts or locates pivot-anchored matches without scanning the text.
- Editorial inference: an adaptive block size chosen after measuring the pivot's maximum gap would remove the main correctness restriction while keeping byte-level storage only when gaps are actually small.
- Editorial inference: a practical implementation should detect any text gap that reaches $k$ during preprocessing and fall back to a larger $k$ or to the zero- and one-pivot search strategies; the paper does not analyze such a safeguard.
- Editorial inference: the approach is naturally suited to wide alphabets; on low-entropy alphabets such as DNA the pivot occurs too often, so false-positive verification would dominate, a limitation the paper itself states.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a new sampled string matching algorithm based on sampling the distances between consecutive occurrences of a chosen pivot character. The text is divided into blocks of size k; the sampled sequence stores the position of each pivot occurrence modulo k, and a block-mapping table τ supports reconstruction of the original pivot positions. For patterns containing the pivot at least twice, the algorithm searches for the sampled pattern (the sequence of distances between consecutive pivots) in the sampled text, then verifies candidate positions in the original text. The paper claims O(n) worst-case and O(n log_σ m / m) average-case time under suitable conditions, while using 2.8%–11% of the text size as extra space, and reports experiments showing speedups over the Ots algorithm of Claude et al.
Significance. The characters-distance sampling idea is simple and potentially useful: it offers a very compact partial index and the experiments indicate practical speedups on English text, especially for short patterns. The asymptotic claims, if established, would be a nice complement to existing sampled string matching results. However, the manuscript contains several load-bearing correctness errors in the core reconstruction formula and in the Search-0 pseudocode, as well as a false lemma and an unproven worst-case theorem, so the current version cannot be accepted as a correct and complete algorithm.
major comments (5)
- [§3.1, Corollary 1 and Fig. 1] The reconstruction formula δ(j) = (τ[b] − 1)k + ˙y[j] is incorrect for pivot occurrences whose position is a multiple of k, because in that case ˙y[j] = 0 and the formula returns (τ[b] − 1)k instead of τ[b]k. For example, with k = 5 and δ(j) = 5, the formula returns 0. Since Get-Position is used by all three search procedures to determine the verification offset, this can cause valid occurrences to be missed or spurious verifications to be performed, invalidating the lossless-filter claim as written. The fix is to store residues in {1, . . . , k} or to special-case the zero residue.
- [§3.3, Search-0 (Fig. 5)] The main loop of Search-0 begins at i = 2 and δ1 is never initialized, so the interval before the first pivot occurrence is never searched; any occurrence of x in y[1..δ(1) − 1] is missed. The loop header 'for i≤2 to nc' also contains a typo. The procedure must either start at i = 1 after computing δ1 or handle the first interval separately.
- [§3.1, Lemma 2] Lemma 2 has a sign error: from Corollary 1 the correct difference is ˙y[i+1] + (τ[b] − τ[a])k − ˙y[i], not ˙y[i+1] + (τ[a] − τ[b])k − ˙y[i]. The stated formula gives wrong values when the two pivots are in different blocks; for example, with k = 5, positions 1 and 7 give −4 instead of 6. Although Corollary 2 is correct and is what the search uses, the lemma as printed is false and should be corrected.
- [§3.3, Theorem 3] The proof of the O(n) worst-case bound is not supplied. It asserts that a KMP-style Verify can 'remember all positions of the text which have already been verified,' but no such mechanism is defined, and the preceding line admits that the naive bound is O(nc·m). Since the number of candidate matches can be Θ(nc), the claimed O(n) worst case needs a precise amortized argument or a different verification strategy.
- [§3.2, Corollary 2 and Theorem 3] The correctness of the distance filter depends on relation (6), which is derived under the assumption Δ(i) < k for all consecutive pivot occurrences in the text. The paper states only that the pivot rank 'must be chosen' to satisfy this condition and does not guarantee it for arbitrary inputs; the complexity theorems also do not state this condition explicitly. Please clarify the exact set of inputs for which the algorithm is correct, or extend the method to handle wrap-around distances.
minor comments (6)
- [§3.1, Example 4] The first distance is incorrectly computed as δ(1) − δ(0) = 3 − 1 = 2; with the stated text δ(1) = 1 and the correct expression is δ(2) − δ(1) = 3 − 1 = 2.
- [Fig. 4 and Fig. 6] Verify's loop tests x[i] against y[s+i], which implies s is a zero-based offset, yet Search-1 passes a one-based start (δ_{i−1} − α + 1) and Search-2 passes δ_i − α; this indexing convention should be made consistent.
- [§3.3, references] In §3.3 the BDM algorithm is cited as [12], but reference [12] is Horspool's algorithm; the optimal-average-case citation should be to [6].
- [§4.1, Figure 8] The text refers to 'Table 8' when discussing the space-consumption figure; it should be Figure 8.
- [Conclusion] The conclusion says the algorithm 'may require only 5% of additional extra space,' while the abstract reports 11% to 2.8%; these numbers should be harmonized.
- [Pseudocode] Several pseudocode typos remain: 'for i≤2' should be 'for i←2' in Figs. 5 and 6, and line 10 of Fig. 6 uses the undefined symbol 'pi−1'.
Circularity Check
No significant circularity: complexity results are conditional on the underlying algorithm and the sampled representations are independently defined.
full rationale
The paper's derivation chain is self-contained rather than circular. The sampled sequences are introduced by explicit definitions: ˙y records pivot positions modulo k (Definition 1) and ¯y records consecutive pivot distances (Definition 2). The reconstruction formulas are derived from these definitions and from the mapping table τ, which is independently defined by equation (3). In particular, Corollary 1 and Lemma 2 derive position and distance reconstruction from the definitions, and Corollary 2 (equation (6)) is a specialization under the stated condition Δ(i) < k. No fitted quantity is renamed as a prediction: the measured speedups and space consumption in Section 4 are reported experimental observations, not outputs derived from tuned parameters. The theoretical claims in Theorems 1-3 are explicitly conditional: they assume an underlying linear or optimal-average string matching algorithm and then bound the added overhead from sampling, verification, and table scanning. This is a legitimate complexity preservation argument, not an equivalence of input and output. There is no load-bearing self-citation: the authors' own prior work appears only as experimental tooling or comparative baselines, not as the source of the main correctness or complexity claims. The acknowledged limitation in Section 5 (poor efficiency on small alphabets) and the residue-0 issue in the absolute-position reconstruction formula of Corollary 1 are correctness or scope concerns, not evidence that the derivation reduces to its inputs. Accordingly, the circularity score is 0.
Assumptions & free parameters
free parameters (2)
- k (block size / position modulus) =
256 in practice; integer variable in the paper
- r (rank of the pivot character) =
varies from 1 to 20 in experiments
assumptions (4)
- domain assumption Characters in the text are equiprobable and independent (random text model).
- domain assumption The chosen pivot satisfies Delta(i) < k for every consecutive pair of pivot occurrences in the text and in the pattern.
- domain assumption The underlying online string matching algorithm used on the sampled text has optimal worst-case and average-case complexity.
- standard math Yao's lower bound Theta(n log_sigma m / m) for average-case online string matching is accepted as the optimal benchmark.
Cite this review
Pith. "Pith review of Efficient Online String Matching Based on Characters Distance Text Sampling." pith.science (2026). https://pith.science/paper/A5I764B5
@misc{pith2026190805930,
author = {Pith},
title = {Pith review of: Efficient Online String Matching Based on Characters Distance Text Sampling},
year = {2026},
howpublished = {\url{https://pith.science/paper/A5I764B5}},
note = {Machine review of arXiv:1908.05930}
}
read the original abstract
Searching for all occurrences of a pattern in a text is a fundamental problem in computer science with applications in many other fields, like natural language processing, information retrieval and computational biology. Sampled string matching is an efficient approach recently introduced in order to overcome the prohibitive space requirements of an index construction, on the one hand, and drastically reduce searching time for the online solutions, on the other hand. In this paper we present a new algorithm for the sampled string matching problem, based on a characters distance sampling approach. The main idea is to sample the distances between consecutive occurrences of a given pivot character and then to search online the sampled data for any occurrence of the sampled pattern, before verifying the original text. From a theoretical point of view we prove that, under suitable conditions, our solution can achieve both linear worst-case time complexity and optimal average-time complexity. From a practical point of view it turns out that our solution shows a sub-linear behaviour in practice and speeds up online searching by a factor of up to 9, using limited additional space whose amount goes from 11% to 2.8% of the text size, with a gain up to 50% if compared with previous solutions.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Apostolico, The myriad virtues of suffix trees, in: A
A. Apostolico, The myriad virtues of suffix trees, in: A. Apo stolico, Z. Galil (Eds.), Combinatorial Algorithms on Words, Vol. 12 of NATO Advanced Science Institutes, Series F, Springer-Verlag, pp. 85–96 (1985)
work page 1985
-
[2]
R.S. Boyer, J.S. Moore. A fast string searching algorithm . Commun. ACM 20(10), 762-772 (1977)
work page 1977
-
[3]
D. Cantone, S. Faro, E. Giaquinta: Adapting Boyer-Moore- like Algorithms for Searching Huffman Encoded Texts. Int. J. Found. Comput. Sci. 23(2), pp. 343–356 (2012)
work page 2012
-
[4]
D. Cantone, S. Faro, A. Pavone: Speeding Up String Matchin g by Weak Factor Recognition. Stringology 2017, pp. 42–50 (2017)
work page 2017
- [5]
-
[6]
M. Crochemore, A. Czumaj, L. Gasieniec, S. Jarominek, T. L ecroq, W. Plandowski, W. Rytter, Speeding up two string-matching algorithms, Alg orithmica 12 (4), pp. 247–267 (1994)
work page 1994
-
[7]
S Faro, T Lecroq, The Exact Online String Matching Problem : a Review of the Most Recent Results, ACM Computing Surveys (CSUR) vol. 45 (2 ), pp. 13 (2013)
work page 2013
-
[8]
S. Faro, T. Lecroq, S. Borz ` ı, S. Di Mauro, and A. Maggio. Th e String Matching Algorithms Research Tool. In Proc. of Stringology , pages 99–111, 2016
work page 2016
Show all 21 references
-
[9]
Ferragina, G
P. Ferragina, G. Manzini. Indexing compressed text. Jour nal of the ACM, 52 (4), pp. 552–581, 2005
2005
-
[10]
Fredriksson and S
K. Fredriksson and S. Grabowski, A general compression a lgorithm that supports fast searching, Information Processing Letters, vol. 100 ( 6), pp. 226–232 (2006)
2006
-
[11]
Grabowski, M
S. Grabowski, M. Raniszewski. Sampling the suffix array wi th minimizers. In Porc. of String Processing and Information Retrieval (SPIRE 2015 ), Lecture Notes in Com- puter Science, vol 9309, Springer, pp. 287–298 (2015)
2015
-
[12]
R. N. Horspool, Practical fast searching in strings, Sof tware: Practice & Experience 10 (6), pp. 501–506 (1980)
1980
-
[13]
Karkkainen, E
J. Karkkainen, E. Ukkonen, Sparse suffix trees, in: Proc. 2 nd Annual International Conference on Computing and Combinatorics (COCOON), LNCS 1 090, pp. 219–230 (1996)
1996
-
[14]
Klein, D
S.T. Klein, D. Shapira. A new compression method for comp ressed matching. In: Data Compression Conference, IEEE. pp. 400–409 (2000)
2000
-
[15]
D. E. Knuth, J. H. Morris, V. R. Pratt, Fast pattern matchi ng in strings, SIAM J. Comput. 6 (2), pp. 323–350 (1977)
1977
-
[16]
A text compression scheme that allows fast searc hing directly in the com- pressed file
Manber. A text compression scheme that allows fast searc hing directly in the com- pressed file. ACM Trans. Inform. Syst., 15(2), pp.124–136 (1 997)
-
[17]
Manber, G
U. Manber, G. Myers, Suffix arrays: A new method for online s tring searches, SIAM J. Comput. 22 (5), pp. 935–948 (1993)
1993
-
[18]
Moura, G
E. Moura, G. Navarro, N. Ziviani, and R. Baeza-Yates. Fas t and flexible word searching on compressed text. ACM Transactions on Informat ion Systems (TOIS), 18(2), pp.113–139 (2000)
2000
-
[19]
Navarro, J
G. Navarro, J. Tarhio. LZgrep: A Boyer-Moore string matc hing tool for Ziv-Lempel compressed text. Software Practice & Experience, vol. 35, p p. 1107–1130 (2005)
2005
-
[20]
Shibata, T
Y. Shibata, T. Kida, S. Fukamachi, M. Takeda, A. Shinohar a, T. Shinohara, S. Arikawa: Speeding Up Pattern Matching by Text Compression. CIAC 2000: pp. 306– 315
2000
-
[21]
A. C. Yao, The complexity of pattern matching for a random string, SIAM J. Comput. 8 (3), pp. 368–387 (1979)
1979
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.