{"id":"92bebefd-5a0c-4198-8d63-9c25cbf9e20a","arxiv_id":"2505.11451","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper applies positive-example regular expression synthesis to date extraction from medical document transcriptions, reporting a precision/recall tradeoff over hand-built regexes.","lead":"This paper tests whether regular expressions generated automatically from example dates can pick dates out of transcribed medical records better than hand-written ones. The authors report that the synthesized patterns catch slightly fewer real dates but produce fewer false alarms, though the paper's own statistics are internally inconsistent.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Synthesized regexes' precision gain may stem from training grammar that omits month-range and contextual date classes present in the bespoke regex.","rationale":"The reader's weakest_assumption identifies the synthetic positive-example vocabulary as hand-selected and untested for generality, and I agree. The most load-bearing concern about the central claim is that the apparent precision improvement may be an artifact of the training grammar omitting date classes that the bespoke regex was designed to capture. Sec IV-G explicitly says month-to-month ranges were not synthesized, and Sec IV-H lists contextual dates, keyword synonyms, and cross-month ranges as future work; these are exactly the complex cases the bespoke regex handled. Without a breakdown of the 228 false negatives or an ablation over grammar coverage, the reader cannot tell whether the synthesis method genuinely trades a little recall for much higher precision, or whether it simply sidesteps the hard cases. This is distinct from the precision/recall formula swap, which is a serious presentation error but can be corrected directly from the confusion matrices. Since the data and code are not available and the statistical mislabeling already warrants rejection, the reader's REJECT verdict should stand. My concern reinforces that verdict rather than changing it, so I recommend UNCHANGED.","tokens_in":10862,"tokens_out":7903,"duration_ms":83354,"concrete_test":"Classify all 228 false negatives of the synthesized regex into (a) month-to-month ranges, (b) day ranges spanning different months, (c) ordinal-plus-preposition or synonym variants absent from the grammar, (d) OCR-induced errors, (e) other. Then rerun Regex+ with an augmented positive-example grammar that adds month-to-month ranges and synonyms (e.g., 'to the'), regenerates timestamps, and recomputes the Fig. 5 confusion matrix on the same 20 transcriptions. If the augmented regex's false positives rise substantially toward the bespoke 2570, or if the false-negative reduction is minimal, the reported tradeoff is not a general property of synthesis but a consequence of omitting hard date classes. A second useful check is to repeat the entire comparison on a held-out set of documents not used to design either the bespoke regex or the synthetic grammar.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central comparison (Fig. 4 vs Fig. 5) contrasts a bespoke regex explicitly built to capture long dates and date ranges, including month-to-month expressions (Sec IV-F), with a synthesized regex whose training grammar deliberately excludes month-to-month ranges, contextual dates, and keyword synonyms (Sec IV-G states 'The months did not have synthesised ranges between them'; Sec IV-H lists these as future work). The 228 false negatives of the synthesized regex are never decomposed by cause. If a large share are month-range, contextual, or synonym-bearing dates that the generator never produced as positive examples, then the observed false-positive reduction and 'slight' recall loss are an artifact of narrowing the grammar before synthesis, not a property of regex synthesis itself. The paper's assertion that 'we captured the entire input space' is therefore unsupported: the input space was restricted to a hand-selected vocabulary (Sec IV-A, IV-G), and the test corpus is the same 20 documents used to motivate that vocabulary. The reader's statistical errors are real, but the confusion matrices still speak for themselves; this grammar gap more directly threatens the central claim that synthesis yields a favorable precision/recall tradeoff for complex dates and date ranges.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an explainable pipeline for extracting dates and date ranges from medical document images. Documents are transcribed with OCR/HCR tools, and the text is then matched against regular expressions. The authors compare three kinds of regexes: highly accessible public regexes, a manually crafted 'bespoke' regex, and regexes produced by a regular expression synthesiser (Regex+) trained on positive-only examples generated by reverse-engineering UNIX timestamps for every date between 1900 and 2100. The central claim is that the synthesized regexes detect far fewer date-like non-dates than the bespoke regex (1402 vs 2570 false positives) at a modest cost in missed real dates (228 vs 160 false negatives), thereby establishing regular expression synthesis as a practical, explainable route to complex date extraction. The paper also reports a small experiment showing an LLM produces unreliable UNIX timestamps.","tokens_in":10992,"tokens_out":6095,"duration_ms":56727,"significance":"The reverse-engineering idea is genuinely interesting: UNIX timestamps have a one-to-many relationship with textual date forms, and converting timestamp ranges into synthetic positive examples for regex synthesis is a novel and potentially useful technique. The evaluation uses real transcribed medical documents with human annotations, which is a strength, and the confusion matrices are included, allowing independent recalculation of the metrics. The explainability motivation is well placed. However, the significance is currently undercut by three issues: swapped precision/recall definitions and labels throughout, a synthetic grammar that omits date-range and contextual forms present in the test corpus, and a same-corpus evaluation without a held-out set. These are repairable but require corrected reporting and additional experiments.","major_comments":[{"comment":"The precision and recall formulas in Sec. II-F are swapped relative to standard definitions: precision should be TP/(TP+FP) and recall should be TP/(TP+FN). The caption labels are likewise swapped; for Fig. 4 the matrix gives standard precision 30.35% and standard recall 87.5%, but the caption reports 'Recall: 30.35%, Precision: 87.5%'. Fig. 5's caption also contains an internal inconsistency: with TP=1052 and FN=228 the standard recall is 82.19%, not 82.96%. Every precision/recall value in the paper therefore needs to be recalculated and the definitions corrected; as written, no quantitative claim based on these metrics is reliable.","section":"Sec. II-F, Figs. 3-5"},{"comment":"The synthetic grammar explicitly excludes month-to-month ranges, contextual dates, and keyword synonyms, and the 20 test documents are the same ones used to motivate the vocabulary. The claim in Sec. IV-G that 'we captured the entire input space' is therefore unsupported: the input space was deliberately restricted. The 228 false negatives of the synthesized regex are never decomposed by cause, so the reader cannot tell whether the precision gain comes from the synthesis procedure itself or simply from a narrower grammar. An ablation that adds the excluded date types to the synthetic examples, or at least a breakdown of false negatives by date type, is needed to support the central comparison.","section":"Sec. IV-G, IV-H"},{"comment":"The bespoke regex was developed on 81 pages from 2 of the 20 documents, and all 20 documents are then used in the main comparison (Figs. 4 and 5). Because the synthetic vocabulary was also derived from the same documents, the evaluation is optimistic for both methods and likely overstates performance on unseen documents. A held-out document set or document-level cross-validation is required before the generalization claim can be made.","section":"Sec. IV-F, IV-A"},{"comment":"The first bullet of the three research findings states that regex synthesis 'achieved higher recall by sacrificing precision.' This directly contradicts the corrected numbers in Figs. 4 and 5, where the synthesized regex has lower recall (82.2% vs 87.5%) and higher precision (42.9% vs 30.4%). This internal contradiction in the discussion of the main result must be resolved.","section":"Sec. IV-H"}],"minor_comments":[{"comment":"The manuscript contains numerous typographical errors (e.g., 'seperators', 'furtherest', 'prepropositions', 'Georgian months' for 'Gregorian months', 'covert' for 'convert', 'seperated'); a careful proofreading pass is needed.","section":"Throughout"},{"comment":"The verbal definition of false positives ('the number of target dates that were wrongly predicted') is incorrect; false positives are non-date strings that the regex matches as dates. This should be reworded alongside the corrected formulas.","section":"Sec. II-F"},{"comment":"The figure caption says dates between 1970 and 2031 while the text says 1970-2030; the ranges should be made consistent.","section":"Fig. 2"},{"comment":"The confusion matrices do not state explicitly whether rows are targets and columns are predictions; please add a sentence or axis labels to clarify the layout.","section":"Figs. 3-5"},{"comment":"The introduction states that 'computer vision and Large Language Models are not able to solve this task,' but only a single small LLM experiment is reported and no computer vision experiments were actually run on the task; the claim should be qualified accordingly.","section":"Sec. I"}],"recommendation":"major_revision","confidential_remarks":"The core idea is worth developing, and the reader's statistical points are correct. I do not see a need for rejection if the authors are willing to correct the metric definitions, recompute all values, add an error decomposition or ablation that accounts for the omitted grammar classes, and evaluate on held-out documents. The pervasiveness of the metric errors means the revision will be substantial, but the problems are within the scope of the manuscript to fix."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is worth a look: reverse-engineering UNIX timestamps to generate positive examples for a regex synthesizer is a neat practical trick for a narrow but real data-cleaning task. The paper does something concrete on real annotated medical transcriptions, and the confusion matrices are at least recoverable despite the caption errors. There is a genuine, modest contribution buried under the presentation problems.\n\nThe problems are real and load-bearing. Precision and recall are defined with swapped formulas in Sec II-F, and the captions of Figures 4 and 5 assign the same swapped labels. The Discussion then says the synthesized regex achieved \"higher recall by sacrificing precision,\" which contradicts both the abstract and the numbers in the matrices. As written, every reported precision/recall value is ambiguous, and I would not trust any of the prose claims without re-deriving them from the confusion matrices.\n\nThe stress-test point lands too. The synthesized regex was trained on a grammar that omits month-to-month ranges, contextual dates, and keyword synonyms, while the bespoke regex was explicitly built to capture those forms. So the observed false-positive reduction and slight recall loss may be partly an artifact of narrowing the training vocabulary, not a property of regex synthesis. The paper claims it \"captured the entire input space,\" but that is unsupported, and the error decomposition by cause is never given. This is not a minor caveat; it directly qualifies the central claim.\n\nThere are smaller issues: the stated leap-year rule is wrong (divisible by 400 is a leap year), the CV/LLM dismissal rests on a very narrow experiment, and no code, data, or synthesized regexes are released, which makes verification hard. The cited literature is honestly engaged, and self-citation is not a problem here.\n\nWould I send this to peer review? Yes, with a serious referee and an explicit request for major revision: fix the metric definitions, re-report all numbers from the confusion matrices, decompose the false negatives by date class, release the code and synthesized regexes, and temper the claims about the input space. The underlying idea deserves referee time, but as written I would not cite or rely on its numbers.","headline":"Plausible practical trick, but the paper garbles its own metrics and the central comparison may be less favorable than it looks once you notice the synthesized regex's training grammar omits exactly the date-range forms the bespoke regex handles.","tokens_in":11599,"tokens_out":1533,"would_cite":false,"duration_ms":18092,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Regular expressions synthesized from reverse-engineered UNIX timestamps extract medical dates with far fewer false positives than hand-built ones.","keywords":["regular expression synthesis","date extraction","UNIX timestamps","explainable AI","medical records","optical character recognition","positive examples"],"falsifier":"Run the pipeline on a held-out collection of medical documents that deliberately include date expressions absent from the generator's vocabulary—month names leading the date, centuries written as words, or date ranges spanning different months. If the synthesized regexes miss those dates at a rate far higher than the bespoke regexes (for example, recall falling well below the 82.96% observed here), the claim that reverse-engineered timestamps cover the space of real date expressions is falsified.","tokens_in":10558,"feed_emoji":"📅","tokens_out":10518,"duration_ms":93043,"temperature":0.7,"pith_summary":"The paper sets out to show that date and date-range extraction from medical documents can be automated without giving up explainability: instead of hand-writing regular expressions, one can synthesize them from positive examples generated by reverse-engineering UNIX timestamps. For every date between 1 January 1900 and 31 December 2100, the paper builds the timestamp, pulls out the day, month, and year, and renders each timestamp into many textual variants; a regular-expression synthesiser then selects a compact regex. On OCR transcriptions of 20 medical documents containing 1,280 annotated dates, the synthesized regexes produced 1,402 date-like false positives versus 2,570 for the manually crafted ones, while missing 228 true dates versus 160. The paper offers this as evidence that the long-standing trade-off between explainable rules and automated learning can be shifted: deterministic logic can be learned from the full input space, not just from examples. This matters because medical decisions hinge on dates, and opaque AI outputs are hard to defend.","feed_headline":"Reverse-engineered timestamps teach regexes to extract medical dates","feed_subtitle":"Synthesized from every timestamp 1900–2100, they flag 1,402 date-like non-dates versus 2,570 for hand-built ones.","key_machinery":"The load-bearing mechanism is a synthetic dataset built by reverse-engineering UNIX timestamps. The paper generates the timestamp for every date from 1900 through 2100, decomposes it into day, month, and year, and renders it as many textual forms—numeric with zero-padding, month names and abbreviations, ordinal quantifiers, separators '-', '.', '/', and space, and hyphenated day ranges—using a hand-selected vocabulary. These positive examples are fed to a regular-expression synthesiser (the paper's implementation of Regex+) that builds a directed acyclic graph of candidate regexes and picks the one with the lowest cost under a minimum-description-length-style measure balancing specificity and simplicity. Because the examples cover the entire date range, synthesis runs without negative examples; the regexes are stored in key:value maps that specify which captured group yields which date part, keeping the final extraction decomposable and inspectable.","core_discovery":"The central claim is that regular expressions created by regular-expression synthesis—fed with positive examples obtained by reverse-engineering UNIX timestamps—detect far fewer sequences of text that look like dates than manually created regexes, at the cost of a slight increase in missed real dates. The paper quantifies this on 20 medical documents: synthesized regexes found 1,052 true dates, missed 228, and flagged 1,402 false positives; bespoke hand-built regexes found 1,120, missed 160, and flagged 2,570 false positives. The mechanism is a many-one mapping: each UNIX timestamp corresponds to multiple text representations, so enumerating all timestamps in a range and rendering them in several formats covers the intended input space, making negative examples unnecessary. The paper presents the overall pipeline—reverse-engineer a many-one mapping, synthesize positive examples, feed them to a synthesiser—as a new approach to learning deterministic, explainable logic for structured data.","pith_inferences":["The same reverse-engineering trick may generalize to any field where a canonical value (a timestamp, serial number, coordinate, or barcode) has many textual renderings: generate the full numeric range, render variants, and let a synthesiser learn the extractor.","A failure analysis of the 228 missed dates would likely show which real-world date shapes are missing from the hand-selected vocabulary; adding those shapes to the generator is a concrete test of the method's ceiling.","The 1,402 residual false positives may stem from format ambiguity (for example, '03/04/1998' being both 3 April and 4 March); pairing the regexes with a calendar-validity post-check or a small negative-example set could raise precision without losing decomposability.","Extending the generator to date ranges that cross month boundaries, which the paper lists as future work, could be done by enumerating timestamp intervals rather than combinatorial string templates."],"forward_implications":["Medical date extraction can be kept explainable: the synthesized regexes decompose into day, month, and year parts that map directly to UNIX timestamps.","Because the synthesiser is re-run on freshly generated examples, adding a new date format, separator, or language requires no manual regex editing.","The roughly halved false-positive count (1,402 versus 2,570) makes the synthesized regexes attractive where date-like text is expensive to review, even at 68 additional missed dates.","The reverse-engineering pipeline is proposed as a general recipe for learning deterministic logic from many-one mappings, not just for dates."],"supporting_citations":[{"why":"Supplies the regular-expression synthesis algorithm (Regex+) that the paper implements to turn positive examples into regexes.","marker":"[23]"},{"why":"Establishes the prior technique of generating examples to infer automata without negative examples, which the paper adapts to restrict generalisation.","marker":"[24]"},{"why":"Shows that interpretable properties can be learned from positive examples only, underpinning the decision not to construct negative examples.","marker":"[19]"},{"why":"Provides the computer-vision baseline for locating date regions on handwritten documents, which the paper argues cannot yield day/month/year parts or timestamps.","marker":"[3]"},{"why":"Represents the earlier HMM-based date-field extraction approach that the paper contrasts with regex-based extraction.","marker":"[2]"},{"why":"The large language model tested as a timestamp generator; its inconsistent outputs motivate the regex route.","marker":"[26]"},{"why":"Provides the open-source vision-language transcription model used to turn page images into text for regex matching.","marker":"[32]"}],"fun_headline_variants":["Timestamp-derived regexes cut false date hits","Reverse-engineered timestamps beat hand-coded date regexes","Fewer date false positives from timestamp-synthesized regexes","Timestamp-reversed regexes slash false dates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline's reach is capped by the manually chosen vocabulary of date formats, separators, month spellings, and year rules used to render the synthetic examples; if real documents contain date expressions outside that vocabulary, the synthesized regexes will miss them, and the paper gives no evidence the vocabulary extends beyond the 20 sampled documents.","fun_headline_variants_meta":{"raw":{"variants":["Timestamp-derived regexes cut false date hits","Reverse-engineered timestamps beat hand-coded date regexes","Fewer date false positives from timestamp-synthesized regexes","Timestamp-reversed regexes slash false dates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000972,"raw_usage":{"total_tokens":4166,"prompt_tokens":1010,"completion_tokens":3156,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":3093}},"tokens_in":626,"tokens_out":3156,"duration_ms":24381,"temperature":1.0,"reasoning_tokens":3093,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:53:30.457640+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pipeline on a held-out collection of medical documents that deliberately include date expressions absent from the generator's vocabulary—month names leading the date, centuries written as words, or date ranges spanning different months. If the synthesized regexes miss those dates at a rate far higher than the bespoke regexes (for example, recall falling well below the 82.96% observed here), the claim that reverse-engineered timestamps cover the space of real date expressions is falsified.","supporting_citations":[{"cited_title":"Regex+: Synthesizing regular expressions from positive examples,","cited_arxiv_id":null,"evidence_quote":"Supplies the regular-expression synthesis algorithm (Regex+) that the paper implements to turn positive examples into regexes."},{"cited_title":"Inferring dfa without negative exam- ples,","cited_arxiv_id":null,"evidence_quote":"Establishes the prior technique of generating examples to infer automata without negative examples, which the paper adapts to restrict generalisation."},{"cited_title":"Learning interpretable temporal properties from positive examples only,","cited_arxiv_id":null,"evidence_quote":"Shows that interpretable properties can be learned from positive examples only, underpinning the decision not to construct negative examples."},{"cited_title":"DateFinder: detecting date regions on handwritten document images based on positional expectancy,","cited_arxiv_id":null,"evidence_quote":"Provides the computer-vision baseline for locating date regions on handwritten documents, which the paper argues cannot yield day/month/year parts or timestamps."},{"cited_title":"Date field extraction from handwritten documents using hmms,","cited_arxiv_id":null,"evidence_quote":"Represents the earlier HMM-based date-field extraction approach that the paper contrasts with regex-based extraction."}],"review_version":1}