Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

MarkushGlyph and OCSRGlyph: Improved Chemical Structure Recognition

T0 review · 4 major / 6 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single image-to-text model can read both ordinary molecules and Markush patent drawings, with stereo-aware training lifting full-stereo OCSR to 93.8% exact match.

desk verdict Solid infra paper: stereo-curated OCSR is the clean win; Markush SOTA is real but thinner than the abstract sells, and they mostly own the gaps. read the letter →

arxiv 2607.28532 v1 pith:4ST3ZSDC submitted 2026-07-30 cs.CV

Alex Andonian , Samuel G Rodriques , Andrew D White , Siddharth M Narayanan This is my paper · ORCID
classification cs.CV
keywords opticalchemicalstructurerecognitionMarkushstructuresCXSMILESvision-languagemodelstereochemistryimage-to-sequencepatentinformaticsparsed-graphequality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Chemistry is still mostly published as pictures. Turning those pictures into line notations is how databases get indexed and how training sets for chemistry models get built. This paper treats both ordinary molecule drawings and Markush structures (drawings that describe a whole family of related compounds) as one image-to-text problem. For single molecules it shows that carefully oversampling chiral examples closes most of the remaining stereochemistry errors and reaches 93.8% full-stereo exact match on the standard USPTO set. For Markush drawings it shows that one early-fusion vision-language model reading the whole crop—scaffold, R-labels, brackets, and printed text together—beats multi-stage pipelines that split vision and text. It also replaces a looser prior score with a strict atom-mapping test that rejects extra or swapped variable groups the older metric missed.

What carries the argument

cxsmiles_opt plus strict parsed-graph equality: an optimized CXSMILES record with inline R-labels, positional-variation and repeat-unit annotations, used as the shared target for both tasks; equality is decided by finding one atom bijection (VF2-style) that preserves the molecular graph, R-group assignments, attachment sets, repeat units, and fragments—not by per-feature recall that can credit extras or label swaps.

What would settle it

Re-annotate a fixed slice of IP5-M/M2S/USPTO-Markush with independent chemist-verified CXSMILES (including stereo and full substituent text), then re-score MarkushGlyph and the prior best system under strict parsed-graph equality; if the ranking or the reported margins reverse, the central Markush claim fails.

Watch

Extended reading notes

Core claim

OCSR and Markush translation can be solved as a single image-to-sequence task. Stereo-enriched training data makes a compact encoder-decoder state of the art on full-stereo OCSR (93.8% canonical exact match on 5,719 USPTO images). An early-fusion vision-language model that ingests the entire Markush image in one pass, without a separate OCR or layout stage, exceeds prior multi-stage systems on IP5-M, M2S, and USPTO-Markush under both the legacy score and a new strict parsed-graph equality metric; mixing a small share of ordinary molecules into training lets the same model handle both input types.

Load-bearing premise

The published Markush benchmark labels and the graph-level scorers are treated as ground truth for a correct translation, even though neither scorer checks the free-text substituent definitions and the older score ignores stereochemistry.

Editorial extensions

If this is right

  • Patent and literature bulk parsers can run one model without first deciding whether a figure is a single molecule or a Markush family.
  • Stereo-aware curriculum alone is enough to push full-stereo OCSR past prior reported numbers on the shared USPTO benchmark.
  • Strict atom-mapping equality becomes the fairer public yardstick, because it fails predictions that add or transpose variable groups the older recall-style score accepted.
  • Majority vote over a handful of samples further lifts Markush accuracy without changing the architecture.
  • Unifying both tasks in one record format makes ordinary molecules the empty-variable special case of Markush output.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If substituent-definition text were folded into the strict metric, measured Markush accuracy would likely drop and expose a second error mode the current tables do not show.
  • The remaining OCSR gap inside the unified Markush model suggests a natural next experiment: scale the ordinary-molecule mixture or add stereo-rich Markush synthetics until one checkpoint matches specialist OCSR.
  • A strict scorer that also demands stereo agreement would tighten the Markush leaderboard the same way canonical exact match tightened OCSR comparisons.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper frames OCSR and Markush structure recognition as image-to-text problems and presents two models: OCSRGlyph, a compact Swin encoder–Transformer decoder trained with stereochemistry-enriched data, and MarkushGlyph, a LoRA-tuned Qwen3.5-2B VLM that reads the full patent crop (structure plus printed text) in one early-fusion stage and emits a unified cxsmiles_opt record. On the 5,719-image USPTO OCSR benchmark, OCSRGlyph reports 93.8% canonical (full-stereo) exact match, above prior published numbers under matched stereo conventions. MarkushGlyph is reported to exceed MarkushGrapher-2 and earlier systems on IP5-M, M2S, and USPTO-Markush under both the legacy cxsmi_equality score and a new strict parsed-graph equality metric based on feature-constrained VF2 isomorphism. The authors also ablate stereo oversampling and the ordinary-molecule mixture share, release code/weights/manifests, and document scorer validation in the appendix.

Significance. If the results hold under tightened evaluation, the work is a clear incremental advance for chemical document AI: a simple stereo-curation recipe that moves full-stereo OCSR past the previous best reported USPTO numbers, and a single-stage VLM that matches or beats multi-stage Markush pipelines without a separate OCR/layout front-end. The strict parsed-graph equality metric is a genuine methodological contribution—it exposes a real failure mode of the prior recall-style scorer (extra/transposed annotations on symmetric scaffolds) and is backed by VF2/RDKit differential checks, fuzz, and determinism tests. Public weights, training/eval code, and prediction dumps materially raise the bar for reproducibility in this subfield. The remaining gap is that Markush accuracy still lags OCSR by a large margin and that the Markush SOTA margin under the stricter metric is thin on the largest set.

major comments (4)
  1. [Table 3, §5.1] Table 3, strict parsed-graph equality, IP5-M column: MarkushGlyph greedy is 52.1% vs MarkushGrapher-2 at 51.1% (≈9/878 examples). That delta is within ordinary binomial noise and is the main support for “exceeds prior systems … under strict equality” on the largest public set. Please report Wilson/bootstrap CIs (or a paired McNemar test on the same 878 IDs) for all three benchmarks under both scorers, and qualify the abstract/§5.1 SOTA wording where the strict IP5-M edge is not significant. Majority-vote (54.7%) may still clear the bar; the single-greedy claim currently does not on IP5-M alone.
  2. [Table 3, §5.1] Table 3 footnote and §5.1: the authors’ MarkushGrapher-2 reproduction on USPTO-Markush reaches only 41.9% under both metrics versus the published 55.0% MarkushGrapher-2 score. The paper still headlines “exceeds … on all three” while mixing published MG2 numbers (cxsmi_equality) with a self-run strict baseline that could not match the published score on one of three sets. This is load-bearing for the comparative claim. Please (i) diagnose the gap (checkpoint, preprocessing, example filter, CXSMILES canonicalization, or denominator), (ii) report a single consistent MG2 baseline under both scorers from one public inference recipe, and (iii) mark any unreproduced cell as such in the main table rather than only in a footnote.
  3. [§4.4, Appendix A.3, Table 3] §4.4 and Appendix A.3 state that neither Markush metric scores the <stable> substituent-definition text, even though MarkushGlyph is trained to emit it and M2S includes that text in the image and target. For M2S in particular, “correct Markush translation” as scored can ignore errors or omissions in the definition prose the model is asked to produce. Either add a simple exact/normalized match (or field-level F1) on <stable> for M2S and report it beside Table 3, or explicitly scope all Markush accuracy claims to “structure+annotations only, substituent text unscored” in the abstract, contributions list, and §5. Leaving the omission only in the methods appendix overstates what the tables measure.
  4. [Table 2, Figure 5, §1, §6] Table 2 vs Table 3 / Figure 5: MarkushGlyph’s USPTO OCSR canonical exact match is 69.0% (5% ordinary-molecule mix) against OCSRGlyph’s 93.8%, and the conclusion correctly notes the unification gap. The multi-task claim in §1 and §6 (“without a priori knowledge of what type of chemical structure is being parsed”) is only weakly supported at current OCSR accuracy. Please either (i) show a practical router/ensemble result (MarkushGlyph for Markush pages, OCSRGlyph for molecule-only crops) with end-to-end PDF-style numbers, or (ii) soften the unified-parser framing to match the measured OCSR gap and the 0–15% mixture ablation.
minor comments (6)
  1. [Figure 3] Figure 3 omits strict parsed-graph bars; pointing readers only to Table 3 makes the looser metric visually dominant. Consider a paired bar plot or dual-axis figure for both scorers.
  2. [§3.1] §3.1 / Appendix A.2: cxsmiles_opt is described clearly, but a short side-by-side of one structure in standard CXSMILES vs cxsmiles_opt in the main text (not only the Rv example) would help non-ChemAxon readers.
  3. [Table 2, Appendix A.1] Table 2 baseline cells mix “own reported” and MolSight-re-reported values (mark m). A one-column note listing canonicalizer version (RDKit) and stereo flags per baseline would reduce residual protocol doubt.
  4. [§5.3, Appendix C] Appendix C Table 4 is strong validation; citing the 541 vs 25 disagreement asymmetry once in §5.3 main text (not only the figure) would better motivate adopting the strict metric as primary.
  5. [Abstract, Appendix A.3] Minor wording: abstract “state-of-the-art OCSR model” is fine under canonical match; ensure the same phrase is not applied unqualified to Markush under strict IP5-M until CIs are shown. Also fix “the a strict” typo in Appendix A.3.
  6. [References] References include 2026 system cards and arXiv preprints (MarkushGrapher-2, MolSight, etc.); keep versions/dates pinned in the camera-ready for longevity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical image-to-text models evaluated on held-out external benchmarks with disjoint training splits.

full rationale

This is a standard empirical CV/cheminformatics paper. OCSRGlyph and MarkushGlyph are trained on curated image–string mixtures (PubChem, USPTO-680K, synthetic/real Markush sources) that the authors state are disjoint from the USPTO OCSR eval split and from the public IP5-M/M2S/USPTO-Markush sets. Reported accuracies are exact-match or graph-isomorphism scores against fixed external labels, not quantities derived from fitted parameters that redefine the target. Stereo enrichment and ordinary-molecule mixture ablations change the training curriculum and are re-measured on the same held-out benchmarks; that is experimental design, not a fit-called-prediction. Adoption of cxsmiles_opt and the legacy cxsmi_equality scorer from MarkushGrapher-2 is format/metric reuse from prior external work, not a self-citation uniqueness chain. The new strict parsed-graph equality metric is an additional, stricter grader the authors apply uniformly to their model and to a reproduced baseline; introducing a metric and also winning under the prior metric does not make the result true by construction. No load-bearing step reduces Eq./claim X to its own inputs. Evaluation caveats (unreproduced MG2 USPTO-M number, unscored <stable> text) are correctness/robustness issues, not circularity.

Assumptions & free parameters 4 free parameters · 6 assumptions · 3 invented entities

This is an empirical CV/ML methods paper. The central claims rest on standard learning assumptions, public benchmark labels as ground truth, RDKit/CXSMILES canonicalization as chemical equality, and engineering choices (backbone, LoRA rank, data mixture weights, decoding) rather than new physical entities or fitted scientific constants.

free parameters (4)
  • Stereo / adjacent-ring oversampling factors (Stereo-200K ×2, adjacent-ring subset ×2) = ×2 on stereo-rich subsets; full effective ~2.21M image-SMILES records
    Curriculum multiplicities chosen after residual stereo-error analysis; ablation shows they move USPTO canonical match from ~90% to 93.8%.
  • MarkushGlyph ordinary-molecule mixture share = 5% (13,158 / 263,158)
    Hand-chosen fraction of OCSR-as-empty-Markush records in the 263k training pool; swept 0–15% in ablations, final model uses 5%.
  • LoRA rank and scale = rank 128, scale 128
    Adapter capacity hyperparameters for Qwen3.5-2B SFT; not derived, selected for the fine-tune.
  • Majority-vote decoding (K, temperature, nucleus) = K=8, T=0.7, p=0.95
    Inference-time hyperparameters that improve reported Markush numbers versus greedy.
assumptions (6)
  • domain assumption RDKit canonical isomeric SMILES string equality (after stated stereo stripping) is a valid OCSR correctness criterion on USPTO.
    §4.4 and App. A.1; matches community practice but depends on toolkit canonicalization and ignores the 8 unparseable refs marked wrong.
  • domain assumption cxsmiles_opt plus canonicalization preserves Markush semantics (R labels, m:/Sg: sections, fragments) across atom orderings.
    §3.1, App. A.2; inherited from MarkushGrapher-2 optimized CXSMILES design.
  • ad hoc to paper Strict equality via feature-constrained VF2 isomorphism on annotated molecular graphs is the right notion of full Markush correctness.
    §4.4, App. A.3–C; authors introduce and validate this predicate to fix legacy scorer holes.
  • ad hoc to paper Neither Markush metric needs to score free-text substituent definitions for the reported accuracy claims.
    Explicit in §4.4: substituent table text is emitted but not checked—material for M2S where text is in-image.
  • domain assumption Early-fusion ViT+LLM reading pixels alone can recover printed labels and constraints without a separate OCR/layout stage.
    §3.3 / Fig. 2 architectural premise contrasting MarkushGrapher-2 staged pipelines.
  • standard math Standard supervised fine-tuning / next-token likelihood on the curated mixtures yields models that generalize to the held-out patent benchmarks.
    Implicit throughout §3–5; usual i.i.d. ML generalization assumption.
invented entities (3)
  • OCSRGlyph independent evidence
    purpose: Specialized ~94M Swin-B encoder + 6-layer decoder image-to-SMILES model for single-molecule OCSR.
    Named system contribution; standard architecture with stereo-focused data curriculum rather than a new physical object.
  • MarkushGlyph independent evidence
    purpose: LoRA-tuned Qwen3.5-2B VLM emitting unified cxsmiles_opt records for Markush and ordinary molecules.
    Named system contribution; evaluates on public benchmarks with released weights.
  • strict parsed-graph equality (Markush metric) independent evidence
    purpose: Stricter CXSMILES correctness test requiring one atom bijection preserving graph, R-groups, m:/Sg:, and fragments.
    New evaluation construct introduced to penalize extra/transposed annotations the legacy score misses; validated in App. C against VF2/RDKit.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MarkushGlyph and OCSRGlyph: Improved Chemical Structure Recognition." pith.science (2026). https://pith.science/paper/4ST3ZSDC

@misc{pith2026260728532,
  author       = {Pith},
  title        = {Pith review of: MarkushGlyph and OCSRGlyph: Improved Chemical Structure Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4ST3ZSDC}},
  note         = {Machine review of arXiv:2607.28532}
}
read the original abstract

Chemical structures appear in patents and the scientific literature as images. For programmatic usage, such as indexing in databases or constructing machine learning model training sets, they must be transformed into line notations. The two common forms of this task are translating an image of a single molecule (optical chemical structure recognition - OCSR) and translating a Markush structure that represents a family of molecules. While prior work in the former case is quite mature, Markush structure parsing remains a challenging task. In this work, we treat both tasks as an image-to-text translation problem. We then propose OCSRGlyph, a state-of-the-art OCSR model, improving performance over prior methods by carefully considering stereochemistry. For the Markush task, we introduce MarkushGlyph, a vision-language model that reads the entire Markush structure as an image. This contrasts with prior systems, which often use multiple stages to separately process visual and text input content. Finally, we introduce a new metric for determining the accuracy of Markush structure translations, handling failure modes present in prior metrics.

Figures

Figures reproduced from arXiv: 2607.28532 by the authors.

Figure 1
Figure 1. From a single molecule to a family of related compounds. Ordinary optical chemical structure recognition converts an image into a text representation of one molecule. Markush structure translation instead outputs a more general representation that encodes a common scaffold together with variable sites and constraints that describe many related compounds. Both tasks require reading the same atoms and bonds from the d… view at source ↗
Figure 2
Figure 2. Architecture of MarkushGlyph. A cropped image and a short prompt are passed to Qwen3.5-2B-Base, which emits the structured cxsmiles_opt record directly after a LoRA supervised fine-tuning stage. An ordinary molecule is the special case whose substituent extension is empty [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Markush recognition on the benchmarks under the MarkushGrapher-2 score. Each group is one benchmark; bars are prior systems (gray) and MarkushGlyph (teal) in two decoding settings. MarkushGlyph is above every prior system on all three benchmarks. Exact values and the strict parsed-graph results are in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Markush qualitative comparison. Five illustrative examples from the Markush benchmarks. Checks and crosses denote pass and fail under the MarkushGrapher-2 scoring function. In the first four examples MarkushGlyph passes and MarkushGrapher-2 fails: from left to right, M…
Figure 5
Figure 5. Figure 5: (a) A capability landscape. Markush ability (horizontal) is the MarkushGrapher-2 combined cxsmi_equality; OCSR ability (vertical) is graph exact match on the USPTO images, one benchmark for every system. As the ordinary-molecule mixture grows from 0 to 15% (teal path),…
Figure 6
Figure 6. Figure 6: A comparison of the MarkushGrapher-2 and strict parsed-graph equality scoring functions. (a) An incorrect model prediction that passes the former scorer but fails ours. The prediction transposes the R1/R2 labels and adds a repeat-unit bracket (Sg:n) over the variable m…
Figure 7
Figure 7. Figure 7: The 344 canonical misses on the USPTO-5704 split into 196 wrong-connectivity, 136 stereo-only, and 12 invalid￾SMILES examples. Wrong-connectivity is the largest residual class, and stereochemistry-focused curation does not address it; the stereo-only examples are the s…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Real Data Closes Synthetic-to-Real Gap in Optical Chemical Structure Recognition

    cs.LG 2026-08 conditional novelty 6.0 of 10

    Fine-tuning optical structure recognizers with labeled real depictions, even at 5 to 10 percent of the training mixture, is the most effective intervention for closing the synthetic-to-real gap.

Reference graph

Works this paper leans on

13 extracted references · 6 canonical work pages · cited by 1 Pith paper

  1. [3]

    doi:10.1093/nar/gkaa971. A. Krasnov, S. J. Barnabas, T. Boehme, S. K. Boyer, and L. Weber. Comparing software tools for optical chemical structure recognition.Digital Discovery, 3:681–693,

    work page doi:10.1093/nar/gkaa971

  2. [4]

    doi:10.1039/D3DD00228D. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo. Swin transformer: Hierarchical vision transformer using shifted windows. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10012–10022,

    work page doi:10.1039/d3dd00228d

  3. [9]

    doi:10.1186/s13321-021-00538-8. K. Rajan, H. O. Brinkhaus, M. I. Agea, A. Zielesny, and C. Steinbeck. DECIMER.ai: an open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications.Nature Communications, 14(1):5045,

    work page doi:10.1186/s13321-021-00538-8

  4. [10]

    doi:10.1038/s41467-023-40782-0. RDKit. RDKit: Open-source cheminformatics.https://www.rdkit.org,

    doi:10.1038/s41467-023-40782-0

  5. [11]

    doi:10.1117/12.912185. E. S. Simmons. Markush structure searching over the years.World Patent Information, 25(3):195–202,

    work page doi:10.1117/12.912185

  6. [2003]

    doi: 10.1016/S0172-2190(03)00073-5. T. Strohmeyer, L. Morin, G. I. Meijer, V. Weber, A. Nassar, and P. Staar. MarkushGrapher-2: End-to-end multimodal recognition of chemical structures,

    work page doi:10.1016/s0172-2190(03)00073-5

  7. [2010]

    doi:10.1145/1815330.1815389. OpenAI. GPT-5.6 system card.https://deploymentsafety.openai.com/gpt-5-6, July

    arXiv

  8. [2012]

    Y. Qian, J. Guo, Z. Tu, Z. Li, C. W. Coley, and R. Barzilay. MolScribe: Robust molecular structure recognition with image-to-graph generation.Journal of Chemical Information and Modeling, 63(7):1925–1934,

    1925

Show all 13 references
  1. [2016]

    Piroi, M

    F. Piroi, M. Lupu, A. Hanbury, W. Magdy, A. P. Sexton, and I. V. Filippov. CLEF-IP 2012: Retrieval experiments in the intellectual property domain. InCLEF 2012 Evaluation Labs and Workshop, Online Working Notes, volume 1178 ofCEUR Workshop Proceedings. CEUR-WS.org,

    2012

  2. [2021]

    S. Kim, J. Chen, T. Cheng, A. Gindulyte, J. He, S. He, Q. Li, B. A. Shoemaker, P. A. Thiessen, B. Yu, L. Zaslavsky, J. Zhang, and E. E. Bolton. PubChem in 2021: New data content and improved web interfaces.Nucleic Acids Research, 49(D1):D1388–D1395,

    2021

  3. [2023]

    17 Qwen Team

    doi: 10.1021/acs.jcim.2c01480. 17 Qwen Team. Qwen3 technical report,

    doi

  4. [2024]

    doi:10.1186/s13321-024-00926-w. L. P. Cordella, P. Foggia, C. Sansone, and M. Vento. An improved algorithm for matching large graphs. InProceedings of the 3rd IAPR-TC15 Workshop on Graph-Based Representations in Pattern Recognition, pages 149–159,

    doi

  5. [2026]

    arXiv:2603.28550. H. Tang, X. Dang, and J. Wang. Fine-tuning DeepSeek-OCR-2 for molecular structure recognition,

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.