Pith. sign in

REVIEW 4 major objections 4 minor 22 references

PPanGGOLiN v2 adds four new pangenome analyses and a leaner HDF5 file

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:00 UTC pith:QWN2GXAK

load-bearing objection A credible software-release paper with real new features, undercut by an underspecified performance benchmark and minimal validation of the new analytics; deserves review but needs a reproducible evidence section. the 4 major comments →

arxiv 2607.24111 v1 pith:QWN2GXAK submitted 2026-07-27 q-bio.QM q-bio.GN

PPanGGOLiN V2: technical enhancement and extended functionalities for prokaryotic pangenome analysis

J{\'e}r{\^o}me Arnoux (LABGeM) , Jean Mainguy (LABGeM) , Adelme Bazin (LBMC UMR 5239) , Guillaume Gautreau (MaIAGE) , T{\'e}o Lemane (LABGeM) , David Vallenet (LABGeM) , A. Calteau (LABGeM) This is my paper
classification q-bio.QM q-bio.GN
keywords pangenomeprokaryotic comparative genomicsgene familiesgraph-based methodgenomic plasticityHDF5 storagesoftware releasepangenome projection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

PPanGGOLiN is a graph-based tool for building and exploring prokaryotic pangenomes — the full gene content of a species or clade rather than a single reference genome. This paper presents the second major release, and its central claim is that version 2 both extends and streamlines the tool: it adds projection of external genomes onto an existing pangenome, extraction of conserved genomic neighborhoods, clustering of regions of genomic plasticity (RGPs, i.e., variable genomic islands), and user-defined metadata on any pangenome element, while a redesign of the HDF5 storage format makes pangenome files substantially smaller and faster to load. The authors argue these improvements are necessary because genomic datasets are growing exponentially and pangenome analyses must keep pace. If the claims hold, v2 lets researchers compare increasingly large genome collections more cheaply and tie pangenome variation to ecological, clinical, or experimental data.

Core claim

On the paper's own terms, the central claim is that PPanGGOLiN v2 is a working system, not just a proposal: a precomputed pangenome graph — gene families as nodes, genomic adjacency as edges, statistically partitioned into persistent, shell, and cloud components — becomes a reference object that supports four new commands without recomputation. 'Projection' assigns genes of an external genome to existing families and then predicts RGPs, insertion spots, and modules. 'Context' extracts the gene families around a target family within a user-defined window via transitive closure. 'Rgp_cluster' groups RGPs by shared gene content using a gene-repertoire relatedness (GRR) score and connected compo

What carries the argument

The central object is the partitioned pangenome graph. Nodes are gene families; edges encode genomic adjacency. A Bernoulli mixture model coupled with a Markov random field partitions the families into persistent, shell, and cloud genomes, using both how often a family appears and where it sits in the genomic neighborhood. All four new features operate on this graph: projection matches external genes to families; context extraction walks a window of neighbors using transitive closure; RGP clustering builds a second graph of RGPs connected by a gene-repertoire relatedness score; and metadata attaches annotations to graph elements. The graph, partitions, and metadata are stored in a single HDF

Load-bearing premise

The performance claim — smaller files and faster loading — rests entirely on Figure 2, which compares v1 and v2 file sizes across genome counts without disclosing which genomes, what parameters, how many replicates, or whether the two versions used identical inputs, so the comparison is not independently verifiable.

What would settle it

Run PPanGGOLiN v1.2.74 and v2.3.0 on the same set of prokaryotic genomes, say 1,000 to 5,000 strains, with identical parameters and inputs, and compare the resulting HDF5 file sizes and loading times; if v2 is not consistently smaller and faster, the paper's headline performance claim is false. The same controlled comparison can also test whether projection assigns genes to the same families and partitions as a full pangenome reconstruction.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Newly sequenced genomes can be annotated against an existing pangenome without rebuilding it, making routine surveillance of new isolates practical.
  • RGPs from many genomes can be clustered by shared gene content, turning the study of mobile genetic element spread into a network analysis.
  • Conserved genomic neighborhoods around genes of interest can be extracted, enabling reconstruction of metabolic pathways or functional systems at species level.
  • Storing metadata in the pangenome file and propagating it to all outputs lets users connect pangenome variation to ecological, clinical, or experimental annotations.
  • Smaller files and faster loading lower the cost of working with pangenomes of thousands of genomes, both interactively and in automated pipelines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper does not report accuracy benchmarks for projection against full recomputation; a natural test is to project a set of genomes onto a reference pangenome and compare the resulting RGP and module calls with those obtained by building a new pangenome from the combined set.
  • The metadata feature implies a route toward pangenome-wide association studies that link mobile-element presence to phenotypes, but the paper leaves the statistical design of such studies unexplored.
  • If the file-size reduction generalizes across taxa, maintenance of species-level pangenome databases becomes more tractable, and incremental updating via projection could become the standard workflow; the paper does not discuss how assignment errors accumulate over successive projections.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper announces PPanGGOLiN v2, an update to an existing graph-based prokaryotic pangenome tool. The authors claim substantial improvements in three dimensions: new analytical features (pangenome projection, genomic context extraction, RGP clustering, and metadata association), a comprehensive software architecture redesign, and performance improvements (smaller HDF5 pangenome files and faster loading). The manuscript describes the new workflow and includes a figure comparing HDF5 file sizes between v1.2.74 and v2.3.0 across increasing genome counts. It does not provide quantitative validation of the new features, nor does it supply benchmark protocols, raw data, or scripts for the performance comparison. The central claim therefore rests on the existence of the software and textual descriptions rather than on evaluative evidence.

Significance. If the performance and feature claims were properly substantiated, the work would be useful to the microbial genomics community: PPanGGOLiN is already widely used, the central HDF5 object design is attractive for reproducibility, and the new features (projection, RGP clustering, context extraction, metadata) fill evident needs in pangenome analysis. Strengths include open distribution via Bioconda/PyPI, integration with MicroScope and Galaxy, and a documentation overhaul. However, the paper's impact is limited by the absence of accuracy benchmarks for the new features, the lack of sensitivity analysis for the GRR threshold, the absence of comparisons with other pangenome tools, and the unverifiable single-figure performance comparison. These omissions bear directly on the headline 'substantial improvements' claim.

major comments (4)
  1. [§Technical enhancements / Figure 2] The performance pillar is not independently assessable. Figure 2 does not state which genomes were used, how replicates were handled, how parameters were set, whether the inputs were identical for both versions, or the computational environment. The associated claim that loading time for spots dropped 'from over several minutes to few seconds' has no supporting measurement. Because smaller/faster files are one of the three advertised improvements, the authors must provide a detailed matched protocol, error bars or replicate points, and the benchmark scripts/data.
  2. [§New features / Projection] The projection feature is described only functionally: genes are assigned to families by sequence similarity, and unmatched genes are placed in the cloud partition. There is no benchmark against a full pangenome recomputation, no precision/recall or accuracy measure for family assignment, and no evaluation of the predicted RGPs, spots, or modules on a known test case. Without such evaluation, the claim that projection is useful for 'comparing newly sequenced genomes against an established reference pangenome' is unsupported, although the feature may exist.
  3. [§RGP clustering] The GRR threshold (0.8 by default) is presented without a definition of the GRR score formula and without any sensitivity analysis or biological validation. It is not clear whether the min or max variant is used or how the threshold was chosen. The clustering quality is not compared against known mobile genetic element families or against alternative clustering thresholds. Since this is a new analytical feature, the authors should show that the default threshold produces robust and meaningful clusters.
  4. [§Genomic context extraction] This section is essentially a pointer to a separate PANORAMA paper; the method itself is not described sufficiently for a reader to evaluate correctness or limitations. Key details—such as how the 'defined genomic window' is chosen, how transitive closure handles large neighborhoods, and how the output relates to the original graph—are absent. The paper needs at least a precise algorithmic description or a focused evaluation, rather than deferring entirely to another manuscript.
minor comments (4)
  1. [Figure 2 caption] Please state the dataset, number of replicates, error bars, and parameter files. Also clarify whether the plotted values are means/medians and which exact versions of v1 and v2 were used beyond the version numbers.
  2. [§Technical enhancements] The phrase 'especially for spots' is ambiguous: does this refer to PPanGGOLiN 'spots' (insertion spots) or to spot-loading in the programmatic API? If it is a technical term, define it or rephrase for general readers.
  3. [§RGP clustering] The sentence 'an edge is added between two nodes if their GRR (min or max) exceeds a defined threshold' is unclear. Define the GRR formula and state whether the min, max, or another aggregation is used in the default workflow.
  4. [Availability] The text states distribution via Bioconda and PyPI but gives no repository URL, version identifier, or documentation link. Include an explicit 'Availability' section with the code repository, version numbers, and benchmark scripts/data deposition.

Circularity Check

0 steps flagged

No significant circularity: the paper is a software/application note whose central claims are existence claims, not derived predictions fitted to their own inputs.

full rationale

This manuscript is a software description rather than a scientific derivation. The central claim is that PPanGGOLiN v2 exists, contains new commands (projection, RGP clustering, context extraction, metadata association), has a redesigned architecture, and produces smaller HDF5 files. None of these are predictions derived from first principles with fitted parameters; they are implemented features and benchmarked outcomes. The only quantitative performance evidence, Figure 2, plots file sizes for v1.2.74 versus v2.3.0 with increasing genome counts, but no equations or fitted parameters are involved. The underspecified benchmark protocol is a reproducibility/evidence-quality concern, not a circularity, because the comparison is not defined in terms of the result it purportedly demonstrates. The statistical partitioning, PanRGP, and PanModule methods are inherited from earlier independently published work (Gautreau et al., 2020; Bazin et al., 2020, 2021), not re-derived here. Self-citations appear (e.g., the context extraction method is referred to the PANORAMA paper and the ecosystem statements cite MicroScope and Galaxy), but these are descriptive pointers rather than load-bearing arguments that reduce a claimed result to a self-citation. A hand-chosen default GRR threshold of 0.8 is a parameter choice, not a fitted input called a prediction. Accordingly, no circular step can be exhibited, and the honest finding is no significant circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The ledger shows the paper's load-bearing dependencies: the v1 statistical model is inherited without re-validation; projection accuracy is assumed to follow from sequence-similarity family assignment; the Figure 2 comparison is assumed to be matched and representative. One hand-chosen parameter (GRR threshold 0.8) governs the new RGP clustering with no sensitivity or justification given. No new biological entities are postulated.

free parameters (2)
  • RGP clustering GRR threshold = 0.8 (default)
    Hand-chosen default in the RGP clustering feature: an edge is added between two RGP nodes if GRR (min or max) exceeds 0.8. No sensitivity analysis or biological justification is provided.
  • Genomic context window size = not stated
    Context extraction pulls gene families "within a defined genomic window around the target"; the window size is never specified in this paper and the method is deferred to the PANORAMA paper.
axioms (3)
  • domain assumption The v1 statistical model (Bernoulli Mixture Model + Markov Random Field on the pangenome graph) partitions gene families into persistent/shell/cloud correctly and remains valid in v2.
    Every new feature builds on this partition; the paper cites Gautreau et al. 2020 but provides no re-validation for v2.
  • domain assumption Gene-to-family assignment by sequence similarity is sufficient for accurate projection of external genomes.
    Projection section: genes are assigned "based on sequence similarity"; no accuracy benchmark against de novo pangenome construction is provided.
  • domain assumption The v1-v2 comparison in Figure 2 used matched inputs and comparable parameters, so the size reduction is attributable to the redesign.
    No dataset, parameters, or protocol are given; the entire file-size claim depends on this being an apples-to-apples comparison.

pith-pipeline@v1.3.0-alltime-deepseek · 6258 in / 15856 out tokens · 124502 ms · 2026-07-31T23:00:55.868262+00:00 · methodology

0 comments
read the original abstract

The exponential growth of genomic data, particularly for microbes, has made pangenomic approaches a gold standard for large-scale comparative genomics. By capturing the full genomic diversity of a species rather than relying on a single reference, pangenomics has transformed microbial genomics, revealing the adaptive potential of bacteria and the evolutionary dynamics underlying functional diversity. Among available tools, PPanGGOLiN distinguishes itself through its graph-based model coupled with statistical gene partitioning. Here we present PPanGGOLiN v2, which introduces substantial improvements across three dimensions: new analytical features that expand what users can investigate, a comprehensive software architecture redesign that improves maintainability and extensibility, and performance improvements that address the computational demands of ever-growing genomic datasets.

Figures

Figures reproduced from arXiv: 2607.24111 by A. Calteau (LABGeM), Adelme Bazin (LBMC UMR 5239), David Vallenet (LABGeM), Guillaume Gautreau (MaIAGE), Jean Mainguy (LABGeM), J{\'e}r{\^o}me Arnoux (LABGeM), T{\'e}o Lemane (LABGeM).

Figure 1
Figure 1. Figure 1: Overview of the PPanGGOLiN v2 workflow. Each rounded box represents a command of the software. Commands marked with * are new in PPanGGOLiN v2. Commands are grouped into two sections. (i) Pangenome construction: starting from genomic data (with annotations or not), the annotate, cluster, graph, and partition commands progressively build the pangenome graph and partition gene families into persistent, shell… view at source ↗
Figure 2
Figure 2. Figure 2: Pangenome HDF5 file size (GB) generated by PPanGGOLiN v1.2.74 vs v2.3.0 across increasing genome counts. PPanGGOLiN v2 consistently produces smaller files, supporting the impact of the v2 file structure redesign. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 17 canonical work pages

  1. [1]

    Arnoux, J., Bonifati, A., Calteau, A., Dumbrava, S., & Gautreau, G. (2024). Integrating complex pangenome graphs. 2024 IEEE 40th International Conference on Data Engineering Workshops ( ICDEW ) , 350--354. https://doi.org/10.1109/ICDEW61823.2024.00052

    arXiv 2024

  2. [2]

    and Calteau, A

    Arnoux, J., Mainguy, J., Bry, L., Fernandez de Grado, Q., Hoblos, Y., Vallenet, D. and Calteau, A. (2026). Panorama : A robust pangenome-based method for predicting and comparing biological systems across species PLOS Computational Biology, https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1013856

    work page doi:10.1371/journal.pcbi.1013856 2026

  3. [3]

    Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: An open source software for exploring and manipulating networks. Proceedings of the International AAAI Conference on Web and Social Media , 3(1), 361--362. https://doi.org/10.1609/icwsm.v3i1.13937

    doi:10.1609/icwsm.v3i1.13937 2009

  4. [4]

    Bazin, A., Gautreau, G., Médigue, C., Vallenet, D., & Calteau, A. (2020). panRGP : A pangenome-based method to predict genomic islands and explore their diversity. Bioinformatics, 36, i651--i658. https://doi.org/10.1093/bioinformatics/btaa792

    work page doi:10.1093/bioinformatics/btaa792 2020

  5. [5]

    Bazin, A., Medigue, C., Vallenet, D., & Calteau, A. (2021). panModule : Detecting conserved modules in the variable regions of a pangenome graph . bioRxiv. https://doi.org/10.1101/2021.12.06.471380

    work page doi:10.1101/2021.12.06.471380 2021

  6. [6]

    Buck, M., Mehrshad, M., & Bertilsson, S. (2022). mOTUpan : A robust bayesian approach to leverage metagenome-assembled genomes for core-genome estimation. NAR Genomics and Bioinformatics , 4(3), lqac060. https://doi.org/10.1093/nargab/lqac060

    work page doi:10.1093/nargab/lqac060 2022

  7. [7]

    M., Hickey, G., Eizenga, J

    Garrison, E., Sirén, J., Novak, A. M., Hickey, G., Eizenga, J. M., Dawson, E. T., Jones, W., Garg, S., Markello, C., Lin, M. F., Paten, B., & Durbin, R. (2018). Variation graph toolkit improves read mapping by representing genetic variation in the reference. Nature Biotechnology, 36(9), 875--879. https://doi.org/10.1038/nbt.4227

    work page doi:10.1038/nbt.4227 2018

  8. [8]

    Gautreau, G., Bazin, A., Gachet, M., Planel, R., Burlot, L., Dubois, M., Perrin, A., Médigue, C., Calteau, A., Cruveiller, S., Matias, C., Ambroise, C., Rocha, E. P. C., & Vallenet, D. (2020). PPanGGOLiN : Depicting microbial diversity via a partitioned pangenome graph. PLOS Computational Biology , 16(3), e1007732. https://doi.org/10.1371/journal.pcbi.1007732

    work page doi:10.1371/journal.pcbi.1007732 2020

  9. [9]

    R., Enns, E., Marinier, E., Mandal, A., Herman, E

    Grant, J. R., Enns, E., Marinier, E., Mandal, A., Herman, E. K., Chen, C., Graham, M., Van Domselaar, G., & Stothard, P. (2023). Proksee: In-depth characterization and visualization of bacterial genomes. Nucleic Acids Research, 51, W484--W492. https://doi.org/10.1093/nar/gkad326

    work page doi:10.1093/nar/gkad326 2023

  10. [10]

    A., Rowe, J., Tomkins-Tinch, C

    Grüning, B., Dale, R., Sjödin, A., Chapman, B. A., Rowe, J., Tomkins-Tinch, C. H., Valieris, R., & Köster, J. (2018). Bioconda: Sustainable and comprehensive software distribution for the life sciences. Nature Methods, 15(7), 475--476. https://doi.org/10.1038/s41592-018-0046-7

    work page doi:10.1038/s41592-018-0046-7 2018

  11. [11]

    Holley, G., & Melsted, P. (2020). Bifrost: Highly parallel construction and indexing of colored and compacted de bruijn graphs. Genome Biology, 21(1), 249. https://doi.org/10.1186/s13059-020-02135-8

    doi:10.1186/s13059-020-02135-8 2020

  12. [12]

    F., Land, M

    Hyatt, D., Chen, G.-L., LoCascio, P. F., Land, M. L., Larimer, F. W., & Hauser, L. J. (2010). Prodigal: Prokaryotic gene recognition and translation initiation site identification. BMC Bioinformatics , 11(1), 119. https://doi.org/10.1186/1471-2105-11-119

    work page doi:10.1186/1471-2105-11-119 2010

  13. [13]

    R., Ahn, T.-H., Karpinets, T., Lund, O., Kora, G., Wassenaar, T., Poudel, S., & Ussery, D

    Land, M., Hauser, L., Jun, S.-R., Nookaew, I., Leuze, M. R., Ahn, T.-H., Karpinets, T., Lund, O., Kora, G., Wassenaar, T., Poudel, S., & Ussery, D. W. (2015). Insights from 20 years of bacterial genome sequencing. Functional & Integrative Genomics, 15(2), 141--161. https://doi.org/10.1007/s10142-015-0433-4

    work page doi:10.1007/s10142-015-0433-4 2015

  14. [14]

    Larralde, M. (2022). Pyrodigal: Python bindings and interface to prodigal, an efficient method for gene prediction in prokaryotes. Journal of Open Source Software, 7(72), 4296. https://doi.org/10.21105/joss.04296

    work page doi:10.21105/joss.04296 2022

  15. [15]

    Li, H., Marin, M., & Farhat, M. R. (2024). Exploring gene content with pangene graphs. Bioinformatics, 40(7), btae456. https://doi.org/10.1093/bioinformatics/btae456

    work page doi:10.1093/bioinformatics/btae456 2024

  16. [16]

    H., Chuvochina, M., Waite, D

    Parks, D. H., Chuvochina, M., Waite, D. W., Rinke, C., Skarshewski, A., Chaumeil, P.-A., & Hugenholtz, P. (2018). A standardized bacterial taxonomy based on genome phylogeny substantially revises the tree of life. Nature Biotechnology, 36(10), 996--1004. https://doi.org/10.1038/nbt.4229

    work page doi:10.1038/nbt.4229 2018

  17. [17]

    S., Wang, J

    Shannon, P., Markiel, A., Ozier, O., Baliga, N. S., Wang, J. T., Ramage, D., Amin, N., Schwikowski, B., & Ideker, T. (2003). Cytoscape: A software environment for integrated models of biomolecular interaction networks. Genome Research, 13(11), 2498--2504. https://doi.org/10.1101/gr.1239303

    doi:10.1101/gr.1239303 2003

  18. [18]

    Snipen, L., & Liland, K. H. (2015). Micropan: An r-package for microbial pan-genomics. BMC Bioinformatics , 16(1), 79. https://doi.org/10.1186/s12859-015-0517-0

    work page doi:10.1186/s12859-015-0517-0 2015

  19. [19]

    pan-genome

    Tettelin, H., Masignani, V., Cieslewicz, M. J., Donati, C., Medini, D., Ward, N. L., Angiuoli, S. V., Crabtree, J., Jones, A. L., Durkin, A. S., Deboy, R. T., Davidsen, T. M., Mora, M., Scarselli, M., Margarit y Ros, I., Peterson, J. D., Hauser, C. R., Sundaram, J. P., Nelson, W. C., Fraser, C. M. (2005). Genome analysis of multiple pathogenic isolates of...

    2005

  20. [20]

    The Galaxy Community. (2024). The galaxy platform for accessible, reproducible, and collaborative data analyses: 2024 update. Nucleic Acids Research, 52, W83--W94. https://doi.org/10.1093/nar/gkae410

    work page doi:10.1093/nar/gkae410 2024

  21. [21]

    A., Gladstone, R

    Tonkin-Hill, G., MacAlasdair, N., Ruis, C., Weimann, A., Horesh, G., Lees, J. A., Gladstone, R. A., Lo, S., Beaudoin, C., Floto, R. A., Frost, S. D. W., Corander, J., Bentley, S. D., & Parkhill, J. (2020). Producing polished prokaryotic pangenomes with the panaroo pipeline. Genome Biology, 21(1), 180. https://doi.org/10.1186/s13059-020-02090-4

    work page doi:10.1186/s13059-020-02090-4 2020

  22. [22]

    Vallenet, D., Calteau, A., Dubois, M., Amours, P., Bazin, A., Beuvin, M., Burlot, L., Bussell, X., Fouteau, S., Gautreau, G., Lajus, A., Langlois, J., Planel, R., Roche, D., Rollin, J., Rouy, Z., Sabatet, V., & Médigue, C. (2020). MicroScope : An integrated platform for the annotation and exploration of microbial gene functions through genomic, pangenomic...

    work page doi:10.1093/nar/gkz926 2020