{"id":"0e630367-a691-41b2-aa05-63733d6b4301","arxiv_id":"2501.01073","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"G2PT represents graphs as node-then-edge token sequences and learns them with GPT-style next-token prediction, matching or beating diffusion baselines on seven graph and molecule datasets.","lead":"This paper introduces G2PT, a Transformer that generates graphs by writing them as a sequence of node tokens followed by edge tokens, like a language model writing text. The approach matches or beats specialized diffusion models on molecule and graph benchmarks, and it can be fine-tuned for designing molecules with target properties.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Degree-based edge ordering is the load-bearing serialization; no evidence it is canonical or transferable, so the 'general graph encoding' claim is not established.","rationale":"The reader's weakest assumption and my load-bearing concern coincide: the reverse degree-based edge-removal ordering in Algorithm 1 is what makes the sequence representation learnable, and Table 9 plus the Conclusion show that ordering choice is decisive. I agree that this threatens the 'general-purpose foundation model' framing, because the representation is then a dataset-specific serialization rather than a canonical graph encoding. I would not move the verdict away from CONDITIONAL: the per-dataset generative numbers may still be valid, the lower-bound argument is a useful contribution, and the order-sensitivity is honestly reported. The proposed experiment would settle whether a single ordering can work across domains or whether the claim must be weakened. The reader's additional requests for error bars, quantitative goal-oriented baselines, and released code remain sensible and orthogonal; they would strengthen the empirical case but do not change the core concern I have identified.","tokens_in":24277,"tokens_out":9637,"duration_ms":103072,"concrete_test":"Train G2PTsmall and G2PTbase on MOSES, GuacaMol, and Planar with three fixed orderings—Algorithm 1, BFS, and a canonical RDKit atom/bond order—holding model size, training budget, and evaluation pipeline fixed, with at least five random seeds per condition. If one ordering is consistently within a small tolerance of the best on every dataset, order sensitivity is not a fundamental obstacle. If, as Table 9 hints, the best ordering differs by dataset and the gaps are large (e.g., FCD differing by more than 1.0), then no canonical ordering exists and the paper should be reframed as per-dataset serialization tuning rather than a general graph pretraining representation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1's degree-based edge-removal order is not an implementation detail: it is the only bridge between a graph and the token sequence the Transformer learns. Table 9 demonstrates the bridge is fragile: on MOSES, G2PTsmall's validity drops from 95.1 to 62.9 and FCD from 1.10 to 7.0 under uniform edge ordering, and the Conclusion concedes 'G2PT is order-sensitive, where different graph domains may prefer different edge orderings.' The lower-bound derivation in §3.2 shows only that maximizing sequence likelihood tightens a bound for a chosen sequence distribution pd(s|G); it does not show that pd(s|G) is canonical, nor that a single ordering transfers across Planar/Tree/SBM and molecular datasets. So the abstract's 'superior generative performance' is a statement about a dataset-specific serialization, and the §1 claim of a 'general-purpose foundation model' based on edge-list tokenization is not established by the reported experiments. This is the most load-bearing weak point because if no universal or canonical ordering exists, the representation is a per-dataset choice rather than a general graph encoding.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"G2PT represents a graph as a token sequence that lists all node tokens, a delimiter token, and then all edge tokens, with the edge order defined by the reverse of a degree-based edge-removal process (Algorithm 1). A Transformer decoder is trained with a standard next-token prediction objective on this serialization, and the paper shows that maximizing sequence likelihood maximizes a lower bound on graph likelihood. The same pre-trained model is fine-tuned for goal-oriented molecular generation via rejection-sampling fine-tuning and PPO, and for graph property prediction by attaching a linear classifier to the last-token representation. Experiments cover four generic graph datasets (Planar, Tree, Lobster, SBM) and three molecular datasets (QM9, MOSES, GuacaMol), plus eight MoleculeNet classification benchmarks, with scaling and edge-ordering sensitivity analyses.","tokens_in":24496,"tokens_out":9524,"duration_ms":85131,"significance":"The work addresses a timely question: whether a plain GPT-style Transformer can serve as a graph generator through an edge-list serialization. The lower-bound derivation in Section 3.2 is sound, the empirical protocol broadly follows established benchmarks, and the paper includes useful analyses of model/data scaling and edge-ordering sensitivity, as well as an ablation showing the benefit of pre-training for property prediction. The code release supports reproducibility. If the claims are confirmed with appropriate uncertainty quantification and with the ordering-sensitivity issue addressed, the edge-token serialization would be a valuable alternative to adjacency-matrix diffusion and autoregressive baselines, with clear computational advantages on sparse graphs.","major_comments":[{"comment":"The degree-based edge-removal order in Algorithm 1 is the only bridge between a graph and the token sequence learned by the model, and the paper's own sensitivity analysis shows this bridge is fragile: on MOSES, replacing it with uniform ordering drops G2PTsmall validity from 95.1 to 62.9 and FCD from 1.10 to 7.0 (Table 9), and the Conclusion states that 'G2PT is order-sensitive, where different graph domains may prefer different edge orderings.' No experiment demonstrates that a single ordering transfers across the generic graph and molecular domains used to support the 'general-purpose foundation model' claim in Section 1. I ask the authors to add cross-domain ordering-transfer evidence (for example, training on one domain and testing ordering choices on another, or comparing the chosen ordering across all four generic and three molecular datasets) or to explicitly reframe the contribution as a dataset-specific encoding rather than a general graph encoding.","section":"Algorithm 1, Table 9, Section 6"},{"comment":"The abstract and Section 5.3 claim 'superior generative performance' on generic graphs, but Table 2 shows G2PTbase is best on 11 of 24 metrics and top-two on 17 of 24, and on Planar degree G2PTsmall (4.7e-3) is an order of magnitude worse than DeFoG (5e-4). Because no standard deviations or repeated-seed results are reported for any MMD metric, differences among near-zero entries (e.g., Orbit 0.00 for several methods) cannot be judged statistically. Please report uncertainty estimates and either soften the superiority claim to 'competitive' or identify the specific metrics on which superiority is claimed.","section":"Section 5.3, Table 2"},{"comment":"The goal-oriented generation results are presented only as density plots, with no quantitative metrics (e.g., fraction of accepted samples, mean property values, validity, uniqueness, or reward) and no baseline method for comparison beyond visual inspection of the pre-trained distribution. The text makes strong statements such as 'RFT overcomes the barrier in the second round' and 'PPO fails to sample data with high rewards' without numerical support. Please add a quantitative evaluation of the fine-tuning procedures, including the validity and diversity of the optimized samples, so that the fine-tuning contribution can be assessed.","section":"Section 5.5, Figure 2"},{"comment":"The molecular generation comparison is incomplete in two respects. First, the QM9 table reports FCD for G2PT but leaves DiGress's FCD as '-' even though DiGress is a primary baseline, so the headline FCD advantage on QM9 cannot be compared against it. Second, the GuacaMol column headers use 'KL Div.↑' and 'FCD↑', which conflicts with the standard GuacaMol evaluation where these quantities are lower-is-better; if the arrows are correct, the definitions should be clarified. Please reconcile the protocol with the cited toolkits and state the validity definition used for each baseline (Section B.6 mentions that the RDKit validity definition can differ from the lenient definition used in some prior work).","section":"Section 5.4, Table 4, Section B.6"}],"minor_comments":[{"comment":"The sentence 'Although pθ(s|G) is not directly compute' has a grammatical error; it should read 'not directly computed'.","section":"Section 3.2"},{"comment":"The sentence 'As discussed in ??' contains an unresolved cross-reference; please fill in the section number.","section":"Section C.2"},{"comment":"The phrase 'o mitigate potential degeneration' should be 'to mitigate potential degeneration'.","section":"Section A.3"},{"comment":"Table 7 reports hyperparameters only for the 10M, 85M, and 300M models, but Section 5.7 and Figure 3 include 707M and 1.5B models; please provide the architecture and training details for these larger models.","section":"Table 7, Section 5.7"},{"comment":"The data-scaling experiment states that multiple sequences are generated from the same graph but does not specify how the multiple sequences are produced (random node orders, random tie-breaking in Algorithm 1, or both); please document the augmentation procedure.","section":"Section 5.7"},{"comment":"The row labels 'A s (Ours)' appear garbled; the two rows labeled 'A' and 's (Ours)' should be clearly distinguished in the table or its caption.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the manuscript is technically sound in its core derivation and the empirical study is substantial, but the 'general-purpose foundation model' framing is stronger than the evidence, mainly because of the ordering-sensitivity results and the lack of uncertainty quantification in the main generative tables. I recommend major revision rather than rejection because the concerns are addressable by added experiments and/or reframing. The self-citations to EDGE, DAGG, and Order Matters are natural given the topic and are not a concern."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate contribution to graph generation, not a revolution. The idea—serialize a graph as all node tokens then all edge triplets, and train a standard GPT-style decoder to predict the sequence—is clean, and the experiments are extensive enough to take the main generative claim seriously. The lower-bound argument in §3.2 is correct: maximizing sequence NLL maximizes a lower bound on graph likelihood, and the paper doesn't overstate what that bound shows.\n\nThe empirical core is the strongest part. On the generic graph benchmarks and MOSES/QM9, G2PT is competitive with or better than recent diffusion and flow baselines, and the scaling analysis (model size and number of sequences per graph) is a nice addition. The paper also ships code, which deserves credit. The fine-tuning experiments for property prediction are a reasonable bonus, and the results are in line with GraphMAE-level performance.\n\nThe soft spots are real but not fatal. First, the degree-based edge-removal ordering is load-bearing. Table 9 shows that a uniform ordering collapses validity and FCD on MOSES, and the conclusion explicitly concedes that different domains may prefer different orderings. That means the 'general graph encoding' claim is too strong; at present the representation is a per-dataset serialization choice. The paper should either provide evidence for a canonical or transferable ordering or temper the foundation-model language. Second, Table 2 reports no error bars for the MMD metrics; given how close some baseline comparisons are, that matters. Third, Section 5.5 shows only density plots for goal-oriented generation, with no quantitative comparison to any existing method—that is the weakest part of the paper. The 'superior performance' phrase in the abstract also oversells the molecular results: on GuacaMol and some novelty metrics, G2PT is not uniformly ahead.\n\nThe stress-test note about order-sensitivity is on target. It doesn't sink the paper, because the authors already admit it and the main generative results hold for the chosen ordering. But it does cap the significance: this is a strong within-subfield architecture/representation result, not a general-purpose foundation model.\n\nRecommendation: send it to review. A serious referee should ask for error bars, a quantitative goal-oriented baseline, and a rewrite of the order-sensitivity discussion. With those, it can be accepted.","headline":"A solid, honest empirical paper that makes a real case for edge-list tokenization for graph generation, but the load-bearing degree-based ordering is a dataset-specific design choice that undercuts the 'general foundation model' claim.","tokens_in":25032,"tokens_out":2436,"would_cite":true,"duration_ms":23164,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that graph generation can be cast as next-token prediction over a node-then-edge token sequence, and that a Transformer trained this way matches or surpasses diffusion-based and other autoregressive baselines on generic…","keywords":["graph generation","next-token prediction","Transformer decoder","edge-list representation","molecular generation","graph property prediction","fine-tuning","tokenization"],"falsifier":"Take a family of regular graphs, where every node has the same degree and Algorithm 1's min-degree choices are arbitrary tie-breaks, and train G2PT with any fixed tie-breaking rule; if validity and MMD statistics fall to the level of uniform ordering, then the degree-based ordering is not what makes the representation work.","tokens_in":24058,"feed_emoji":"🧬","tokens_out":6753,"duration_ms":58256,"temperature":0.7,"pith_summary":"This paper argues that graph generation can be treated as a language-modeling problem: a graph is written out as a token sequence that lists all nodes first and then all edges, and a Transformer decoder is trained to predict the next token. The proposed model, G2PT, learns graph structure by maximizing the likelihood of such sequences, which the authors show is equivalent to maximizing a lower bound on the graph likelihood and requires only O(n + m) tokens instead of the O($n^{2}$) entries of an adjacency matrix. Across four generic graph datasets and three molecular benchmarks, G2PT matches or surpasses permutation-invariant diffusion and autoregressive baselines, and it can be fine-tuned by rejection sampling or PPO for goal-oriented molecular generation and by a supervised head for graph property prediction. If these results hold, sparse graph generation, molecular design, and graph representation learning can share one Transformer-based sequence model.","feed_headline":"Edge-list tokens beat adjacency matrices for graph generation","feed_subtitle":"A decoder-only Transformer predicts node tokens then edge tokens, matching or beating diffusion models on seven datasets.","key_machinery":"The load-bearing object is a token-based graph representation: every graph is serialized as node-type/index tokens, a separator token, then edge triples (source index, destination index, edge type), with special start and end tokens. Edge order is fixed by Algorithm 1, which repeatedly removes the edge whose source has the minimum degree, breaks ties by destination degree, and reverses the removal list, so generation proceeds from a dense core to the low-degree periphery. This ordering turns an unordered graph into a one-dimensional language over a unified vocabulary, giving the Transformer a next-token prediction task and reducing the number of variables from O($n^{2}$) adjacency entries to O(m) actual edges; the ordering choice is what carries the model's ability to learn structure.","core_discovery":"The central claim is that an autoregressive Transformer trained with next-token prediction over node-then-edge token sequences is a general and competitive graph generative model. The paper identifies the edge-ordering rule as the main design decision: the reverse of a degree-based edge-removal process (Algorithm 1) produces a compact dense-core-first construction order, and the model learns the distribution over these ordered token sequences. G2PT is shown to outperform strong discrete-diffusion and flow baselines on QM9, MOSES, and GuacaMol for molecules and on Planar, Tree, Lobster, and SBM for generic graphs, while the same pre-trained model, fine-tuned, improves QED, SA, and GSK3-beta objectives and produces graph-level embeddings competitive with self-supervised baselines on eight classification tasks.","pith_inferences":["The order sensitivity the paper concedes suggests that a learned or canonical 'assembly grammar' for graphs could be the next step; if such a universal ordering exists, G2PT-style models could become cross-domain foundation models rather than per-dataset encoders.","Because the representation uses only existing edges, very large sparse graphs such as social networks or knowledge graphs are a natural stress test the paper does not run; sequence length grows with edges, not with the square of nodes.","Property prediction via the final-token embedding could be extended to regression, link prediction, or conditional generation with prompts, since the paper only demonstrates classification.","The comparison between rejection sampling fine-tuning and PPO suggests that distribution-shaping under hard constraints may be more reliably achieved by iterative filtering than by reward optimization when the target region is rare."],"forward_implications":["Sparse graphs are generated with O(n + m) tokens instead of the O(n^2) entries of an adjacency matrix, making sequence length proportional to the actual number of edges.","Once graphs are token sequences, standard Transformer techniques—model scaling, data augmentation, rejection sampling fine-tuning, and PPO—apply directly to graph generation without new architecture design.","The same pre-trained sequence model can be fine-tuned for property classification using the final-token embedding, matching self-supervised graph learners on eight molecule classification tasks.","Edge-ordering choice is a first-class design decision: BFS and degree-based orderings support strong generation, while uniform ordering collapses validity, so any claim about the representation must be paired with an ordering strategy.","Larger models and multiple sequences per graph improve validity until saturation, with G2PT scaling from 1M to 1.5B parameters across molecular datasets."],"supporting_citations":[{"why":"Defines sequential adjacency-matrix generation; it is the adjacency-matrix approach that G2PT's token representation is contrasted against.","marker":"You et al., 2018b"},{"why":"DiGress is the main discrete-diffusion baseline and supplies the evaluation protocol used for QM9 and generic graph datasets.","marker":"Vignac et al., 2022"},{"why":"MOSES provides the molecular benchmark and the evaluation metrics of validity, uniqueness, FCD, SNN, and scaffold similarity.","marker":"Polykovskiy et al., 2020"},{"why":"GuacaMol supplies the de novo molecular design benchmark and its evaluation pipeline.","marker":"Brown et al., 2019"},{"why":"MoleculeNet provides the eight classification datasets and the scaffold-split protocol used for graph property prediction.","marker":"Wu et al., 2018a"},{"why":"GraphMAE defines the data-splitting protocol and serves as the leading self-supervised baseline for the property-prediction comparison.","marker":"Hou et al., 2022a"},{"why":"PPO is the reinforcement-learning fine-tuning algorithm used for goal-oriented molecular generation.","marker":"Schulman et al., 2017"},{"why":"Supplies the pre-training loss term that G2PT incorporates into PPO to prevent model degeneration during fine-tuning.","marker":"Zheng et al., 2023"}],"fun_headline_variants":["Autoregressive token order beats diffusion for graphs","G2PT: next-token prediction for graph generation","Node-then-edge tokens outdo adjacency matrices","Graph transformer excels from generation to prediction","One pre-trained model for molecules and networks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reverse of a degree-based edge-removal process is a good, general way to order the edges of a graph for a language model to learn; if different graph domains need different edge orderings, then G2PT is a dataset-specific encoder rather than a general graph foundation.","fun_headline_variants_meta":{"raw":{"variants":["Autoregressive token order beats diffusion for graphs","G2PT: next-token prediction for graph generation","Node-then-edge tokens outdo adjacency matrices","Graph transformer excels from generation to prediction","One pre-trained model for molecules and networks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000269,"raw_usage":{"total_tokens":1586,"prompt_tokens":873,"completion_tokens":713,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":643}},"tokens_in":489,"tokens_out":713,"duration_ms":7306,"temperature":1.0,"reasoning_tokens":643,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:35:23.060168+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a family of regular graphs, where every node has the same degree and Algorithm 1's min-degree choices are arbitrary tie-breaks, and train G2PT with any fixed tie-breaking rule; if validity and MMD statistics fall to the level of uniform ordering, then the degree-based ordering is not what makes the representation work.","supporting_citations":[],"review_version":1}