{"id":"afcd49ca-ccdf-4d25-8f4f-add244e17bcc","arxiv_id":"2411.14662","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A transformer with multiplicity-aware attention preserves permutation invariance, reduces complexity relative to Set Transformer, and improves persistence diagram classification over a PersLay baseline.","lead":"The authors introduce a Multiset Transformer that adds a learnable multiplicity bias to standard attention, so repeated points in a persistence diagram get extra weight without duplicating entries. They report that this architecture beats the PersLay baseline on most graph classification benchmarks while using less memory and computation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim of outperforming existing neural network methods is contradicted by the paper's own Appendix A: Set Transformer was not benchmarked on real PDs and the authors hypothesize it would outperform MST.","rationale":"The reader's weakest assumption focused on the PERSLAY baseline being copied without error bars and possibly under a different protocol. That is a real concern, but the more load-bearing issue is the paper's own Appendix A: the closest attention-based alternative, the Set Transformer, was not run on the real-world PD benchmarks, and the authors explicitly hypothesize it would outperform MST in most real-world scenarios. This directly undercuts the broad central claim 'outperforms existing neural network methods' regardless of whether the PERSLAY comparison is protocol-matched. I therefore partially agree with the reader: both concerns concern baseline adequacy, but the omitted Set Transformer comparison is the sharper and more self-evident problem. The existing CONDITIONAL verdict remains appropriate because the architecture itself is plausible and the permutation-invariance proofs are credible; the condition is that the empirical superiority claims must be narrowed or supported by direct comparison with Set Transformer and by a properly matched PERSLAY baseline with variance reported.","tokens_in":18462,"tokens_out":9284,"duration_ms":94641,"concrete_test":"Run the Set Transformer on the PD classification benchmark for the smallest datasets (MUTAG, COX2, IMDB-B, IMDB-M) using the exact 10-fold x 5-run protocol of Section 6.2.4 and the same PD features, using clustering only if needed for tractability. If ST reaches or exceeds MST accuracy on those datasets, the abstract's general outperformance claim must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and Section 6.2.4 claim MST 'outperforms existing neural network methods' in persistence diagram representation learning, and Section 6.2.4 specifically claims it exceeds PERSLAY on a majority of datasets. The experiments, however, benchmark only PERSLAY (Table 2), with PERSLAY values copied from Carrière et al. (2020, Table 7) without standard deviations. Appendix A (Table 4) reports that on synthetic multisets the Set Transformer is comparable or better (11-class: ST 42.02±1.85 vs MST 41.14±2.24) and then explicitly states that ST was not included in Table 2 due to computational resources and that 'we hypothesize that the ST, owing to its higher model complexity, is likely to outperform the MST in most real-world scenarios.' This is a direct admission that the headline superiority claim has not been tested against a closely related attention-based multiset method and may be false in the real-world setting. At minimum, the claim must be narrowed to 'outperforms PERSLAY', and the protocol-matching issues (no standard deviations for PERSLAY, possible different cross-validation repetitions) must be resolved.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Multiset Transformer (MST), an attention-based architecture for representation learning on multisets, with applications to persistence diagram (PD) vectorization. MST separates a multiset into a base set and a multiplicity vector, and adds a learnable multiplicity bias to the attention weights, preserving multiplicities across equivariant layers and using a pool-decomposition scheme for invariance. The paper proves permutation equivariance of the multiset self-attention block and permutation invariance of the multiset attention with learnable queries, and gives a complexity analysis showing that MST avoids the duplicate-expansion cost of the Set Transformer. Experiments include a synthetic multiset classification task and graph classification on nine real-world datasets, where MST is compared with PersLay as the only neural baseline, plus an ablation study of the multiplicity terms.","tokens_in":18643,"tokens_out":10507,"duration_ms":87980,"significance":"If the claims are properly supported, the MST is a useful architectural contribution: it is one of the first attention mechanisms that explicitly consumes multiset structure, and the multiplicity bias is a principled way to respect the distinction between unique items and their multiplicities. The theoretical proofs (Theorems 5.1 and 5.2) are clean for the single-head case, and the complexity reduction relative to the Set Transformer (Section 5.5) is real and potentially important for large persistence diagrams. The paper also provides source code and detailed hyperparameters, which aids reproducibility. However, the empirical contribution is currently not at the level of the theoretical one: the only neural baseline is PersLay with copied numbers, and the paper's own Appendix A concedes that a stronger attention-based baseline may outperform MST on real data. The overall significance is therefore moderate, contingent on a more careful empirical comparison.","major_comments":[{"comment":"The abstract and Section 6.2.4 claim that MST outperforms existing neural network methods for persistence diagram representation learning, but Table 2 compares only against PersLay. Appendix A (Table 4) reports that the Set Transformer matches or exceeds MST on the 11-class synthetic task (ST 42.02±1.85 vs. MST 41.14±2.24) and the authors state that they 'hypothesize that the ST, owing to its higher model complexity, is likely to outperform the MST in most real-world scenarios.' This internal admission shows that the general superiority claim is not established; the claim must be narrowed to 'outperforms PersLay' or additional attention-based baselines must be evaluated on the real-world datasets.","section":"Abstract and Section 6.2.4"},{"comment":"The PersLay baseline values are copied from Carrière et al. (2020, Table 7) without standard deviations, so the reader cannot assess whether the reported differences are statistically significant. The paper states that it 'adopts the experimental settings described in Carrière et al. (2020)' (Section 6.2.3), but does not demonstrate that the folds, seeds, diagram types, and hyperparameter tuning are identical across the two studies. Since the performance differences in some rows are small (e.g., NCI1 Ordinary: 69.12±0.17 vs. PersLay 68.9), the claim that MST 'exceeds' PersLay requires either rerunning PersLay under the same protocol or providing a statistical comparison that accounts for the variability of both methods.","section":"Section 6.2.4, Table 2"},{"comment":"The text claims that 'consistently across all datasets, the MST model, when fully equipped with PD and its associated multiplicities, demonstrates superior performance over its counterparts,' but Table 3 shows the opposite for IMDB-M, where MST (w/o mult.) achieves 50.64±0.41 versus 50.33±0.17 for the full MST. In addition, the NCI1 row reports the MST value as 68.65±1.23 in Table 2 but 68.65±0.14 in Table 3, despite the statement that the MST column in Table 3 is the same as in Table 2. The overclaim and the numerical inconsistency should be corrected.","section":"Section 6.2.5, Table 3"},{"comment":"The synthetic experiment defines class labels by the element with the highest frequency in each multiset, so the multiplicity-aware attention has an advantage by construction. This makes the experiment a valid sanity check of the mechanism, but it does not provide external evidence of general performance superiority. The text should frame it as such and avoid suggesting that the synthetic gains transfer to real-world tasks without additional support.","section":"Section 6.1"}],"minor_comments":[{"comment":"The title contains a stray space: 'Persistence Diagram s' should read 'Persistence Diagrams'.","section":"Title"},{"comment":"The heading contains a typo: 'Multset Attention with Learnable Queries' should be 'Multiset Attention with Learnable Queries'.","section":"Section 5.2.3 heading"},{"comment":"The proof states that 'the softmax function is applied element-wise (or row-wise)'; standard softmax is row-wise, so the parenthetical should be removed or corrected to avoid confusion.","section":"Appendix B.1"},{"comment":"The notation is inconsistent: n denotes the number of unique elements in the multiset here, while in Section 5.2 n and m denote numbers of queries and keys. Please unify the notation to prevent ambiguity.","section":"Section 5.5"},{"comment":"The DBSCAN eps values are listed per dataset, but no sensitivity analysis is provided; since clustering is a preprocessing step that affects both accuracy and complexity, a brief discussion of the sensitivity to eps (or a reference to the chosen values) would strengthen the presentation.","section":"Table 6"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a clean architectural idea with solid permutation guarantees and a clear complexity advantage over the Set Transformer. The main risk is the empirical comparison: the headline claim of outperforming existing neural network methods rests on a single baseline with copied numbers, and the paper's own appendix undercuts the claim. This is fixable by rerunning baselines and narrowing the claims, so I do not see it as a rejection. I would also suggest the editor ask for a discussion of the 'first' claim regarding multiset attention, since the related-work survey may not be exhaustive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The Multiset Transformer is a sensible, incremental extension of Set Transformer: add a normalized multiplicity bias to the attention weights, prove permutation equivariance/invariance for the single-head case, and keep the architecture simple. That part is solid. The proofs are correct as far as I can tell, the complexity reduction from O(n^2 m^2) to O(n^2) for multiset inputs is real, and the ablation study in Table 3 shows the multiplicity term helps on real PDs. The clustering preprocessing is also a pragmatic idea, and the fact that it often costs little accuracy is worth knowing.\n\nThe soft spots are all on the empirical side. The abstract and Section 6.2.4 claim MST outperforms existing neural network methods, but the only neural baseline is PersLay, with numbers copied from Carrière et al. (2020) without standard deviations. That is fragile enough on its own. Worse, Appendix A says Set Transformer was not run on real PDs due to compute limits, and the authors explicitly hypothesize that ST would outperform MST in most real-world scenarios because of its higher model complexity. That sentence sits in direct tension with the headline claim. The synthetic experiment also defines labels by the highest-frequency element, so the multiplicity bias is advantaged by construction; the real-data ablation is more informative. The DBSCAN eps values are hand-set per dataset, which makes the clustering results a bit less clean, though not disqualifying.\n\nNone of this is fatal. The architecture is clearly presented, the theory checks out, and the idea is new enough to be worth publishing. But the 'outperforms existing neural network methods' sentence should be narrowed to 'outperforms PersLay on the tested datasets,' and the authors should either run Set Transformer or, if compute is truly prohibitive, say plainly that a head-to-head comparison with ST is future work. The abstract's 'first' claim is fine; I have not seen a multiset-specific attention layer in the cited literature.\n\nWho is this for? Someone working on PD vectorization or on attention mechanisms for multiset inputs will get value. It is not a breakthrough, but it is a working method with verifiable properties. I would send it to peer review, with the expectation that the empirical claims get tightened and at least one additional baseline is attempted.\n\nRecommendation: accept for peer review, but flag the overbroad empirical claims and the appendix contradiction as major revision items.","headline":"Useful multiset attention variant with clean proofs, but the 'outperforms existing methods' claim rests on a single copied baseline and is contradicted by the paper's own appendix.","tokens_in":19223,"tokens_out":1572,"would_cite":true,"duration_ms":16640,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","55N31"],"pacs":[],"model":"deepseek-v4-flash","headline":"The Multiset Transformer treats persistence diagrams as multisets, adding a learnable multiplicity bias to attention to preserve permutation invariance and cut complexity.","keywords":["multiset transformer","persistence diagrams","topological data analysis","permutation invariance","attention mechanism","graph classification","representation learning","multiplicity bias"],"falsifier":"Rerun PersLay and MST from scratch on the same ten folds, the same number of repetitions, and the same train/test protocol for all nine graph datasets; if PersLay equals or beats MST on a majority of entries, the central superiority claim fails.","tokens_in":18200,"feed_emoji":"📊","tokens_out":7486,"duration_ms":63335,"temperature":0.7,"pith_summary":"Persistence diagrams record the births and deaths of topological features, and they naturally contain duplicate points; treating them as lists loses this multiplicity and inflates cost. This paper proposes the Multiset Transformer, an attention architecture that keeps a base set and a multiplicity vector separate, adds a learnable multiplicity bias to attention, and proves the resulting self-attention is permutation equivariant while the learned-query attention is permutation invariant. If correct, it is the first attention network built for multisets rather than sets, it cuts the quadratic cost of a duplicated input by a factor of the maximum multiplicity, and it gives a neural vectorization of persistence diagrams that beats the PersLay baseline on most of the tested graph datasets. The practical payoff is a representation learning pipeline that can also cluster a persistence diagram first and still keep accuracy, making large diagrams feasible.","feed_headline":"Multiset Transformer beats PersLay on persistence diagrams","feed_subtitle":"A multiset-aware attention network uses point multiplicities to beat PersLay and slash duplication cost.","key_machinery":"The load-bearing mechanism is the multiplicity-bias term $B$ injected inside the softmax attention. It is built only from the multiplicity vectors $(M_Q-\\mathbf{1})(M_X-\\mathbf{1})^\\top$, normalized by its Frobenius norm, so it vanishes when all multiplicities are one and the model reduces to ordinary set attention. This term is what makes the equivariance and invariance proofs go through while directing extra attention to high-multiplicity points. Around it, the architecture follows the pool-decomposition scheme: permutation-equivariant Multiset Attention Blocks (SAB and IMAB) carry the base set through hidden layers, and an invariant block with learned queries (MAB$_Q$) produces the final representation, with multiplicities re-entering at the invariant layer.","core_discovery":"The central claim is that a persistence diagram should be encoded as a multiset $(X, M_X)$ — a base set of distinct points plus their multiplicities — and processed by attention with a dedicated multiplicity-bias term. The multiset-enhanced attention is $A(Q,X)=(\\mathrm{softmax}(QX^\\top/\\sqrt{d})+\\alpha B)X$ with $B=(M_Q-\\mathbf{1})(M_X-\\mathbf{1})^\\top/(\\|(M_Q-\\mathbf{1})(M_X-\\mathbf{1})^\\top\\|_F+\\varepsilon)$, and the learned-query variant uses a learnable vector in place of $\\alpha$. The paper proves Theorem 5.1 (permutation equivariance of multiset self-attention) and Theorem 5.2 (permutation invariance of learned-query attention), then stacks equivariant blocks before an invariant pooling layer. Because the base set is not expanded by multiplicities, the complexity is $O(n^2)$ and $O(nq)$ instead of $O(n^2m^2)$ and $O(nmq)$. Empirically, on ordinary and extended persistence diagrams of nine graph classification datasets, the Multiset Transformer reports accuracies above the PersLay baseline on a majority of entries, and an ablation shows the multiplicity term drives most of the gain.","pith_inferences":["I would expect the same multiplicity-bias attention to transfer to bag-of-words and multiple-instance learning, where multiplicities are already available; a quick test would be replacing set pooling in those pipelines and measuring accuracy versus complexity.","The paper's Appendix A notes that on the 11-class synthetic task the Set Transformer slightly edges out MST, which suggests the duplication-expanded model can still win when expressiveness matters more than cost; a hybrid that starts from multiplicities but widens the base set on demand is an untested middle ground.","The clustering results raise a question the paper leaves open: if clustering to 1% of the input can preserve or even improve accuracy, then some high-multiplicity points carry most of the signal; one could test this directly by ranking clusters by lifespan and ablating low-lifespan clusters.","A natural next benchmark is comparing MST against Persformer and other PD vectorizers on the same folds, since the paper only compares against PersLay numbers and notes Set Transformer is infeasible on real datasets."],"forward_implications":["On multiset inputs, MST has $O(n^2)$ time and space for SAB and $O(nq)$ for IMAB, versus $O(n^2m^2)$ and $O(nmq)$ for Set Transformer, so diagrams with high duplication become feasible.","Because clustering preprocessing maps a large diagram to few representative points with absorbed multiplicities, MST can process very large persistence diagrams with only small accuracy loss.","The ablation results imply multiplicities are not incidental: removing them lowers accuracy on most real-world datasets, for example by 3.62% on IMDB-B extended diagrams.","The synthetic experiments show MST without multiplicity is near random on 5- and 11-class tasks, while MST with multiplicity reaches 88.86% and 41.14%, establishing that the design goal of allocating attention by frequency is met."],"supporting_citations":[{"why":"Supplies the scaled dot-product attention and multihead mechanism that MST modifies with its multiplicity bias.","marker":"Vaswani et al. (2017)"},{"why":"Defines Set Transformer, the SAB/ISAB block terminology, and the O(n^2m^2)/O(nmq) baseline MST is compared against.","marker":"Lee et al. (2019)"},{"why":"Provides PersLay, the experimental protocol, graph topological signatures, and the baseline accuracies in Table 2.","marker":"Carrière et al. (2020)"},{"why":"Introduces the pool-decomposition/Deep Sets formulation that grounds the permutation invariance architecture.","marker":"Zaheer et al. (2017)"},{"why":"Prior transformer-based PD vectorization (Persformer) that MST positions itself against as a multiset-aware alternative.","marker":"Reinauer et al. (2021)"}],"fun_headline_variants":["Multiset Transformer uses multiplicity-aware attention to beat PersLay","Multiset Transformer: lower cost, higher accuracy on persistence diagrams","Multiset-aware attention beats PersLay and cuts complexity on persistence diagrams","Multiset Transformer: multiplicity cuts cost and boosts diagram accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The empirical superiority claim depends on assuming that the PersLay numbers quoted from an earlier paper were produced under the same ten-fold, repeated-run protocol as the new Multiset Transformer results, so the two columns can be compared directly.","fun_headline_variants_meta":{"raw":{"variants":["Multiset Transformer uses multiplicity-aware attention to beat PersLay","Multiset Transformer: lower cost, higher accuracy on persistence diagrams","Multiset-aware attention beats PersLay and cuts complexity on persistence diagrams","Multiset Transformer: multiplicity cuts cost and boosts diagram accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000973,"raw_usage":{"total_tokens":4123,"prompt_tokens":918,"completion_tokens":3205,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":3133}},"tokens_in":534,"tokens_out":3205,"duration_ms":22560,"temperature":1.0,"reasoning_tokens":3133,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:03:55.145580+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun PersLay and MST from scratch on the same ten folds, the same number of repetitions, and the same train/test protocol for all nine graph datasets; if PersLay equals or beats MST on a majority of entries, the central superiority claim fails.","supporting_citations":[],"review_version":1}