{"id":"965ddb71-d680-4d44-976a-c385e7ab2982","arxiv_id":"2411.14460","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A single hypergraph encoder and G-Former, pretrained on 25 million tables, can be appended to different LLMs to improve their performance on table, knowledge graph, and database structured knowledge grounding tasks.","lead":"LLaSA turns tables, knowledge graphs, and databases into one hypergraph format, encodes them with a pretrained graph transformer, and feeds the compressed result to a large language model as soft prompts. The paper reports that this one encoder, pretrained once on tables, improves structured-data question answering across four different LLMs, including with parameter-efficient fine-tuning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The universal structured-data claim rests on table-only pretraining transferring to graph tasks; Table 3 ablates pretraining only on the held-in average, mixing 7 table tasks with 3 graph tasks, so per-domain transfer is unverified.","rationale":"The reader identified the same load-bearing premise: table-only pretraining must transfer to knowledge-graph and database hypergraphs built with the same conversion rules. I agree that this is the central unverified assumption. The manuscript's own ablation (Table 3) supports the value of pretraining and of the GNN broadly, but it aggregates over ten held-in datasets, of which only three are graph-based (KVRet, CompWebQ, DART in the held-in set). No per-task ablation is given, so the reader cannot determine whether the pretraining gain comes from table tasks or graph tasks. This matters because the pretrained modules are decoupled from any specific LLM and are, as described, used frozen at the downstream stage; if they are frozen, their ability to handle graph-structured hypergraphs must come from table pretraining plus the shared hypergraph formulation. The directionality ambiguity in Section 3.1 compounds the concern: if the reverse-relation node is connected to both head and tail just as the normal relation node is, the incidence structure cannot distinguish h--r-->t from t--r-->h, and the GNN branch would be unable to reason about directed KG edges. A check of the released code for CompWebQ/KVRet/DART conversion can resolve this quickly. The reader's conditional verdict remains appropriate: the framework is plausible and the ablations are encouraging, but the universal claim needs per-domain evidence before acceptance. I therefore recommend no change to the conditional verdict rather than a stronger or weaker one.","tokens_in":15012,"tokens_out":10092,"duration_ms":108626,"concrete_test":"Run the frozen-LLM setting from Table 3 (LLaSA-Llama2-7B) with the pretrained versus randomly initialized GNN and G-Former, and report results separately for the three KG tasks (CompWebQ, KVRet, DART) and for the table tasks, instead of only Avg-I and Avg-O. If the pretrained-checkpoint deltas on the KG tasks are within noise of zero or negative while table-task deltas are large, the table-only pretraining does not transfer to graph hypergraphs and the universal structured-data claim is unsupported for the graph portion. Repeating with three seeds and reporting standard deviations would also settle whether the 0.2-point average SOTA difference is meaningful.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that one table-pretrained hypergraph encoder and G-Former, apparently frozen during instruction tuning (Eq. 8-9 defines p_theta with theta as the LLM parameters, and Section 3.3.2 says 'we only use the graph transformer module pretrained in the pretraining stage'), improves accuracy on knowledge-graph tasks such as CompWebQ, KVRet, and DART as well as on table tasks. The only evidence for the pretraining contribution is Table 3, which reports Avg-I and Avg-O across all held-in and held-out datasets but never separates the three KG tasks from the table tasks. Because pretraining is performed exclusively on 25 million web tables (Section 4.3), and because the graph-to-hypergraph conversion (Section 3.1) introduces relation and reverse-relation hyperedges that have no direct counterpart in table rows or columns, it is entirely possible that the gains shown in Table 3 are driven by table tasks while the frozen encoder contributes little or nothing on KG tasks. If so, the abstract's claim that one pretrained encoder 'enhance[s] their ability to process different types of structured data' fails for the graph portion of the benchmark, and the KG results would rest on the serialized-text input plus soft prompts rather than on the pretrained hypergraph modality. A related ambiguity in Section 3.1 is that both head and tail nodes are described as connected to the relation hyperedge, and the reverse-relation node is added without specifying which node attaches to it; if both nodes attach to both hyperedges, the incidence structure is direction-symmetric and the GNN cannot represent directed triples. The released code should be checked to resolve this. The transfer question is the more load-bearing issue because it concerns the method's core universality, independent of the thin 0.2-point SOTA margin.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LLaSA, a framework that converts structured data (tables, knowledge graphs, databases) into a unified hypergraph representation, encodes it with a HyTrel-style hypergraph transformer, compresses the encoded nodes into fixed-length query tokens with a Q-Former-like G-Former, and feeds those tokens as soft prompts to a frozen or LoRA-tuned LLM. The hypergraph encoder and G-Former are pretrained with two self-supervised objectives, graph-dependent answer generation and graph-text matching, on 25 million web tables from TaBERT. The authors evaluate LLaSA on ten held-in SKG tasks (seven table-related and three knowledge-graph tasks) and three held-out table tasks, across four base LLMs and two tuning strategies. They report consistent gains over the base LLMs in the frozen-LLM setting, smaller gains under LoRA tuning, and claim that LLaSA with LoRA surpasses the prior full-parameter-tuning state of the art, StructLM.","tokens_in":15303,"tokens_out":10524,"duration_ms":97986,"significance":"If the central claims hold, the paper would make a useful contribution: a single pretrained hypergraph encoder and G-Former that can be plugged into different LLMs as a structured-data adapter, avoiding LLM-coupled GNN pretraining. The idea of unifying tables, graphs, and databases as hypergraphs is appealing, and the paper provides initial evidence across several base models. The ablation study in Table 3 supports the importance of pretraining, the GNN, and the G-Former, and the authors are transparent about the known weakness of the answer-generation objective, which they mitigate with graph-text matching. However, the headline state-of-the-art claim rests on a very small average margin, and the transfer of table-only pretraining to knowledge-graph hypergraphs is not isolated in the ablations. The significance of the work is thus conditional on additional experiments that separate per-domain effects and quantify uncertainty.","major_comments":[{"comment":"The headline claim that LLaSA 'outperforms previous SOTA method using full parameters tuning' rests on an average held-in score of 66.9 versus StructLM's 66.7, a difference of 0.2 points. No standard deviations, number of seeds, or significance tests are reported, and on the held-out tasks LLaSA's average (27.2) is actually lower than StructLM's (27.7). Several individual task margins are also very small (e.g., HybridQA 62.9 vs 62.6, WikiSQL 87.1 vs 87.0, KVRet 72.3 vs 72.2). The paper uses the word 'significantly' in Section 4.4 without statistical support. As it stands, the SOTA claim is not supported by the reported evidence.","section":"Abstract, Section 4.4, Table 1"},{"comment":"The universal structured-data claim depends on the assumption that pretraining exclusively on 25 million web tables transfers to knowledge-graph hypergraphs. Table 3 reports only Avg-I and Avg-O, which aggregate seven table tasks (WikiTQ, HybridQA, FeTaQA, TabMWP, WikiSQL, TabFact, ToTTo) and three graph tasks (KVRet, CWQ, DART), so the marginal contribution of pretraining on the graph subset is not identifiable. Table 2's frozen-LLM rows show per-dataset gains on KVRet, CWQ, and DART, but those comparisons add the full LLaSA machinery (400M-parameter encoder plus G-Former plus soft prompts) and do not isolate the pretrained initialization. This concern is reinforced by Section 3.1, where graph conversion introduces relation and reverse-relation hyperedges that have no direct counterpart in table rows or columns. Please report the ablation per task or at least per domain, and consider including a graph-only pretraining comparison.","section":"Section 3.1, Section 4.3, Table 3"},{"comment":"The manuscript never explicitly states whether the hypergraph encoder and G-Former are frozen or updated during task-specific instruction tuning. The sentence 'we only use the graph transformer module pretrained in the pretraining stage' is ambiguous: it could mean that the text transformer is discarded while the graph transformer remains frozen, or that the graph transformer is initialized from pretraining and further trained. Equations (8)-(9) define p_theta only over LLM parameters theta, but Eq. (7) includes the G-Former and the fully connected layer without specifying their trainable status. The reusable-adapter claim depends on this distinction: if the 400M-parameter encoder and G-Former are retrained for each base LLM, then Table 2 demonstrates per-LLM fine-tuning rather than a single frozen structured-data adapter. Please state the exact trainable parameter sets for each finetuning strategy and, if these modules are frozen, confirm this explicitly.","section":"Section 3.3.2, Eq. (7)"}],"minor_comments":[{"comment":"The abstract contains a typo: 'Graph Neutral Networks' should be 'Graph Neural Networks'.","section":"Abstract"},{"comment":"Equation (6) appears to have a misplaced parenthesis: the intended form is likely Softmax(omega(X W_K)^T)(X W_V), not Softmax(omega(X W_K)^T (X W_V)), since the softmax should operate on the attention scores rather than on the product with the values.","section":"Eq. (6)"},{"comment":"The graph-to-hypergraph construction is underspecified: after introducing a reverse-relation node, the text does not state whether the reverse-relation hyperedge connects to the head entity, the tail entity, or both, nor how directionality is encoded in the attention mechanism. Please clarify.","section":"Section 3.1, Figure 3"},{"comment":"The HGT 7B-M and G-Retriever 7B-M results are described as re-implemented 'under our framework,' but no hyperparameters, training schedules, or validation checks are reported. Since these baselines are used to argue that projector-based GNN-LLM strategies underperform, the comparison needs more detail to be convincing.","section":"Section 4.2, Table 1"},{"comment":"The Graph-Text Matching objective is central to the alignment claim, but the negative sampling procedure is not described. Please specify how negative (graph, text) pairs are constructed and how many negatives are used per positive example.","section":"Section 3.3.1, Appendix A"},{"comment":"There are inconsistent model names: Table 1 lists 'LlaSA 7B-M' while the text and Table 3 use 'LLaSA'; Table 3 says 'LLaSA Llama-7B' but the accompanying text refers to Llama2-7B. Please standardize.","section":"Table 1, Table 3"},{"comment":"The statement that LLaSA 'only requiring 400M trainable parameters' is imprecise because the LoRA adapters in the base LLM are also trainable; the paper should distinguish between the pretrained modules' parameter count and the total number of trainable parameters under each strategy.","section":"Section 4.4"},{"comment":"The tables would be easier to interpret if the datasets were explicitly grouped by structured-data type (table, knowledge graph, database), since the paper's universal-encoder claim is per-type.","section":"Tables 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the venue and the core idea is promising, but the headline SOTA claim and the universal-transfer claim both need stronger evidence before publication. I would encourage the authors to add seed-level variance, per-domain ablations, and an explicit statement of which modules are frozen during instruction tuning."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"LLaSA is worth a serious look. The concrete contribution is a unified hypergraph representation for tables and knowledge graphs, a HyTrel-style encoder pretrained on 25M web tables with two self-supervised objectives, and a Q-Former-style cross-attention module (G-Former) that compresses the hypergraph into soft prompts for any LLM. The strongest evidence is in the frozen-LLM setting: with Mistral-7B, the average across the ten held-in tasks goes from 42.9 to 55.4, and the gains appear on the knowledge-graph tasks too (KVRet 43.7->65.7, CWQ 55.4->67.1). That is a strong signal that table-only pretraining transfers to graph-structured hypergraphs, which directly addresses the stress-test concern. The ablations also give the pretraining and the G-Former credit for the gains, and the paper is honest about its limitations (fixed query tokens, 2K context, no long-context evaluation).\n\nThe soft spots are real but mostly presentation. The 'outperforms full-parameter SOTA' headline rests on a 0.2-point average lead (66.9 vs 66.7 in Table 1) with no error bars or significance tests. In the LoRA setting the gains over the plain LLM are small on held-in tasks; the impressive wins are frozen. Calling the evaluation 'tables, graphs, and databases' overstates things: there is no actual database task in the benchmark, WikiSQL is table QA. That is a minor wording issue, not a method flaw.\n\nThe one substantive gap: the pretraining ablation (Table 3) reports averages over mixed table and graph tasks, so the transfer of the table-pretrained encoder to KG tasks is never shown per domain. The frozen numbers suggest it works, but the paper should just run that breakdown, especially for DART where the LoRA result is slightly negative. There is also a small ambiguity in the graph-to-hypergraph construction: the reverse-relation node is introduced without saying which nodes attach to it. If both head and tail attach to both relation hyperedges, direction is carried only by the hyperedge text labels. Probably fine, but the released code should confirm.\n\nFor a reader: this is a useful framework reference for structured-data adapters and a fair baseline for future work. It deserves peer review, not a desk reject, and I would ask for significance testing, the per-domain ablation, and a corrected 'database' claim before accepting.","headline":"LLaSA's unified hypergraph adapter is a solid engineering contribution with real frozen-LLM gains; the SOTA claim is thin and per-domain graph transfer is never directly ablated.","tokens_in":15947,"tokens_out":3732,"would_cite":true,"duration_ms":35247,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One hypergraph encoder lifts four LLMs on structured data.","keywords":["structured knowledge grounding","hypergraph","G-Former","soft prompts","self-supervised pretraining","LoRA fine-tuning","graph-text alignment","large language models"],"falsifier":"Take the knowledge-graph tasks CompWebQ, KVRet, and DART and ablate the hypergraph soft prompts entirely, leaving the same serialized input and LoRA training: the central claim predicts a clear accuracy drop; if scores stay flat, the reported gains come from text and tuning rather than from the structured-data modality. A second check is to pretrain the encoder on tables only and evaluate on graphs: if graph accuracy matches a text-only baseline, the claimed cross-type transfer has failed.","tokens_in":14772,"feed_emoji":"📊","tokens_out":9244,"duration_ms":82413,"temperature":0.7,"pith_summary":"LLaSA proposes that tables, knowledge graphs, and databases can all be converted into one universal hypergraph format and encoded by a single pretrained graph neural network, so that a large language model receives both the serialized text and a compact set of learned soft-prompt tokens derived from the structure. The authors pretrain this hypergraph encoder together with a cross-attention compressor called G-Former on 25 million web tables, deliberately without coupling them to any particular LLM, and then attach the same pretrained components to several different LLMs. On ten structured-knowledge-grounding tasks, adding the hypergraph tokens improves average performance across base models, and with low-rank (LoRA) fine-tuning the resulting model edges past a state-of-the-art generalist trained with full parameter updates while using far fewer trainable parameters. The claim that matters is that structured information lost during linear serialization can be recovered as a separate, reusable modality.","feed_headline":"One hypergraph encoder lifts four LLMs on structured data","feed_subtitle":"One table-pretrained adapter serves many language models and beats a full-parameter-tuned generalist with LoRA.","key_machinery":"The unifying object is the hypergraph built from any structured input, together with the two-stage encoder-compressor that reads it. HyperTrans alternates Node2Hyperedge and Hyperedge2Node set-attention layers with a hyperedge fusion MLP, producing representations of every cell or entity; G-Former then uses a fixed set of learnable query tokens, cross-attending to these representations and compressing them into a handful of soft tokens. The two self-supervised objectives, answer generation conditioned on the graph and a graph-text matching score, make the query tokens carry textual meaning before any LLM is attached. Because the pretraining is LLM-free, the same encoder and G-Former can be frozen or lightly tuned and plugged into a new language model, with the soft tokens placed in front of the serialized input.","core_discovery":"The paper's central claim is that a single hypergraph encoder, pretrained once with self-supervised objectives and decoupled from any particular language model, can serve as a universal structured-data interface for LLMs. Tables become hypergraphs by treating cells as nodes and rows and columns as hyperedges; knowledge graphs become hypergraphs by treating entities as nodes and relations as hyperedges, with reverse-relation nodes preserving direction. A structure-aware transformer called HyperTrans encodes each hypergraph, and a G-Former, modeled on the Q-Former design from vision-language pretraining, compresses the node representations into a fixed number of query tokens that are appended to the LLM's text embeddings as soft prompts. Two pretraining tasks, graph-dependent answer generation and graph-text matching, align the encoder with text without involving any LLM. Across ten benchmark tasks and four base models, the framework improves results over text-only prompting, and in the LoRA-tuned setting with Mistral-7B it reaches an average of 66.9 versus 66.7 for a full-parameter-tuned generalist baseline, with only about 400 million trainable parameters. The authors conclude that the pretrained encoder transfers across tables, knowledge graphs, and databases even though pretraining saw only tables.","pith_inferences":["Directly pretraining on knowledge-graph and database hypergraphs, not just tables, would test and likely strengthen the claimed universality; the current results only show that table pretraining transfers.","The fixed number of query tokens, which the authors flag as a limitation, makes the approach a candidate for adaptive query budgets: graphs with hundreds of nodes may need more than the current ten tokens.","The soft-prompt design suggests a practical recipe for any new LLM: append the same pretrained hypergraph interface rather than distilling graph structure into the model's weights.","A clean controlled experiment, removing the soft prompts or replacing the pretrained encoder with a random one, would quantify how much of the gain is due to the structured modality rather than the extra trainable parameters."],"forward_implications":["If the central claim is right, a single structured-data encoder can be reused across many LLMs, removing the need to re-pretrain a GNN for each new base model.","Freezing the LLM still gives large gains, roughly ten points on average, so the hypergraph path can be used even when the LLM cannot be fine-tuned.","LoRA fine-tuning yields small improvements on held-in tasks but consistently larger improvements on held-out datasets, suggesting the contribution is genuine structure understanding rather than memorization.","With about 400 million trainable parameters and LoRA, the framework edges past a full-parameter-tuned generalist baseline, so parameter-efficient tuning suffices for this modality."],"supporting_citations":[{"why":"Supplies the HyTrel hypergraph-enhanced tabular representation approach, including the HyperTrans architecture used as LLaSA's default hypergraph encoder.","marker":"(Chen et al., 2023a)"},{"why":"Provides the 25 million web tables used to pretrain the hypergraph encoder and G-Former, and defines the table serialization context the method improves on.","marker":"(Yin et al., 2020)"},{"why":"Provides the Q-Former design and the two-stage pretraining idea that G-Former adapts to compress hypergraph representations into fixed query tokens.","marker":"(Li et al., 2023)"},{"why":"Defines the full-parameter-tuned generalist baseline that LLaSA compares against and surpasses with LoRA tuning.","marker":"(Zhuang et al., 2024)"},{"why":"One of the GNN-plus-LLM baselines re-implemented and trained under the same framework for a fairer comparison.","marker":"(He et al., 2024)"},{"why":"One of the GNN-plus-LLM baselines re-implemented and trained under the same framework for a fairer comparison.","marker":"(Jin et al., 2024)"},{"why":"Defines the UnifiedSKG benchmark suite and the text-to-text formulation of structured knowledge grounding tasks that LLaSA builds on.","marker":"(Xie et al., 2022)"},{"why":"Provides the TableLlama generalist table model with long-context LoRA tuning that LLaSA is compared against.","marker":"(Zhang et al., 2024b)"}],"fun_headline_variants":["One hypergraph encoder lifts LLMs on tables, graphs, DBs","Universal hypergraph adapter beats full-parameter tuning with LoRA","Tables-only pretraining yields a hypergraph encoder for many LLMs","LLaSA: a single encoder for structured data, tuned with LoRA to beat SOTA","Hypergraph encoder unifies structured data, then LoRA tunes it past SOTA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that pretraining on 25 million tables alone teaches the encoder and G-Former to represent knowledge-graph and database hypergraphs well enough to improve answers, even though no graph or database example is seen during pretraining.","fun_headline_variants_meta":{"raw":{"variants":["One hypergraph encoder lifts LLMs on tables, graphs, DBs","Universal hypergraph adapter beats full-parameter tuning with LoRA","Tables-only pretraining yields a hypergraph encoder for many LLMs","LLaSA: a single encoder for structured data, tuned with LoRA to beat SOTA","Hypergraph encoder unifies structured data, then LoRA tunes it past SOTA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00169,"raw_usage":{"total_tokens":6771,"prompt_tokens":1097,"completion_tokens":5674,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":713,"completion_tokens_details":{"reasoning_tokens":5574}},"tokens_in":713,"tokens_out":5674,"duration_ms":45273,"temperature":1.0,"reasoning_tokens":5574,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:18:14.621783+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the knowledge-graph tasks CompWebQ, KVRet, and DART and ablate the hypergraph soft prompts entirely, leaving the same serialized input and LoRA training: the central claim predicts a clear accuracy drop; if scores stay flat, the reported gains come from text and tuning rather than from the structured-data modality. A second check is to pretrain the encoder on tables only and evaluate on graphs: if graph accuracy matches a text-only baseline, the claimed cross-type transfer has failed.","supporting_citations":[],"review_version":1}