Pith. sign in

REVIEW 4 major objections 5 minor 43 references

Graph World Model

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The Graph World Model (GWM) claims that a single world model can handle both structured and unstructured data by representing the world state as a multimodal graph and every task as an action node, and it reports matching or beating…

desk verdict A useful framework paper with a likely circular matching experiment and an unsupported 'single GWM' claim; the abstract oversells what the tables show. read the letter →

arxiv 2507.10539 v1 pith:NSNZLZMD submitted 2025-07-14 cs.LG

Tao Feng , Yexin Wu , Guanyu Lin , Jiaxuan You This is my paper
classification cs.LG
keywords worldmodelgraphfoundationmultimodallearningactionnodemessagepassingmulti-hopaggregationretrieval-augmentedgenerationzero-shotgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The Graph World Model (GWM) claims that a single model can act as a world model for both structured and unstructured data by representing the current world state as a graph and representing every task as an extra "action node" attached to that graph. The paper argues this is enough to unify six seemingly unrelated jobs—multimodal generation and matching, recommendation, graph prediction, multi-agent question answering, retrieval-augmented generation, and planning—under one set of weights. On those six benchmarks the same GWM is reported to match or exceed domain-specific baselines, to improve as graph neighborhoods grow from one to several hops, and to transfer to unseen tasks with little or no fine-tuning. If correct, graph structure would serve as a common substrate for multimodal foundation models rather than a separate subfield.

What carries the argument

The load-bearing object is the action node: a task is represented as a node that queries state nodes, either by direct reference (intended actions at node, edge, or graph level) or by embedding similarity (unintended actions such as RAG queries). Around it sits a generic message-passing step that propagates information across the graph—token-level aggregation for GWM-T, and for GWM-E a simplified GCN that computes $\tilde{A}^l X_e$ for $l$ hops with normalized adjacency $\tilde{A} = D^{-1/2} A D^{-1/2}$, keeping the first $L$ hops and projecting them into the decoder. This mechanism lets one model treat generation, prediction, and planning as the same operation: read the graph, aggregate neighborhoods, then decode the next state through the same LLM and diffusion heads.

What would settle it

Train one GWM model on all six tasks exactly as Section 5 describes, then evaluate it on held-out splits without any per-task fine-tuning; the single-model claim stands only if this joint checkpoint reproduces the reported per-task numbers. Separately, rerun the multi-hop ablation with hop count forced to zero: Figure 4 predicts every task should drop by at least the reported margin, so a flat curve would falsify the graph-benefit claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that a world model does not need to choose between unstructured and structured data: both can live in one graph state whose nodes carry text, image, or table content and whose edges are either explicit relations or embedding similarities. Tasks are added to the same graph as action nodes—node-level, edge-level, or graph-level "intended" actions that point directly at target nodes, or "unintended" actions that retrieve top-k similar nodes. GWM instantiates this in two ways: GWM-T converts all modalities to text and performs token-level message passing before instruction-tuning an LLM and Stable Diffusion, while GWM-E encodes modalities separately and performs embedding-level multi-hop aggregation through a simplified GCN-style propagation followed by a small projector that feeds a frozen LLM and diffusion decoder. The paper reports that one GWM trained across six tasks matches or exceeds domain-specific baselines, that multi-hop graph information consistently helps (with diminishing returns at higher hops), and that zero-shot and few-shot transfer to new tasks works.

Load-bearing premise

The paper's cross-task claim hinges on one model trained once on all six tasks with a single set of weights; if the reported numbers really came from separately trained or per-task-tuned checkpoints, the generalization story collapses.

Editorial extensions

If this is right

  • A single GWM checkpoint can be applied to tasks in world prediction, world generation, and world optimization without per-task architecture changes, as demonstrated on six tasks.
  • Graph structure carries usable signal beyond token content: multi-hop aggregation improved GWM-E on all six tasks relative to a no-graph baseline, with the largest gains on graph-related tasks.
  • GWM with a 2k-token context can outperform 128k-context LLMs on long-context RAG, indicating that retrieval over a chunk-similarity graph can substitute for extreme context length.
  • GWM-E achieves comparable or better results than GWM-T while consuming roughly 5 to 10 times fewer tokens, suggesting embedding-level message passing is the more scalable variant.
  • Zero-shot and few-shot experiments on Agent and RAG tasks show that training on other tasks transfers; on RAG, zero-shot GWM even beats training on RAG alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test this, but if the joint-training claim holds at larger scale, the natural next test is whether a GWM can absorb entire new datasets by adding their nodes and edges at inference time, without any weight update—action nodes would make each new benchmark a query rather than a new model.
  • The RAG result suggests a concrete extension: replace top-k similarity retrieval with learned multi-hop routing over the chunk graph, and compare against full-context models on longer documents where the 2k context limit binds.
  • The action-node formulation also implies a compositional interface: a task defined by multiple action nodes (for example, retrieve, then reason, then generate) could be chained without retraining, provided the decoders can consume the intermediate graph states.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes the Graph World Model (GWM), a framework that represents the world state as a multi-modal graph and represents tasks as action nodes. Two instantiations are introduced: GWM-T, which converts all modalities into text and performs token-level message passing, and GWM-E, which operates in an embedding space with modality-specific encoders and embedding-level message passing. The authors claim that 'the same GWM outperforms or matches domain-specific baselines' across six tasks: multi-modal generation and matching, recommendation, graph prediction, multi-agent collaboration, retrieval-augmented generation, and planning and optimization. They further claim that multi-hop graph structures consistently help and that GWM shows strong zero-shot/few-shot generalization. The paper includes experiments on Goodreads, Multi-Modal-Paper, Amazon Baby/Sports/Clothing, Cora, PubMed, HIV, AgentClinic, LongBench v2, and ALFWorld, with code released at GitHub.

Significance. If the central claims were fully supported, the paper would be a meaningful step toward a unified model that handles both structured and unstructured multi-modal data across diverse task families. The scope of the evaluation is unusually broad, and the public code release is a practical strength. The two variants, GWM-T and GWM-E, together with the reported efficiency gains of GWM-E (about 1/4 of GWM-T's training time and about 1/14 of its token length), are useful contributions. However, the empirical support currently falls short of the abstract's central claim, the joint-training protocol for the 'same GWM' is not described, and the matching-task setup raises a circularity concern. The significance is therefore conditional on a substantial revision that clarifies and, where necessary, corrects the claims.

major comments (4)
  1. [Section 5.1, Tables 2 and 3] The abstract and introduction claim that 'the same GWM outperforms or matches domain-specific baselines' performance,' but the paper's own tables contradict this. In Table 2 on Goodreads, INSTRUCTG2I achieves CLIP 50.37 and DINOv2 25.54, while the best GWM variant (GWM-T) reaches 47.46 and 20.91, and GWM-E reaches 45.23 and 20.87. In Table 3 on Goodreads matching, CLIP FT achieves Accuracy 92.60, Recall 92.58, and F1 92.61, while GWM-E reaches 88.82, 89.73, and 89.06 and GWM-T reaches 84.22, 85.66, and 85.29. The claim should be revised to match the reported numbers, or additional results should be provided that justify the current wording.
  2. [Section 5 (Implementation details), Table 12, Section 5.3] The claim that one 'same GWM' produces the reported results requires a precise joint-training protocol, but none is given. The manuscript states only 'We train and test a single GWM on all tasks,' while Table 12 lists separate hyperparameters for GWM-T LLM, GWM-T SD, GWM-E LLM, and GWM-E SD, and Section 5.3 describes additional training configurations ('Single Data,' 'Zero-shot,' 'Fine-tuned GWM'). It is therefore unclear whether Tables 2-8 come from one multi-task model trained on all six datasets, from component-level shared configurations, or from per-task runs. Please specify the exact multi-task objective, the sampling schedule over datasets, how the LLM and SD decoders are optimized in a single run, and the number of independent runs; if possible, release training logs or checkpoints that demonstrate the claimed joint model.
  3. [Section 2.2 and Section 5, multi-modal matching task] The multi-modal matching task appears to be solvable without learning cross-modal alignment, because the graph edges used as input encode the label. In the paper's formalism, implicit edges Em are constructed from embedding similarity, and the matching task is described as predicting correspondences between modalities 'including cross-modality similarity relationships' that are edges; the action node is then asked to decide whether two nodes should be connected. If the candidate edge is present in the input graph, or is implied by the same embeddings that serve as node features, the model can read the edge or the precomputed similarity rather than perform the matching. Please add an ablation that removes the candidate edge (or its similarity feature) from the input and report whether the model still solves the task.
  4. [Tables 2-8 and Section 5.3] All experimental results are reported as single deterministic numbers with no error bars, no number of seeds, and no significance tests. This matters because several 'matches' or 'outperforms' statements rest on differences of less than one percentage point; for example, in Table 5 Cora node classification, GWM-E reaches 83.03 versus GAT's 82.76, and in HIV graph classification, GWM-T reaches 92.20 versus OFA's 92.04. Please report mean and standard deviation over at least three seeds and, where appropriate, paired significance tests. The same applies to the zero-shot/few-shot comparisons in Figure 5.
minor comments (5)
  1. [Section 3 and Section 4 headings] The section headings 'Token-based GFM' and 'Embedding-based GFM' use the abbreviation GFM, but the paper's terminology is GWM; please correct for consistency.
  2. [Table 2 and Table 3 captions] The captions state that GWM achieved the best results, which is inconsistent with the values in the tables; INSTRUCTG2I is best on Goodreads generation and CLIP FT is best on Goodreads matching.
  3. [Figure 4] The y-axis of Figure 4 is labeled 'Performance Across Hops' with numerical values but no units, and the exact scores per setting are not listed; please add units and, if available, error bars.
  4. [Section 3.3 and Section 4.3] The phrase 'Instruction tuning of SD' is a misnomer because Eq. (2) is a diffusion denoising loss rather than an instruction-tuning objective; consider renaming it 'Diffusion fine-tuning' or similar.
  5. [Appendix A.1] One occurrence reads 'Multi-Model-Paper' instead of 'Multi-Modal-Paper'; please fix the typo.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: the multi-modal matching sub-result defines the target correspondence as an input graph edge and then predicts that edge; the rest of the framework is self-contained.

  1. self definitional [Section 2.3 (Multi-modal generation and matching), Table 14, Appendix A.1]
    "The multi-modal matching task (Rombach et al., 2022), similar to CLIP’s pre-training task (Radford et al., 2021), predicts the correspondence between modalities. It treats each modality as a state node and the correspondences between modalities (including cross-modality similarity relationships) (Jin et al., 2024) as edges. Here, the action node is at the edge level."

    The paper defines the matching task's ground truth as the graph's edges ('the correspondences between modalities ... as edges'), and Table 14's action prompt asks the model to 'determine whether they correspond with each other.' The input state graph, including these edges, is then passed through message passing (Eq. 1 for GWM-T; A and X_e for GWM-E in Section 4.2), so the positive/negative label is present as edge presence/absence in the model's input.

full rationale

Aside from the matching subtask, the GWM pipeline is self-contained: the token/embedding encoders, message-passing equations (Eq. 1 and Eq. 4), and projector/instruction tuning are described with explicit objectives and tested against external baselines. No load-bearing self-citation or imported uniqueness theorem is used; citations to the authors' prior graph work are contextual only. The 'same GWM' claim is under-specified—no joint multi-task loss or sampling schedule is given, and Table 12 lists separate hyperparameters for the LLM and SD decoders—but under-specification is an evidentiary gap, not circularity. The matching subtask, however, defines the label as an input edge and then asks the model to output that same edge, so Table 3's matching results are partially forced by construction. The CLIP-as-encoder-and-metric overlap in multi-modal generation is a metric-design concern, not a derivation-chain circularity. Overall, the circularity is partial and localized to one headline domain.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central claim rests on several domain assumptions about graph representation and similarity, plus hand-chosen hyperparameters such as hop count and top-k. The action node is an invented conceptual entity without independent empirical handle beyond the in-paper results.

free parameters (5)
  • Number of hops L
    The multi-hop aggregation uses L hops, and the experiments vary hops from 1 to 4. The optimal hop count is chosen per task without a principled rule, and the paper notes over-smoothing with too many hops.
  • Top-k in RAG = 5
    For retrieval-augmented generation, the top-5 chunks are selected by embedding similarity. This value is chosen without sensitivity analysis.
  • LoRA rank = 8
    GWM-T uses LoRA with rank 8 for efficient training. The choice is reported but not justified or varied.
  • Maximum token length = 2k
    All models use a maximum token length of 2k. This limits the amount of context and is a hand-chosen hyperparameter.
  • Projector dimensions = 2048*4096 (LLM), 2048*768 (SD)
    The multi-hop projector uses MLPs of these dimensions. These are architectural choices that influence capacity but are not swept.
assumptions (5)
  • domain assumption Representing a task as an action node linked to state nodes is a sufficient and general way to unify diverse tasks.
    The whole framework depends on this modeling choice. The paper asserts it in Section 2.2 but does not prove that all tasks can be faithfully expressed this way.
  • domain assumption Converting images and tables into text (GWM-T) or embeddings (GWM-E) preserves the information needed for the downstream tasks.
    The method relies on lossless or near-lossless transduction of modalities, which is not guaranteed, especially for tables with complex structure.
  • domain assumption Similarity-based edges capture meaningful relationships for the downstream tasks.
    Edges for RAG, planning, and matching are constructed from embedding similarity. This assumes similarity in embedding space corresponds to relevance or correspondence.
  • domain assumption Pretrained models (CLIP, BERT, LLaMA, Stable Diffusion) provide sufficiently good representations for all six tasks.
    The method does not train these backbones from scratch; it relies on their pretrained knowledge being compatible with graph aggregation.
  • domain assumption A single model trained jointly on all six tasks does not suffer from negative transfer that would prevent it from matching task-specific baselines.
    This is the central empirical assumption. The paper reports results but does not compare with per-task training, so negative transfer is not measured.
invented entities (1)
  • Action node
    purpose: A node in the world graph that represents a task or action, linked to target state nodes via reference or similarity, and used to query the state and generate outputs.
    The action node is a modeling construct introduced by the paper. Its only evidence is the empirical performance of the system; there is no external falsifiable prediction that does not depend on the model itself.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Graph World Model." pith.science (2026). https://pith.science/paper/NSNZLZMD

@misc{pith2026250710539,
  author       = {Pith},
  title        = {Pith review of: Graph World Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NSNZLZMD}},
  note         = {Machine review of arXiv:2507.10539}
}
read the original abstract

World models (WMs) demonstrate strong capabilities in prediction, generation, and planning tasks. Existing WMs primarily focus on unstructured data and cannot leverage the ubiquitous structured data, often represented as graphs, in the digital world. While multiple graph foundation models have been proposed, they focus on graph learning tasks and cannot extend to diverse multi-modal data and interdisciplinary tasks. To address these challenges, we propose the Graph World Model (GWM), a world model that supports both unstructured and graph-structured states with multi-modal information and represents diverse tasks as actions. The core of a GWM is a generic message-passing algorithm to aggregate structured information, either over a unified multi-modal token space by converting multi-modal data into text (GWM-T) or a unified multi-modal embedding space by modality-specific encoders (GWM-E). Notably, GWM introduces action nodes to support diverse tasks, where action nodes are linked to other nodes via direct reference or similarity computation. Extensive experiments on six tasks from diverse domains, including multi-modal generation and matching, recommendation, graph prediction, multi-agent, retrieval-augmented generation, and planning and optimization, show that the same GWM outperforms or matches domain-specific baselines' performance, benefits from multi-hop structures, and demonstrates strong zero-shot/few-shot capabilities on unseen new tasks. Our code for GWM is released at https://github.com/ulab-uiuc/GWM.

Figures

Figures reproduced from arXiv: 2507.10539 by the authors.

Figure 1
Figure 1. Multi-modal world state transition can be modeled via graphs. We model the current state as a graph and each node contains one or more modalities from image, table, and text. Further, the world action is modeled as an action node that queries the current state nodes. We categorize actions into two types: intended actions, which include three levels—node, edge, and graph—and unintended actions, whose implementation i… view at source ↗
Figure 2
Figure 2. Instantiations of GWM. (a) Multi-modal generation and matching contains two sub-tasks. For multi-modal generation, it models modal clusters as nodes, whereas for multi-modal matching, it models nodes for each modality. It includes edges that represent inter-modal correspondences and cross-modal similarities. For these two types of subtasks, there are node-level and edge-level action nodes, respectively. (b) The stat… view at source ↗
Figure 3
Figure 3. Framework of GWM. For both token-based and embedding-based GFM, we initially unify the multi-modal current state into graph nodes, conduct message passing, and then combine actions to predict the next state across different modalities through respective decoders. The key distinctions are: 1) Token-based GWM integrates multi-modalities into text, whereas embedding-based GWM uses modality-specific encoders to process … view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: GWM boosts zero-shot/few-shot performance on multi-agent collaboration (Agent) and retrieval-augmented generation (RAG) tasks. Note that “-T” and “-E” respectively represent the experimental results of GWM-T and GWM-E. It can be observed that GWM can quickly adapt to n…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 15 canonical work pages

  1. [1]

    Long- bench v2: Towards deeper understanding and reason- ing on realistic long-context multitasks

    Bai, Y ., Tu, S., Zhang, J., Peng, H., Wang, X., Lv, X., Cao, S., Xu, J., Hou, L., Dong, Y ., et al. Long- bench v2: Towards deeper understanding and reason- ing on realistic long-context multitasks. arXiv preprint arXiv:2412.15204,

    arXiv

  2. [4]

    Relational Multi-Task Learning: Modeling Relations between Data and Tasks

    Cao, K., You, J., and Leskovec, J. Relational multi-task learning: Modeling relations between data and tasks. arXiv preprint arXiv:2303.07666,

    work page Pith review arXiv

  3. [9]

    Retrieval-augmented generation for large language models: A survey

    Gao, Y ., Xiong, Y ., Gao, X., Jia, K., Pan, J., Bi, Y ., Dai, Y ., Sun, J., and Wang, H. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997,

    arXiv

  4. [10]

    In Multi-Modal-Paper, there are 58565 text-nodes, 7380 figure-nodes, and 6792 table-nodes

    Data statistics for multi-modal generation and match- ing. In Multi-Modal-Paper, there are 58565 text-nodes, 7380 figure-nodes, and 6792 table-nodes. Dataset #Node #Edges Goodreads 93,475 637,210 Multi-Modal-Paper 72,737 51,840 • SD-1.5 FT: Stable Diffusion v1.5 models, each fine-tuned separately on the training splits of the Goodreads and Multi-Modal-Pap...

    work page 2022

  5. [11]

    It includes three datasets of different scales, with the sizes ranging from small to large as follows: Baby, Sports, and Clothing

    Data statistics for recommendation. It includes three datasets of different scales, with the sizes ranging from small to large as follows: Baby, Sports, and Clothing. Dataset #User #Item #Edges Sparsity Baby 19,445 7,050 160,792 99.883% Sports 35,598 18,357 296,337 99.955% Clothing 39,387 23,033 278,677 99.969% A.2. Recommendation Dataset descriptions. In...

    work page 2015

  6. [13]

    Kipf, T. N. and Welling, M. Semi-supervised classifica- tion with graph convolutional networks. arXiv preprint arXiv:1609.02907,

    arXiv

  7. [14]

    Li, X. L. and Liang, P. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190,

    arXiv

  8. [15]

    How to train your dragon: Di- verse augmentation towards generalizable dense retrieval

    Lin, S.-C., Asai, A., Li, M., Oguz, B., Lin, J., Mehdad, Y ., Yih, W.-t., and Chen, X. How to train your dragon: Di- verse augmentation towards generalizable dense retrieval. arXiv preprint arXiv:2302.07452,

    arXiv

Show all 43 references
  1. [16]

    One for all: Towards training one graph model for all classification tasks

    Liu, H., Feng, J., Kong, L., Liang, N., Tao, D., Chen, Y ., and Zhang, M. One for all: Towards training one graph model for all classification tasks. arXiv preprint arXiv:2310.00149, 2023a. Liu, H., Li, C., Wu, Q., and Lee, Y . J. Visual instruction tun- ing. Advances in neura...

    arXiv

  2. [17]

    Dy- namic llm-agent network: An llm-agent collaboration framework with agent team optimization

    Liu, Z., Zhang, Y ., Li, P., Liu, Y ., and Yang, D. Dy- namic llm-agent network: An llm-agent collaboration framework with agent team optimization. arXiv preprint arXiv:2310.02170, 2023b. Madotto, A., Lin, Z., Winata, G. I., and Fung, P. Few-shot bot: Prompt-based learning for...

    arXiv

  3. [18]

    Masked transformer for neighhourhood-aware click-through rate prediction

    Min, E., Rong, Y ., Xu, T., Bian, Y ., Zhao, P., Huang, J., Luo, D., Lin, K., and Ananiadou, S. Masked transformer for neighhourhood-aware click-through rate prediction. CoRR, abs/2201.13311,

    arXiv

  4. [19]

    A content-driven micro-video recommen- dation dataset at scale

    Ni, Y ., Cheng, Y ., Liu, X., Fu, J., Li, Y ., He, X., Zhang, Y ., and Yuan, F. A content-driven micro-video recommen- dation dataset at scale. arXiv preprint arXiv:2309.15379,

    arXiv

  5. [20]

    Dinov2: Learning robust visual features without supervision

    Oquab, M., Darcet, T., Moutakanni, T., V o, H., Szafraniec, M., Khalidov, V ., Fernandez, P., Haziza, D., Massa, F., El- Nouby, A., et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193,

    arXiv

  6. [21]

    Instruc- tion tuning with gpt-4

    Peng, B., Li, C., He, P., Galley, M., and Gao, J. Instruc- tion tuning with gpt-4. arXiv preprint arXiv:2304.03277,

    arXiv

  7. [22]

    Graph retrieval-augmented generation: A survey

    Peng, B., Zhu, Y ., Liu, Y ., Bo, X., Shi, H., Hong, C., Zhang, Y ., and Tang, S. Graph retrieval-augmented generation: A survey. arXiv preprint arXiv:2408.08921,

    arXiv

  8. [23]

    Modeling relational data with graph convolutional networks

    Schlichtkrull, M., Kipf, T., Bloem, P., Van Den Berg, R., Titov, I., and Welling, M. Modeling relational data with graph convolutional networks. arxiv. arXiv preprint arXiv:1703.06103,

    arXiv

  9. [25]

    Alfworld: Aligning text and embodied environments for interactive learning

    Shridhar, M., Yuan, X., C ˆot´e, M.-A., Bisk, Y ., Trischler, A., and Hausknecht, M. Alfworld: Aligning text and embodied environments for interactive learning. arXiv preprint arXiv:2010.03768,

    2010 arXiv

  10. [26]

    How crucial is transformer in decision transformer? arXiv preprint arXiv:2211.14655,

    Siebenborn, M., Belousov, B., Huang, J., and Peters, J. How crucial is transformer in decision transformer? arXiv preprint arXiv:2211.14655,

    arXiv

  11. [28]

    Graph attention networks.arXiv preprint arXiv:1710.10903, 2017a

    Veliˇckovi´c, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y . Graph attention networks.arXiv preprint arXiv:1710.10903, 2017a. Veliˇckovi´c, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y . Graph attention networks.arXiv preprint arXiv:...

    arXiv

  12. [29]

    doi: 10.18653/v1/P19-1248

    Association for Computa- tional Linguistics. doi: 10.18653/v1/P19-1248. URL https://aclanthology.org/P19-1248/. Wang, Z., Wang, Z., Srinivasan, B., Ioannidis, V . N., Rang- wala, H., and ANUBHAI, R. Biobridge: Bridging biomedical foundation models via knowledge graphs. In The ...

    doi

  13. [30]

    ivideogpt: Interactive videogpts are scalable world models

    Wu, J., Yin, S., Feng, N., He, X., Li, D., Hao, J., and Long, M. ivideogpt: Interactive videogpts are scalable world models. arXiv preprint arXiv:2405.15223, 2024a. Wu, S., Sun, F., Zhang, W., Xie, X., and Cui, B. Graph neural networks in recommender systems: a survey.ACM Comp...

    arXiv

  14. [31]

    Y ., Prabhumoye, S., McAleer, S., Bisk, Y ., Salakhutdinov, R., Li, Y ., and Mitchell, T

    Wu, Y ., Fan, Y ., Min, S. Y ., Prabhumoye, S., McAleer, S., Bisk, Y ., Salakhutdinov, R., Li, Y ., and Mitchell, T. Agen- tkit: Flow engineering with graphs, not coding. arXiv preprint arXiv:2404.11483, 2024b. Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Ge- niesse, C.,...

    arXiv

  15. [32]

    Adding conditional control to text-to-image diffusion models

    Zhang, L., Rao, A., and Agrawala, M. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3836–3847, 2023a. 13 Graph World Model Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., ...

    1904

  16. [33]

    Retrieval-augmented generation for ai-generated content: A survey

    Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y ., Fu, F., Yang, L., Zhang, W., and Cui, B. Retrieval-augmented generation for ai-generated content: A survey. arXiv preprint arXiv:2402.19473,

    arXiv

  17. [34]

    E., and Elhoseiny, M

    Zhu, D., Li, L. E., and Elhoseiny, M. Value memory graph: A graph-structured world model for offline reinforcement learning. arXiv preprint arXiv:2206.04384,

    arXiv

  18. [35]

    Language agents as optimizable graphs

    Zhuge, M., Wang, W., Kirsch, L., Faccio, F., Khizbullin, D., and Schmidhuber, J. Language agents as optimizable graphs. arXiv preprint arXiv:2402.16823,

    arXiv

  19. [36]

    and our curated Multi-Modal- Paper dataset. (1) Goodreads: The Goodreads dataset is a large-scale col- lection of book-related metadata, textual descriptions, and cover images, widely used in prior multi-modal research (Jin et al., 2024). The Goodreads dataset is structured as...

    2024

  20. [39]

    Molecular struc- tures and graph representations are generated from SMILES strings, with atoms (nodes) and bonds (edges) described using natural language

    that contains over 40,000 compounds annotated for their ability to inhibit HIV replication. Molecular struc- tures and graph representations are generated from SMILES strings, with atoms (nodes) and bonds (edges) described using natural language. Baselines details. The setting...

    2017

  21. [40]

    prompt- ing, where the model is provided with a limited number of in-context examples to guide task-specific reasoning without requiring fine-tuning. A.5. Retrieval-augmented generation The purpose of Retrieval-Augmented Generation (RAG) is to enhance the generation capabiliti...

    2020

  22. [42]

    Here we finetune it on the dataset of this task

    : It is a versatile language model designed by Google Research, which treats every language problem as a text-to-text task, enhancing its adaptability across a broad range of NLP applications. Here we finetune it on the dataset of this task. B. Hyper-parameters For the GWM-E, ...

  23. [43]

    This task is to generate a response that integrates information from the retrieved documents to accurately address the user’s query

    Task description and output comparison of retrieval-augmented generation. This task is to generate a response that integrates information from the retrieved documents to accurately address the user’s query. Here we utilize one case of LongBench v2 dataset as examples. We show ...

    1990

  24. [2009]

    Their details are as follows: • BM25: A widely-used ranking function in information sparse retrieval

    and Dragon (Lin et al., 2023)—and three long-context LLMs (128k), including Mistral Large 2 4, Command R+ 5, and GPT-4o mini6. Their details are as follows: • BM25: A widely-used ranking function in information sparse retrieval. It inputs the retrieved context along with the q...

    2023

  25. [2014]

    From local to global: A graph rag approach to query-focused summarization

    Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., and Larson, J. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130,

    arXiv

  26. [2015]

    Graph convolutional reinforcement learning

    Jiang, J., Dun, C., Huang, T., and Lu, Z. Graph convolutional reinforcement learning. arXiv preprint arXiv:1810.09202,

    arXiv

  27. [2016]

    J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685,

    arXiv

  28. [2017]

    Agentclinic: a multimodal agent bench- mark to evaluate ai in simulated clinical environments

    Schmidgall, S., Ziaei, R., Harris, C., Reis, E., Jopling, J., and Moor, M. Agentclinic: a multimodal agent bench- mark to evaluate ai in simulated clinical environments. arXiv preprint arXiv:2405.07960,

    arXiv

  29. [2018]

    Instructg2i: Synthesizing images from multimodal at- tributed graphs

    Jin, B., Pang, Z., Guo, B., Wang, Y .-X., You, J., and Han, J. Instructg2i: Synthesizing images from multimodal at- tributed graphs. arXiv preprint arXiv:2410.07157,

    arXiv

  30. [2019]

    M., Hao, Y ., Stoeckius, M., Smibert, P., and Satija, R

    Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W. M., Hao, Y ., Stoeckius, M., Smibert, P., and Satija, R. Comprehensive integration of single-cell data. cell, 177(7):1888–1902,

    1902

  31. [2020]

    Bruce, J., Dennis, M

    URL https:// arxiv.org/abs/2005.14165. Bruce, J., Dennis, M. D., Edwards, A., Parker-Holder, J., Shi, Y ., Hughes, E., Lai, M., Mavalankar, A., Steiger- wald, R., Apps, C., et al. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning,

    2005 arXiv

  32. [2021]

    Llaga: Large language and graph assistant

    Chen, R., Zhao, T., Jaiswal, A., Shah, N., and Wang, Z. Llaga: Large language and graph assistant. arXiv preprint arXiv:2402.08170, 2024a. Chen, S., Hong, Z., Xie, G., Peng, Q., You, X., Ding, W., and Shao, L. Gndan: Graph navigated dual attention network for zero-shot learnin...

    arXiv

  33. [2022]

    Exploring the po- tential of large language models (llms) in learning on graphs

    Chen, Z., Mao, H., Li, H., Jin, W., Wen, H., Wei, X., Wang, S., Yin, D., Fan, W., Liu, H., et al. Exploring the po- tential of large language models (llms) in learning on graphs. ACM SIGKDD Explorations Newsletter, 25(2): 42–61, 2024b. Cui, H. and Gao, Y . A universal world mo...

    2023

  34. [2023]

    E., Ranjan, R., Robinson, J., Ying, R., You, J., and Leskovec, J

    Fey, M., Hu, W., Huang, K., Lenssen, J. E., Ranjan, R., Robinson, J., Ying, R., You, J., and Leskovec, J. Rela- tional deep learning: Graph representation learning on relational databases. arXiv preprint arXiv:2312.04615,

    arXiv

  35. [2024]

    E., and Cohan, A

    Beltagy, I., Peters, M. E., and Cohan, A. Long- former: The long-document transformer. arXiv preprint arXiv:2004.05150,

    2004 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.