Neuro-Symbolic Meta-Policies for Temporal Knowledge-Graph Memory under Partial Observability

Michael Cochez; Taewoon Kim; Vincent Fran\c{c}ois-Lavet

REVIEW 2 major objections 5 minor 34 references

Neuro-Symbolic Meta-Policies for Temporal Knowledge-Graph Memory under Partial Observability

T0 review · 2 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A controller that learns which memory heuristic to apply at each step outperforms fixed symbolic rules in a partially observable knowledge-graph task while staying fully inspectable.

desk verdict A modest, honest paper whose central generalization claim is undercut by an internal inconsistency about what the held-out split actually is. read the letter →

arxiv 2607.18368 v3 pith:5OUXBLII submitted 2026-07-20 cs.AI

Taewoon Kim , Vincent François-Lavet , Michael Cochez This is my paper · ORCID

classification cs.AI

keywords neuro-symbolicreinforcementlearningpartialobservabilitytemporalknowledge-graphmemorymeta-policysymbolicheuristicsRDFgraphannotationsRoomKGneuralnetworks

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Partially observable agents must decide what to record, retrieve, and discard over time; this paper argues that those memory decisions can be learned without giving up symbolic control. The proposed meta-policy reads a memory made of RDF fact triples annotated with timestamps and access counts, and at every step chooses among three named retrieval rules (MRA, MRU, MFU), three exploration rules of the same family, and three forgetting rules (FIFO, LRU, LFU). The paper reports that, in the RoomKG navigation-and-query environment with a long-term memory cap of 512 facts, learned selection—particularly with the qualifier-aware StarE-GNN encoder—achieves the best held-out question-answering accuracy among the symbolic, neural, and neuro-symbolic systems compared, while every decision remains traceable to a named heuristic. A sympathetic reader would care because the result suggests that adaptive memory management does not require an opaque neural memory, and that picking among existing symbolic strategies is itself a learnable skill.

What carries the argument

The load-bearing object is the annotated RDF memory graph: every stored fact is a base subject-predicate-object triple plus statement-level qualifiers—time_added, last_accessed, num_recalled—so each memory item carries exactly the metadata that the heuristics rank. The symbolic action space is finite and named: MRA (most recently added), MRU (most recently used), and MFU (most frequently used) for question answering and exploration, and FIFO, LRU, and LFU for forgetting. The meta-policy encodes memory with a graph neural network (GCN, R-GCN, or StarE-GNN), pools node embeddings with head-specific attention into three value heads, and picks the highest-valued heuristic for each category; a de

What would settle it

Run the best StarE-GNN meta-policy on held-out rooms whose layouts were never seen in training, across many seeds; if its test accuracy no longer beats the best fixed symbolic baseline by more than the noise level, the claim of improved held-out generalization from learned heuristic selection is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that memory management can be cast as a meta-decision problem: instead of learning what to remember, the agent learns which existing symbolic memory heuristic to apply at each step. In RoomKG at capacity 512, the qualifier-aware StarE-GNN configuration reaches 47.28 held-out test QA accuracy, ahead of the strongest fixed symbolic combination (46.28) and far ahead of end-to-end LSTM/Transformer baselines (around 9–11). The authors interpret the pattern as showing that adaptivity and inspectability need not be traded off: the controller's choices are named heuristics over explicit annotations, and qualifier-aware encoding is what lets the value heads exploit the re

Load-bearing premise

The central claim depends on the held-out test being a genuinely different generalization target and the roughly one-point lead over the best fixed baseline being more than seed noise; the reported protocol permutes question order rather than introducing new rooms, and the standard deviations are the same size as the margin.

Editorial extensions

If this is right

Adaptive memory control can stay inspectable: every QA, exploration, and forgetting choice is a named heuristic over explicit metadata, so failures can be traced to a concrete rule.
Encoder choice matters for heuristic selection: qualifier- and relation-aware encoders (StarE-GNN, R-GCN) outperform topology-only GCN, matching the information the heuristics depend on.
Modular three-head selection beats a single 27-way combinatorial head, suggesting that separate value heads with denser supervision are better for this decision problem.
End-to-end prediction over the full 245-action combinatorial space is poorly suited to bounded-memory temporal KG tasks; the symbolic substrate carries most of the performance.
Because fixed symbolic baselines themselves differ, which heuristic is active is a consequential variable even without learning—reinforcing the case for learned meta-selection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial: the same meta-selection mechanism should transfer to any memory system whose operations are named and ranked over metadata—buffer-pool replacement, cache eviction, episodic-memory stores—because the method's assumptions are only that memory is an annotated graph and operations are named heuristics.
Editorial: the abstract claims generalization to train/test room splits, while the experimental section describes the held-out test as the same dynamics with permuted question order; showing the result on genuinely unseen layouts would be the direct way to substantiate that stronger reading of the claim.
Editorial: a straightforward extension would be to vary the heuristic library and measure test accuracy as a function of library size; the authors expect a poor library would cap learned selection, so the curve should rise and plateau.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the contribution is real but narrow — learned selection among named symbolic memory heuristics via three value heads — and the paper is refreshingly honest about its limitations. But the abstract says "train/test room splits" while Section 4 describes the test set as "same underlying dynamics with permuted question order." That is a direct contradiction. If the test set contains no new rooms or layouts, the headline claim about generalization to held-out rooms is not supported by the protocol as written. The authors need to resolve this before the paper is taken at face value.

What is actually new: the three-head meta-policy (QA/explore/forget) over a 3x3x3 symbolic action space, with encoder choice (GCN/R-GCN/StarE-GNN) as a proper ablation. The idea of keeping execution symbolic while learning which heuristic to fire is clean, and the qualitative traces make the inspectability claim concrete. Code and data are released, which is real evidence. The authors also explicitly state that the substrate is prior work, so the self-citation pattern is fine.

Soft spots: (1) The split inconsistency is load-bearing. If the test is just a shuffled question order, the agent may have memorized the room layout, and Table 1 does not test the claimed "train/test room splits." (2) The best result is 47.28 vs 46.28 for a strong fixed baseline; with standard deviations around 1.0 and no significance test, that gap is not established beyond noise. (3) Only capacity 512 is evaluated, and only one benchmark family. The authors list these limitations themselves, so this is not an oversight, but it does cap the strength of the claims.

The paper is coherent on its own terms. The math is standard DQN with separate heads; no fitting-as-prediction is happening. The method is sensible and the empirical question is well-posed. I would not call the 1-point difference statistically meaningful based on what is reported.

Recommendation: send it to peer review, but with a firm request to resolve the split inconsistency and to report confidence intervals or a significance test. The contribution is legitimate and deserves referee time. I would not cite it in my own work until the split question is clarified; if clarified, it would be a useful reference for neuro-symbolic memory control.

Referee Report

2 major / 5 minor

Summary. The paper studies memory management in RoomKG, a partially observable gridworld whose observations and hidden state are represented as RDF graphs. It proposes a neuro-symbolic meta-policy that, given an encoded representation of the agent's bounded temporal knowledge-graph memory, selects among named symbolic heuristics for question answering (MRA/MRU/MFU), exploration (MRA/MRU/MFU), and forgetting (FIFO/LRU/LFU), and then executes the selected heuristics symbolically. The method is instantiated with GCN, R-GCN, or StarE-GNN encoders, head-specific attention pooling, and three DQN-style value heads (with a combinatorial 27-way head as an ablation). Experiments at memory capacity 512 compare fixed symbolic baselines, end-to-end LSTM/Transformer agents, and neuro-symbolic variants. The paper reports that StarE-GNN achieves the best held-out QA accuracy (47.28) over five seeds, and it provides qualitative traces showing interpretable switching among named heuristics.

Significance. If the empirical claims are supported, the paper offers a clean and timely formulation: learned selection over explicit symbolic memory operators rather than opaque latent memory policies, while preserving inspectability. The open-source code and the explicit limitations section are strengths. However, the empirical support is currently not conclusive: the described test protocol does not match the 'room split' claim, and the main advantage over strong fixed symbolic controls is within one standard deviation with no significance testing. The conceptual contribution can stand after a corrected evaluation or appropriately narrowed claims.

major comments (2)

[Abstract / §1 / §4 (Experimental Setup)] The abstract and §1 claim 'train/test room splits' and 'strong train/test generalization', but §4 defines the held-out environment as 'same underlying dynamics with permuted question order.' This does not hold out rooms or layouts; test performance could reflect memorization of a fixed room graph. The protocol as written therefore does not test the headline generalization claim. Please state exactly what differs between train and test. If new-room generalization is intended, add an evaluation on unseen room configurations; otherwise revise the abstract and conclusions to claim only robustness to question-order permutation.
[Table 1 / §5.1] The central quantitative claim is that StarE-GNN is 'best' among all systems. With 5 seeds, StarE-GNN gives 47.28 ± 0.98, while the strongest fixed symbolic controls give 46.28 ± 0.89 and 46.28 ± 1.46, and R-GCN gives 47.00 ± 1.67. The point differences are less than the reported standard deviations, and no paired significance test, confidence interval, or effect size is reported. A ~1.0-point gain over a strong baseline cannot support 'best held-out performance' without such an analysis. Report seed-paired tests or per-episode bootstrap intervals, or soften the comparative claim.

minor comments (5)

[§3.2, Eq. (6)] The StarE-GNN message function is written abstractly with ϕr and ψ; give the concrete instantiation used in the experiments, since the qualifier-awareness claim rests on this choice.
[Table 1 caption] The caption says 'BFS denotes breadth-first exploration in the symbolic baselines,' but no row or column is labeled BFS. Clarify or remove.
[§4.2] The phrase 'head-specific handling for QA as described in Section 4' appears inside Section 4 itself and should refer to Section 3.4.
[Figure 3] State whether the Q-value trace is from a single seed and how representative it is; the supplement may contain all heads, but the main text should give the selection criterion.
[Abstract] Minor typo: 'meta-policy that learnswhich' should be 'meta-policy that learns which.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the central result is an empirical RL measurement; the abstract's 'room splits' wording and the small effect size are generalization/statistical concerns, not circularity.

full rationale

The paper's derivation chain is an empirical reinforcement-learning evaluation rather than a first-principles derivation. The learned meta-policy is trained by off-policy Q-learning on training episodes and evaluated on a separately described held-out test protocol; no parameter is fitted to the test quantity and then reported as a prediction. The symbolic heuristics and RoomKG substrate come from the authors' prior work [21], but the paper explicitly disclaims those as its contribution ('The contribution of this paper is this meta-policy layer and its controlled evaluation, not the benchmark or the memory substrate itself'), and it is a released, code-reproducible benchmark rather than an unverified self-citation used to force a conclusion. The encoders (GCN, R-GCN, StarE-GNN) are standard models cited from external literature. The main result is a measured point estimate (StarE-GNN test QA 47.28 vs. best fixed S-TKG 46.28), so it does not reduce to the paper's own definitions or fitted inputs by construction. Two concerns are worth noting but are not circularity: (1) the abstract claims 'train/test room splits' while Section 4 describes the held-out environment as 'same underlying dynamics with permuted question order', so the new-room generalization wording is not supported by the protocol as written; and (2) the advantage over the best fixed baseline is within one reported standard deviation with no significance test. These are correctness/evidence issues, not cases where a prediction is equivalent to its input. The paper's own limitation statement ('evidence covers one benchmark family and one primary capacity (512)') further confirms that the claims are bounded empirical claims, not circular derivations.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical or formal entities are postulated; the 'meta-policy' is a conventional learned controller over an existing heuristic space. The central claim depends on the hand-chosen capacity and heuristic library, and on the questionable assumption that the test protocol measures generalization.

free parameters (2)

Long-term memory capacity K = 512
Central results are reported only at K=512, a value taken from the prior RoomKG sweep; the paper does not test other capacities, so the headline result is conditional on this hand-selected budget.
Heuristic library (QA MRA/MRU/MFU, explore MRA/MRU/MFU, forget FIFO/LRU/LFU) = 3 heuristics per category
The learned meta-policy can only select among these hand-picked symbolic operators; the authors acknowledge that a poor library caps performance. This is a design choice, not fitted to test data.

assumptions (4)

domain assumption RoomKG environment dynamics and observation model as defined in [21]
The paper builds on the RoomKG benchmark from the authors' prior work; hidden-state schedules and object movement rules are taken as given.
domain assumption RDF triple annotations (time_added, last_accessed, num_recalled) faithfully summarize memory usefulness
The heuristic ranking and the value heads both operate on these annotation values; if these metadata are insufficient, the meta-policy cannot recover.
ad hoc to paper The test protocol (same underlying dynamics, permuted question order) constitutes a valid held-out evaluation
This assumption underlies the 'generalization' claim and is in tension with the abstract's 'room splits'; the protocol is described but not justified as a generalization test.
domain assumption Reward is generated solely by QA correctness, so exploration/forgetting credit can be learned through TD on downstream QA reward
The learning objective in Eqs. (15)-(16) relies on this reward structure; it may not transfer to settings with other rewards.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Neuro-Symbolic Meta-Policies for Temporal Knowledge-Graph Memory under Partial Observability." pith.science (2026). https://pith.science/paper/5OUXBLII

@misc{pith2026260718368,
  author       = {Pith},
  title        = {Pith review of: Neuro-Symbolic Meta-Policies for Temporal Knowledge-Graph Memory under Partial Observability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5OUXBLII}},
  note         = {Machine review of arXiv:2607.18368}
}

read the original abstract

Partially observable reinforcement learning requires deciding what to retain, retrieve, and forget over time. We introduce a neuro-symbolic meta-policy that learns which symbolic memory heuristic to apply at each decision point while keeping execution symbolic. Our setting uses temporal knowledge-graph memory in RoomKG, where hidden state and observations are represented as Resource Description Framework (RDF) graphs and memory is augmented with temporal RDF triple annotations. The model combines knowledge-graph encoding of memory contents with value heads for question answering, exploration, and forgetting, yielding a controller that is both adaptive and inspectable. This gives the work a direct Semantic Web grounding through RDF-based representation, annotation-compatible graph semantics, and graph-based symbolic operations over explicit memory state. On train/test room splits at long-term memory capacity of 512, the qualifier-aware StarE-GNN configuration achieves the best held-out performance among the compared symbolic, neural, and neuro-symbolic systems while preserving step-level traceability of memory-management decisions.

Figures

Figures reproduced from arXiv: 2607.18368 by the authors.

**Figure 1.** Bird’s-eye schematic of the hidden state at t = 99 (st=99) in RoomKG, showing spatial layout and entity placement. This view is only a schematic for intuition: the actual environment state and agent-facing world are represented in the RDF knowledgegraph world ( [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** Knowledge-graph view of the same hidden state at t = 99 (st=99), expressed as RDF structure. Node colors indicate semantic categories: rooms (yellow), agent (purple), static objects (blue), moving objects (green), and walls (grey). This is the structural form that motivates relation-aware encoders and explicit memory annotations in the proposed meta-policy. The full original figure is provided in the supplemental … view at source ↗

**Figure 3.** Per-step Q-value trajectories for the QA head of the learned StarE-GNN neurosymbolic model on the held-out test trace. Each curve corresponds to one symbolic QA heuristic, showing how preference shifts over semantically named choices rather than over opaque latent actions. The full three-head figure set, including exploration and forgetting, is provided in the supplemental material. are better matched to the decisi… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Memory-state graph at the end of the held-out test trace. For visibility, edge labels use parentheses (N), where N is the number of memories associated with the same RDF triple. Because one RDF triple can correspond to multiple annotated statements, these are condense…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 2 canonical work pages

[1]

In: International Conference on Learning Rep- resentations (2022), https://openreview.net/forum?id=4rLw09TgRw9

Alivanistos, D., Berrendorf, M., Cochez, M., Galkin, M.: Query embedding on hyper-relational knowledge graphs. In: International Conference on Learning Rep- resentations (2022), https://openreview.net/forum?id=4rLw09TgRw9

2022
[2]

In: International Conference on Learning Representations (2021)

Arakelyan, E., Daza, D., Minervini, P., Cochez, M.: Complex query answering with neural link predictors. In: International Conference on Learning Representations (2021)

2021
[3]

In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)

Bacon, P.L., Harb, J., Precup, D.: The option-critic architecture. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)

2017
[4]

In: International Conference on Learning Representations (2015)

Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: International Conference on Learning Representations (2015)

2015
[5]

W3C Recommendation (Feb 2014), https://www.w3.org/TR/rdf11-concepts/, 25 February 2014

Cyganiak, R., Wood, D., Lanthaler, M.: Rdf 1.1 concepts and abstract syntax. W3C Recommendation (Feb 2014), https://www.w3.org/TR/rdf11-concepts/, 25 February 2014

2014
[6]

Journal of Artificial Intelligence Research65, 1–30 (2019)

Francois-Lavet, V., Rabusseau, G., Pineau, J., Ernst, D., Fonteneau, R.: On overfitting and asymptotic bias in batch reinforcement learning with partial ob- servability. Journal of Artificial Intelligence Research65, 1–30 (2019). https: //doi.org/10.1613/jair.1.11478

work page doi:10.1613/jair.1.11478 2019
[7]

In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (2020)

Galkin, M., Trivedi, P., Maheshwari, G., Usbeck, R., Lehmann, J.: Message passing for hyper-relational knowledge graphs. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (2020)

2020
[8]

d’Avila Garcez, A., Dutra, A.R.R., Alonso, E.: Towards symbolic reinforcement learning with common sense (2018), https://arxiv.org/abs/1804.08597

arXiv 2018

Show all 34 references

[9]

org/abs/1410.5401

Graves, A., Wayne, G., Danihelka, I.: Neural turing machines (2014), https://arxiv. org/abs/1410.5401

2014 arXiv
[10]

Nature538, 471–476 (2016) 18 T

Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska- Barwinska, A., Colmenarejo, S.G., Grefenstette, E., Ramalho, T., Agapiou, J.P., Badia, A.P., Hermann, K.M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Sum- merfield, C., Blunsom, P., Kavukcuoglu, K., H...

2016
[11]

In: The Semantic Web: Research and Applications

Gutierrez, C., Hurtado, C.A., Vaisman, A.A.: Temporal RDF. In: The Semantic Web: Research and Applications. Lecture Notes in Computer Science, vol. 3532, pp. 93–107. Springer (2005)

2005
[12]

Harini, S.I., Shroff, G., Srinivasan, A., Faldu, P., Vig, L.: Neuro-symbolic meta reinforcement learning for trading (2023), https://arxiv.org/abs/2302.08996

2023 arXiv
[13]

W3C working draft, W3C (Mar 2026), https://www.w3.org/TR/sparql12-query/

Hartig, O., Seaborne, A., Taelman, R., Williams, G., Tanon, T.P.: SPARQL 1.2 query language. W3C working draft, W3C (Mar 2026), https://www.w3.org/TR/sparql12-query/

2026
[14]

Hausknecht, M., Stone, P.: Deep recurrent q-learning for partially observable mdps (2017), https://arxiv.org/abs/1507.06527

2017 arXiv
[15]

W3C Recommendation (Feb 2014), https://www.w3.org/TR/rdf11-mt/, 25 February 2014

Hayes, P.J., Patel-Schneider, P.F.: Rdf 1.1 semantics. W3C Recommendation (Feb 2014), https://www.w3.org/TR/rdf11-mt/, 25 February 2014

2014
[16]

Neural Comput.9(8), 1735–1780 (Nov 1997)

Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput.9(8), 1735–1780 (Nov 1997). https://doi.org/10.1162/neco.1997.9.8.1735

1997 doi
[17]

ACM Computing Surveys54(4), 1–37 (Jul 2021)

Hogan, A., Blomqvist, E., Cochez, M., D’amato, C., Melo, G.D., Gutierrez, C., Kirrane, S., Gayo, J.E.L., Navigli, R., Neumaier, S., Ngomo, A.C.N., Polleres, A., Rashid, S.M., Rula, A., Schmelzeisen, L., Sequeda, J., Staab, S., Zimmermann, A.: Knowledge graphs. ACM Computing Su...

2021 doi
[18]

Artificial Intelligence101(1-2), 99–134 (1998)

Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artificial Intelligence101(1-2), 99–134 (1998). https://doi.org/10.1016/S0004-3702(98)00023-X

1998 doi
[19]

W3C candidate recommendation snapshot, W3C (Apr 2026), https://www.w3.org/TR/rdf12-concepts/

Kellogg, G., Hartig, O., Champin, P.A., Seaborne, A.: RDF 1.2 concepts and ab- stract data model. W3C candidate recommendation snapshot, W3C (Apr 2026), https://www.w3.org/TR/rdf12-concepts/

2026
[20]

W3C working draft, W3C (Apr 2026), https://www.w3.org/TR/rdf12-turtle/

Kellogg, G., Seaborne, A., Tomaszuk, D.: RDF 1.2 turtle. W3C working draft, W3C (Apr 2026), https://www.w3.org/TR/rdf12-turtle/

2026
[21]

Kim, T., François-Lavet, V., Cochez, M.: Temporal knowledge-graph memory in a partially observable environment (2026), https://arxiv.org/abs/2408.05861

2026
[22]

In: International Conference on Learning Representations (2017)

Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (2017)

2017
[23]

In: Meila, M., Zhang, T

Landajuela, M., Petersen, B.K., Kim, S., Santiago, C.P., Glatt, R., Mundhenk, N., Pettit, J.F., Faissol, D.: Discovering symbolic policies with deep reinforcement learning. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Pr...

2021
[24]

In: Raedt, L.D

Mitchener, L., Tuckey, D., Crosby, M., Russo, A.: Detect, understand, act: A neuro- symbolic hierarchical reinforcement learning framework (extended abstract). In: Raedt, L.D. (ed.) Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCA...

2022 doi
[25]

Nature518(7540), 529–533 (2015)

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves,A.,Riedmiller,M.,Fidjeland,A.K.,Ostrovski,G.,Petersen,S.,Beattie,C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep ...

2015 doi
[26]

Journal of Web Semantics12-13, 3–21 (2012)

Motik, B.: Representing and querying validity time in RDF and OWL: A logic- based approach. Journal of Web Semantics12-13, 3–21 (2012). https://doi.org/ 10.1016/j.websem.2011.11.004 Neuro-Symbolic Meta-Policies for Temporal KG Memory 19

2012 doi
[27]

In: International Conference on Learning Representations (2017)

Parisotto, E., Salakhutdinov, R.: Neural map: Structured memory for deep re- inforcement learning. In: International Conference on Learning Representations (2017)

2017
[28]

In: Proceedings of the 34th International Conference on Machine Learning (2017)

Pritzel, A., Uria, B., Srinivasan, S., Puigdomènech, A., Vinyals, O., Hassabis, D., Wierstra, D., Blundell, C.: Neural episodic control. In: Proceedings of the 34th International Conference on Machine Learning (2017)

2017
[29]

In: The Semantic Web: ESWC 2018

Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: The Semantic Web: ESWC 2018. pp. 593–607. Springer (2018)

2018
[30]

Artificial Intelligence112(1), 181–211 (1999)

Sutton, R.S., Precup, D., Singh, S.: Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence112(1), 181–211 (1999). https://doi.org/https://doi.org/10.1016/S0004-3702(99)00052-1

1999 doi
[31]

In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R

Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, L., Polosukhin, I.: Attention is all you need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30 ...

2017
[32]

Foundations and Trends®in Machine Learning11(3–4), 219–354 (Dec 2018)

Vincent, F.L., Peter, H., Riashat, I., Marc, G.B., Joelle, P.: An introduction to deep reinforcement learning. Foundations and Trends®in Machine Learning11(3–4), 219–354 (Dec 2018). https://doi.org/10.1561/2200000071

2018 doi
[33]

Wayne, G., Hung, C.C., Amos, D., Mirza, M., Ahuja, A., Grabska-Barwinska, A., Rae, J., Mirowski, P., Leibo, J.Z., Santoro, A., Gemici, M., Reynolds, M., Harley, T., Abramson, J., Mohamed, S., Rezende, D., Saxton, D., Cain, A., Hillier, C., Silver, D., Kavukcuoglu, K., Botvinic...

2018 arXiv
[34]

In: International Conference on Learning Representations (2019)

Zambaldi, V., Raposo, D., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D., Lillicrap, T., Lockhart, E., Shanahan, M., Langston, V., Pascanu, R., Botvinick, M., Vinyals, O., Battaglia, P.W.: Deep reinforcement learning with rela- tional inductive biases....

2019

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

In: International Conference on Learning Rep- resentations (2022), https://openreview.net/forum?id=4rLw09TgRw9

Alivanistos, D., Berrendorf, M., Cochez, M., Galkin, M.: Query embedding on hyper-relational knowledge graphs. In: International Conference on Learning Rep- resentations (2022), https://openreview.net/forum?id=4rLw09TgRw9

2022

[2] [2]

In: International Conference on Learning Representations (2021)

Arakelyan, E., Daza, D., Minervini, P., Cochez, M.: Complex query answering with neural link predictors. In: International Conference on Learning Representations (2021)

2021

[3] [3]

In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)

Bacon, P.L., Harb, J., Precup, D.: The option-critic architecture. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)

2017

[4] [4]

In: International Conference on Learning Representations (2015)

Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: International Conference on Learning Representations (2015)

2015

[5] [5]

W3C Recommendation (Feb 2014), https://www.w3.org/TR/rdf11-concepts/, 25 February 2014

Cyganiak, R., Wood, D., Lanthaler, M.: Rdf 1.1 concepts and abstract syntax. W3C Recommendation (Feb 2014), https://www.w3.org/TR/rdf11-concepts/, 25 February 2014

2014

[6] [6]

Journal of Artificial Intelligence Research65, 1–30 (2019)

Francois-Lavet, V., Rabusseau, G., Pineau, J., Ernst, D., Fonteneau, R.: On overfitting and asymptotic bias in batch reinforcement learning with partial ob- servability. Journal of Artificial Intelligence Research65, 1–30 (2019). https: //doi.org/10.1613/jair.1.11478

work page doi:10.1613/jair.1.11478 2019

[7] [7]

In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (2020)

Galkin, M., Trivedi, P., Maheshwari, G., Usbeck, R., Lehmann, J.: Message passing for hyper-relational knowledge graphs. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (2020)

2020

[8] [8]

d’Avila Garcez, A., Dutra, A.R.R., Alonso, E.: Towards symbolic reinforcement learning with common sense (2018), https://arxiv.org/abs/1804.08597

arXiv 2018