{"id":"11fbba68-ba01-46de-b124-e85cdce3d5f5","arxiv_id":"2412.13023","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A new relational neurosymbolic Markov model class enforces logical constraints in sequential deep probabilistic models, trained with a differentiable particle filter.","lead":"This paper introduces a new kind of AI model that combines deep learning with logical rules to handle sequences, guaranteeing that the rules are never broken. The model and its training algorithm scale to longer sequences and handle out-of-distribution cases better than previous neurosymbolic systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The cluster-factorisation independence claimed in Eqs. (8)-(9) is unproved and is false when a single shared observation such as 'hit' couples multiple enemies, so the 'exact' local inference in the RBPF is actually approximate and the scalability/guarantee claims are overstated.","rationale":"The central claim of the paper is not merely that NeSy-MMs define a model class; it is that the proposed Rao-Blackwellised particle filter can compute the needed conditional probabilities 'exactly' per cluster, thereby making end-to-end learning feasible where exact NeSy inference would not scale. Equation (9) is the only place in Section 4 where this exactness is established. The passage around Eq. (8) is presented as a general property of conditional distributions ('can be factorised'), but no independence proof is offered and the 'maximal number of clusters' is never algorithmically defined. In the discriminative benchmark the observation is a global hit variable combining attacks from all enemies, so the conditional distribution over enemy locations is exactly the kind of explaining-away posterior where a product over enemies is wrong. Appendix C relies on the same exact conditional in Eqs. (16)-(17) to argue that the Log-Derivative/RLOO gradients can be computed by automatic differentiation; an approximate conditional makes those gradients biased and weakens the 'unbiased approximate gradient' statement as applied. This is a correctness risk, not merely a disagreement with standard practice: the paper itself labels the local computation exact, and the method's scalability advantage over existing NeSy systems depends on keeping B larger than 1. I would not reject the paper: the model class and experiments are plausible, and the issue could be repaired by proving a concrete conditional-independence condition under which Eq. (9) holds, or by explicitly reframing the cluster factorisation as an approximation and studying its bias. As it stands, however, the manuscript overstates the exactness of its inference and therefore the strength of its guarantees. The secondary concern that the printed NetHack rules omit wall constraints is real and worth fixing, but it is less central than the cluster-factorisation gap.","tokens_in":14,"tokens_out":8299,"duration_ms":149537,"concrete_test":"On the discriminative model with two enemies on a 3x3 grid, fix the agent at the center and give each enemy a uniform prior over the 8 surrounding cells. Condition on the single observation hit=true at time t=1, enumerate the exact joint posterior over the two enemy cells, and compare it with the product of the two per-enemy marginal posteriors. If the total variation distance or KL divergence is nonzero, Eq. (9) is false for the deployed cluster choice. As a second check, instrument the implementation to compute p(X_{t+1} | x_t, Z_{t+1}) both with separate enemy clusters (B=2) and with one joint cluster (B=1); any discrepancy confirms that the code is using an approximation where the paper claims exactness.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is Eq. (8)-(9): p(X_{t+1} | x_t, Z_{t+1}) is assumed to factor over clusters, and the paper asserts without proof that this is the 'maximal' cluster decomposition. In general, conditioning on a common observation creates dependence, so the factorization is exact only when Z decomposes per cluster without a shared factor. In the discriminative experiment, Z_t is a single binary variable 'hit' that is a disjunction over all enemies: p(hit_t | E^1_t, ..., E^K_t, agent_t) is one joint factor. After conditioning on hit_t = true, the enemy positions E^i are coupled via explaining away. Concretely, with two enemies each independently at adjacent cell A or non-adjacent cell B, prior uniform, the exact posterior satisfies P(E1=A,E2=A | hit) = 1/3, while the product of marginals is 4/9; hence Eq. (9) is not a theorem. Either the factorization is an approximation, contradicting the paper's 'exact local inference' wording, or B must be 1 whenever observations couple all enemies, in which case the claimed scalability over exact NeSy inference disappears. The recursive gradient construction in Appendix C (Eqs. 16-17) also relies on p(X_T | x_{T-1}, Z_T) being exact, so the error propagates into the RLOO gradient estimator used for learning.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces relational neurosymbolic Markov models (NeSy-MMs), a class of sequential probabilistic models that combine Markov dynamics with relational logical constraints and neural parameterizations. The model is defined by the factorization in Eq. (3), and inference is performed with a Rao-Blackwellised particle filter that conditions on observations through exact local NeSy inference, aided by a cluster factorization of the state space. The authors also propose a gradient-estimation scheme based on RLOO and report experiments on generative image-sequence modeling and discriminative trajectory classification, claiming strong guarantees, scalability beyond existing NeSy systems, and improved out-of-distribution generalization.","tokens_in":19930,"tokens_out":7489,"duration_ms":66853,"significance":"If the technical claims are correct, this is a valuable step toward sequential neurosymbolic models: it addresses a real scalability bottleneck, supports both discriminative and generative tasks, and demonstrates that relational constraints can be injected into deep Markov models. The paper also contributes two new benchmarks, reports hyperparameters and seeds, and states that code is provided, which are strengths for reproducibility. However, the central guarantees of exactness and unbiased learning currently rest on an unstated conditional-independence assumption and on a recursive factorization that is not generally valid, so the significance is contingent on fixing or carefully qualifying these points.","major_comments":[{"comment":"The factorization p(X | Z) = ∏_i p(X_i | Z) is presented as an equality without stating the required conditional-independence assumption. In general, conditioning on a shared observation couples the variables. In the discriminative experiment (§5.2, Appendix D.2), the observation hit(T) is a logical disjunction over all enemies, so conditioning on hit=true induces dependence among enemy locations via explaining away. Concretely, with two enemies each independently at adjacent cell A or non-adjacent cell B, prior uniform, and hit = (E1=A or E2=A), the exact posterior satisfies P(E1=A, E2=A | hit) = 1/3, while the product of marginals is 4/9. Thus Eq. (9) is not a theorem in this setting. Either the factorization is an approximation, which would contradict the 'exact' wording in §4.1 and the gradient derivation in Appendix C, or the method applies only when observations decompose per cluster, which would exclude the main discriminative experiment. The authors need to state the assumption, prove it for their settings, or explicitly label the factorization as approximate and adjust the guarantee claims accordingly.","section":"§4.2, Eqs. (8)-(9)"},{"comment":"The notion of 'maximal number of clusters' B is undefined, and no procedure is given for computing the clusters or for verifying that the factors in Eq. (10) are well-defined conditional distributions. Even if one takes the clusters to be connected components of some factor graph of p(X | Z), variables within a cluster need not become independent after conditioning on Z, so the refined factorization p(F_i, I_i | x_t, Z) = p(F_i | I_i, x_t, Z) p(I_i | x_t, Z) is always true, but the cross-cluster product in Eq. (9) is the nontrivial step. The paper should specify the cluster-identification algorithm and prove the factorization under explicit conditions, or state that the cluster product is an approximation.","section":"§4.2, Eq. (10)"},{"comment":"The recursive factorization of p(x_{0:T} | Z_{0:T}) in Eq. (16) is not generally correct. The equation writes p(x_{0:T} | Z_{0:T}) = p(x_T | x_{T-1}, Z_{t+1}) p(x_{0:T-1} | Z_{0:T-1}), but the second factor is the filtering distribution at time T-1, not the conditional distribution of the past given the full observation sequence including Z_T. The past states are generally dependent on future observations. For example, with T=1, deterministic transition x_1 = x_0, and observation z_1 = x_1, p(x_0 | z_0, z_1) differs from p(x_0 | z_0), so the product on the right-hand side is not the joint posterior. Consequently, Eq. (17), which computes log p(x_{0:T} | Z_{0:T}) as a sum of per-step proposal conditionals p(x_t | x_{t-1}, Z_t), substitutes the proposal log-density for the target log-density. The RLOO estimator in Eq. (15) then is not unbiased for the objective in Eq. (11). The authors need to derive the gradient estimator with proper importance weighting or explicitly present the method as an approximate, biased estimator.","section":"Appendix C, Eqs. (16)-(17)"},{"comment":"The logic program for the generative task does not encode wall constraints. Section 5.1 states that the agent moves in a grid 'surrounded by walls', and the claims in §5.3 say that NeSy-MMs 'perfectly adhere to the mechanics of the game' and 'provably satisfy' constraints. However, the rules in Appendix D.2 move the agent by one cell for each action with no check that the target cell is inside the grid or is not a wall. As printed, the program permits moves that walk through or outside the walls. The authors should either provide the actual program including wall checks, or qualify the guarantee claim to the movement rules as written rather than the full environment mechanics.","section":"Appendix D.2 and §5.1"}],"minor_comments":[{"comment":"Equation (10) is missing a comma in the conditioning set: it should read p(F_i | I_i, x_t, Z_{t+1}), not p(F_i | I_i x_t, Z_{t+1}).","section":"§4.2, Eq. (10)"},{"comment":"The first factor in Eq. (16) uses the time index Z_{t+1} although the current time is T; this should be Z_T, which would at least make the notation internally consistent.","section":"Appendix C, Eq. (16)"},{"comment":"The text says the RBPF assumes p(X_{t+1} | X_t, Z_{t+1}) 'can be computed exactly' and later describes the results as 'exact local inference'. Given the cluster-factorization issue in §4.2, these statements should be qualified to avoid overclaiming.","section":"§4.1"},{"comment":"The class-balance percentages in Table 1 are useful, but the paper does not state how the agent death label is determined in the generated data; adding one sentence on the death condition would improve reproducibility.","section":"§5.2 and Table 1"}],"recommendation":"major_revision","confidential_remarks":"The technical issues are internal and fixable within the manuscript's scope: the cluster factorization needs an explicit assumption or an approximation label, and the gradient-estimation derivation needs either a correct target log-density or an honest approximate treatment. The empirical results are interesting, but the advertised guarantees of exactness and unbiasedness are currently not supported. I would not recommend rejection because the core model and experimental setup are promising; however, the paper should not be accepted until these load-bearing points are resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things about arXiv:2412.13023. First, the core model class is genuinely new and worth building on: NeSy-MMs are the first sequential deep probabilistic models that put relational logic into the state transition and observation, and the Rao-Blackwellised particle filter that avoids resampling is a neat trick. Second, the paper's headline guarantee—'provably satisfy' relational constraints with 'exact' local inference—is not supported by the math as written. Eq. (8) factorizes the conditional p(X|Z) as a product over clusters without stating the conditional-independence assumption. In their own discriminative experiment, Z_t is a single binary 'hit' that is a disjunction over all enemies. Conditioning on hit couples the enemy locations via explaining away. A simple two-enemy example gives the exact posterior P(E1=A,E2=A|hit)=1/3 while the product of marginals is 4/9. So Eq. (9) is not a theorem. The clusters are only independent if Z decomposes per cluster; otherwise either B=1 (no scalability) or the factorization is an approximation. The paper does not acknowledge this. Since the gradient estimator in Appendix C relies on p(X_T | x_{T-1}, Z_T) being exact, the bias propagates into learning.\n\nThe generative experiment has a related omission: the logic program in Appendix D.2 defines the four movement rules but no wall constraints, even though the task is a walled grid and the paper claims 'perfectly adhere to the mechanics of the game.' Either the model cannot represent walls, or the printed program is incomplete. That is a reproducibility gap.\n\nWhat the paper does well: the experimental setup is thorough—five seeds, hyperparameter search, dataset generator, runtime reporting—and the OOD gains from relational structure are real and interesting. The comparison to VAEL and Deep-HMM is thin but tolerable given that no existing NeSy system scales to these horizons. The benchmarks themselves are a contribution.\n\nBottom line: this deserves a serious referee, but the referee should push hard on the theoretical claim. The model class and experiments can stand as an approximate method; the authors need to either prove the factorization under stated conditions or present the method as approximate and quantify the error. As written, the guarantee claim is overstated.","headline":"Genuinely new model class with a clever RBPF, but the advertised 'exact' inference rests on an unproven and likely false cluster factorization; the paper needs major revision before its guarantees are credible.","tokens_in":20429,"tokens_out":3250,"would_cite":true,"duration_ms":30566,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new class of Markov models provably satisfies relational logical constraints in sequential tasks.","keywords":["neurosymbolic AI","hidden Markov models","relational logic","Rao-Blackwellised particle filter","differentiable inference","probabilistic logic programming","sequential generative models","out-of-distribution generalisation"],"falsifier":"Take the discriminative task's two-enemy setting and compute the exact joint posterior over the two enemies' locations after observing a hit. If conditioning on the hit couples the enemies, then the cluster product in Eq. (8) will not hold exactly and the filter's 'exact' local inference is approximate; the paper does not report such a check.","tokens_in":19421,"feed_emoji":"🧩","tokens_out":7458,"duration_ms":66327,"temperature":0.7,"pith_summary":"The paper introduces relational neurosymbolic Markov models (NeSy-MMs), sequential probabilistic models in which each hidden state has a subsymbolic neural part and a symbolic relational-logic part, and transitions can be logical, neural, or a mixture. The central claim is that this factorisation lets a single end-to-end differentiable model guarantee relational logical constraints, such as safety or game rules, at every time step of a sequence. To make the idea practical, the paper develops a differentiable Rao-Blackwellised particle filter that replaces resampling with exact inference over the finite symbolic variables and feeds those exact probabilities into unbiased gradient estimators. The experiments show the approach generating rule-following image sequences, classifying trajectories with partially unknown enemy behaviour, imposing new constraints at test time, and generalising out of distribution better than transformer and deep-HMM baselines.","feed_headline":"A new Markov model provably obeys logical rules in sequences","feed_subtitle":"Rao-Blackwellised particle filtering makes neural learning compatible with relational logic at long horizons.","key_machinery":"The load-bearing object is the factorised transition in Eq. (3), written as $p_\\varphi(S_0\\mid N_0)p(N_0)p(Z_0\\mid S_0)\\prod_t p_\\varphi(S_{t+1}\\mid S_t,N_{t+1})p(N_{t+1})p(Z_{t+1}\\mid S_{t+1})$, with $N_t$ the neural/subsymbolic part, $S_t$ the relational symbolic part, and $Z_t$ observations. Inference rests on a Rao-Blackwellised particle filter: rather than resampling, the filter computes the conditional $p_\\varphi(X_{t+1}\\mid x_t,Z_{t+1})$ exactly when the state is finite, and splits that conditional into independent clusters by Eq. (8), a factorisation into a product over clusters that are claimed to be conditionally independent given the observation. Exact finite probabilities make unbiased discrete gradient estimators such as RLOO applicable, while infinite-domain variables are handled by differentiable particle filtering.","core_discovery":"The paper's central claim is that the Markovian factorisation over neurosymbolic states $X_t=(N_t,S_t)$, with transition $p_\\varphi(S_{t+1}\\mid S_t,N_{t+1})$ governed by relational logic and neural predicates, is the first deep sequential probabilistic model that integrates relational logical constraints and provably satisfies them. Under this factorisation, relations constrain both the inside of one time slice and the movement from one slice to the next, so a property such as \"the agent is safe\" can be guaranteed throughout the whole trajectory. The paper further claims that its Rao-Blackwellised particle filter, combined with cluster factorisation and state-of-the-art gradient estimation, scales inference and learning to time horizons well beyond those reachable by existing exact and approximate neurosymbolic systems, while still training neural components end to end.","pith_inferences":["Editorial: if the cluster-factorisation independence in Eq. (8) is not exact for a given observation model, then what the paper calls exact local inference is an approximation; an explicit check on the discriminative game task would settle this for that benchmark.","Editorial: the same construction could be applied to neural language generation, where a relational symbolic state tracking predicates over generated tokens would let a decoder guarantee global logical constraints, provided the cluster independence holds for the constraints used.","Editorial: one testable extension is to replace exact cluster inference with approximate inference for the continuous parts, trading the exactness guarantee for applicability to relational dynamics with many continuous variables."],"forward_implications":["Logical constraints such as safety properties can be guaranteed at every time step of a generated or classified sequence, not just in a single static inference.","NeSy-MMs handle both generative tasks, such as producing image sequences that follow rules and actions, and discriminative tasks, such as classifying trajectories with partially unknown neural transitions.","New constraints can be imposed at test time without retraining, so a model can refuse to enter a forbidden region mid-trajectory while still following the rest of the instructions.","The relational symbolic state gives better out-of-distribution generalisation than transformer and deep-HMM baselines in the paper's discriminative experiments, and scales to sequence lengths where exact neurosymbolic baselines time out."],"supporting_citations":[{"why":"Supplies the Rao-Blackwellised particle filter scheme that lets the paper replace non-differentiable resampling with exact local conditional computation.","marker":"Murphy and Russell 2001"},{"why":"Provides exact probabilistic inference machinery used for the finite symbolic parts of the NeSy-MM state.","marker":"Kisa et al. 2014"},{"why":"Gives the RLOO estimator used to obtain unbiased gradients of the discrete symbolic probabilities.","marker":"Kool, van Hoof, and Welling 2019"},{"why":"Contributes the CatLog-derivative trick for differentiable sampling of categorical distributions, supporting the discrete gradient-estimation step.","marker":"De Smet, Sansone, and Zuidberg Dos Martires 2023"},{"why":"Provides differentiable particle filtering without modifying the forward pass, used for gradient estimation over infinite-domain variables.","marker":"Scibior, Masrani, and Wood 2021"},{"why":"Defines the discrete-continuous probabilistic logic programming semantics that the NeSy-MM factorisation and running examples use.","marker":"De Smet et al. 2023"},{"why":"Introduces the variational-autoencoder logic baseline and the Mario-style generative setting the paper extends.","marker":"Misino, Marra, and Sansone 2022"},{"why":"Supplies the MiniHack grid-world environment from which the generative trajectories are built.","marker":"Samvelyan et al. 2021"}],"fun_headline_variants":["NeSy Markov models: provable logic, long horizons","Rao-Blackwellised NeSy: logical sequences that scale","Neural+relational: guaranteed rules in sequences","Provably logical Markov models for long sequences","End-to-end differentiable NeSy with constraint guarantees"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole inference scheme assumes that, once the observations are fixed, the symbolic state splits into conditionally independent clusters, so each cluster can be solved separately with no error.","fun_headline_variants_meta":{"raw":{"variants":["NeSy Markov models: provable logic, long horizons","Rao-Blackwellised NeSy: logical sequences that scale","Neural+relational: guaranteed rules in sequences","Provably logical Markov models for long sequences","End-to-end differentiable NeSy with constraint guarantees"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000154,"raw_usage":{"total_tokens":1178,"prompt_tokens":879,"completion_tokens":299,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":222}},"tokens_in":495,"tokens_out":299,"duration_ms":3486,"temperature":1.0,"reasoning_tokens":222,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:30:50.722954+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the discriminative task's two-enemy setting and compute the exact joint posterior over the two enemies' locations after observing a hit. If conditioning on the hit couples the enemies, then the cluster product in Eq. (8) will not hold exactly and the filter's 'exact' local inference is approximate; the paper does not report such a check.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Rao-Blackwellised particle filter scheme that lets the paper replace non-differentiable resampling with exact local conditional computation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides exact probabilistic inference machinery used for the finite symbolic parts of the NeSy-MM state."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the RLOO estimator used to obtain unbiased gradients of the discrete symbolic probabilities."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the variational-autoencoder logic baseline and the Mario-style generative setting the paper extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MiniHack grid-world environment from which the generative trajectories are built."}],"review_version":1}