{"id":"1ef75e90-5620-472f-ad37-bfdac77460d1","arxiv_id":"2412.18347","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"The Constitutional Filter integrates a neuro-symbolic rule model into Bayesian tracking and learns how much to trust the rules, but its reported improvement is based on in-sample tuning.","lead":"The paper introduces the Constitutional Filter, a particle filter that uses rules written as a probabilistic logic program to help track agents expected to obey those rules. The authors test it on real marine traffic data and report better tracking when the rules are trusted, but the main evidence is weakened by tuning the trust level on the same data used for evaluation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The KDE-based constitutional likelihood is estimated from the filter's own prior, so Eq. 15 double-counts the predictive distribution; the update is not a valid Bayes update and reported gains may be self-reinforcement rather than information.","rationale":"The central claim is that CoFi improves tracking by incorporating a constitutional likelihood in a Bayesian update. The KDE construction in Section IV-D is the sole derivation of that likelihood, and it is the point where the argument is least secure. Drawing samples from the filter's own predictive prior and estimating a density over the resulting satisfaction probabilities cannot produce a likelihood for the current state-measurement pair; it produces a marginal distribution of the satisfaction score under the prior. Multiplying the measurement likelihood by this density in Eq. 15 means the update is influenced by the same predictive distribution it is meant to correct, so the posterior is biased toward the prior and any improvement can be a self-reinforcement artifact. This concern is independent of the trust-ratio leakage: even with a fair train/test split, the update would not be a proper Bayes filter. A synthetic exact-posterior comparison directly tests whether the KDE-based update approximates the true conditional posterior. If it does not, the theoretical foundation and the empirical interpretation both fail.","tokens_in":12634,"tokens_out":7243,"duration_ms":71543,"concrete_test":"Construct a synthetic tracking problem with a known constant-velocity model and a known constitution satisfaction probability q(x,z). Treat C_t as an observed binary variable drawn as C_t ~ Bernoulli(q(x_t,z_t)) and compute the exact posterior p(x_t|c_1:t,z_1:t) by a particle filter with likelihood q(x_t,z_t) if c_t=1 and 1-q(x_t,z_t) if c_t=0. Run CoFi's KDE-based update on the same observations, increasing particle count and KDE sample size. If the KDE-based posterior does not converge in KL divergence or mean absolute error to the exact posterior, Eq. 15 is not a valid Bayes update and the Figure 8 mechanism is unsubstantiated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-D samples x_t^(n) from the filter's own predictive prior p(x_t|c_1:t-1,z_1:t-1) and z_t^(n) from p(z_t|x_t), then runs a KDE on the scalar values P(C_t|x_t^(n),z_t^(n)). As written, this KDE estimates the prior-predictive distribution of the satisfaction probability, not the conditional likelihood p(c_t|x_t,z_t) for a fixed state-measurement pair. The constitution satisfaction event has no observed value c_t; P(C_t|x_t,z_t) is a deterministic function of the state and measurement. A valid likelihood for an unobserved Bernoulli event would be Bernoulli(q(x,z)), not a smooth density over q-values drawn from the prior. Because the KDE evaluates a query particle at a density that is high where the prior predictive was already concentrated, Eq. 15 multiplies the measurement likelihood by a prior-derived factor, double-counting the predictive distribution. No consistency argument shows the resulting distribution equals p(x_t|c_1:t,z_1:t). Setting τ=0 in Eq. 16 removes the constitution, but the claimed benefit is for τ>0, so the self-referential construction remains the load-bearing part of the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Constitutional Filter (CoFi), an extension of recursive Bayesian estimation in which a neuro-symbolic \"Constitution\"—a probabilistic first-order logic program over states, measurements, and environmental features—contributes an extra likelihood factor p(c_t|x_t,z_t) to the belief update (Eq. 15). A scalar trust ratio τ (Eq. 16) blends this constitutional likelihood with a uniform term, with τ=0 recovering a standard particle filter. The method is evaluated on real-world AIS marine traffic data, where τ is calibrated per trust feature (vessel type, waterway binding, anchoring classifier) to maximize tracking accuracy; the paper reports that a trust-calibrated CoFi outperforms a baseline particle filter and recovers baseline performance when τ=0.","tokens_in":12885,"tokens_out":4498,"duration_ms":45419,"significance":"If the central construction were sound, CoFi would be a valuable contribution: it connects probabilistic logic programming and statistical relational maps with recursive state estimation, it is evaluated on real data, and the authors provide an open-source implementation. The interpretability of the Constitution and the idea of learned trust are appealing. However, the paper's core claim—that Eq. 15 is a valid Bayesian update using a constitutional likelihood—rests on an unjustified density-estimation step over the filter's own prior, and the reported performance gains are obtained by calibrating τ on the same data used for evaluation. These are load-bearing issues, not presentation problems.","major_comments":[{"comment":"The constitutional likelihood is not derived as a likelihood. Given a fixed Constitution, P(C_t|x_t,z_t) from Eq. (11) is a deterministic function q(x,z) of the state and measurement. A valid likelihood for the unobserved event c_t=1 would be Bernoulli(q(x,z)), or its density q(x,z)^{c}(1-q(x,z))^{1-c} if c_t were observed. Instead, Section IV-D samples x_t from the filter's own predictive prior p(x_t|c_{1:t-1},z_{1:t-1}) and z_t from p(z_t|x_t), forms S={q(x_t^{(n)},z_t^{(n)})}, and applies KDE. The resulting density, evaluated at q(x_t,z_t), estimates the prior-predictive distribution of q, not the conditional likelihood p(c_t|x_t,z_t). Substituting this quantity into Eq. (15) multiplies the measurement likelihood by a factor derived from the same predictive distribution that the update is supposed to correct. No consistency argument is given, so Eq. (15) is not established as a Bayes update for p(x_t|c_{1:t},z_{1:t}).","section":"IV-D, Eq. (15)"},{"comment":"The trust-ratio interpolation in Eq. (16) is dimensionally and semantically unclear. The paper alternates between treating P(C_t|x_t,z_t) as a probability (Eq. 11) and treating p(c_t|x_t,z_t) as a continuous density over KDE outputs (Section IV-D). The uniform term U(0,1) is a density for a continuous variable on [0,1], not a likelihood for a binary compliance event, and no observed value c_t appears anywhere in the data. Consequently, the mixture p_τ(ct|xt,zt)=τ p(ct|xt,zt)+(1−τ)U(0,1) does not have a clear probabilistic interpretation, and the claim that τ=0 reduces exactly to the standard Bayesian update is not supported by the definitions given.","section":"IV-F, Eq. (16)"},{"comment":"The experimental evaluation is in-sample. Section V-D states that \"CoFi chooses τ to maximize its tracking accuracy for agents with the respective trust features\" and that this is computed as an offline task on historical AIS data. Figure 8 then reports errors at the optimal τ values selected for each vessel group. No training/test split, cross-validation, or held-out evaluation is described. Therefore the central claim of Section V-E—that \"CoFi provides more accurate tracking as soon as τ>0\"—is a statement about in-sample fitting, not a predictive statement about new trajectories. The claim that CoFi 'learns to trust' agents would require evaluation on data not used to choose τ.","section":"V-D, V-E, Figs. 7-8"}],"minor_comments":[{"comment":"The set J over which the sum-product is computed should be defined explicitly as the set of models satisfying the query constitution(X,Y); otherwise the sum over all models would not yield P(C_t|x_t,z_t).","section":"IV-C, Eq. (11)"},{"comment":"The KDE bandwidth and kernel are not specified; the resulting density, and hence the behavior of Eq. (15), may be sensitive to these choices, so a sensitivity analysis or at least the chosen parameters should be reported.","section":"IV-D"},{"comment":"The paper assumes the trust features and the appropriate trust are time-invariant throughout a journey, but one of the trust features is an anchoring classifier that can change over time; this assumption should be justified or relaxed.","section":"V-D"},{"comment":"The y-axis label 'Relative Error' is undefined; the text mentions 'relative mean absolute error' but the reference value (e.g., relative to the particle filter error or to the true position scale) is not stated, and no absolute errors, standard deviations, or sample sizes are reported for the curves.","section":"V-E, Fig. 8"},{"comment":"The notation for the trust ratio is inconsistent: Eq. (16) writes τ(ψ_t) but the surrounding text and experiments use τ without explicitly showing the dependence on ψ for each vessel group.","section":"IV-F, Eq. (16)"}],"recommendation":"reject","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read the CoFi paper with interest. The core idea is genuinely new: coupling hybrid probabilistic logic programs to recursive Bayesian filtering, with a trust ratio that lets the filter down-weight an unreliable constitution. That is a real step beyond the authors' earlier ProMis planning work, and the maritime example is well chosen. The paper is clearly written and they ship code, which I value.\n\nWhat the paper does well: the Constitution encoding of AIS-based rules is concrete and interpretable, the trust ratio is a sensible safeguard, and the runtime analysis is honest about the cost of neuro-symbolic inference. The qualitative results in Figures 5-6 suggest the Constitution does capture real traffic patterns.\n\nThe soft spots are serious, and they land on the central claim. First, the constitutional likelihood in Section IV-D is built by KDE over samples drawn from the filter's own predictive prior. The stress-test note is right: for a fixed (x,z), P(C|x,z) is a deterministic scalar, and the KDE over prior-sampled values estimates a prior-predictive density, not a likelihood for an unobserved satisfaction event. Multiplying the measurement likelihood by this density double-counts the prior and can reinforce the filter's existing belief. A Bernoulli likelihood would be the natural construction, and the paper offers no consistency argument for Eq. 15 as a valid Bayes update. Second, the trust ratio tau is tuned on the same AIS data used in Figure 8, so the accuracy gain over the particle filter is in-sample fitting. Section V-D says \"offline learning,\" but there is no train/test separation; the paper's headline claim - \"more accurate tracking as soon as tau > 0\" - is not a predictive validation. Third, the baseline is a plain particle filter; a map-aware filter using the same spatial relations (e.g., a potential field over depth or distance to waterway) would be a fairer test of whether the logic layer earns its keep.\n\nThese flaws are critical to the empirical claim as stated, but they are fixable. The self-referential likelihood can be replaced with a proper observation model for c_t; the trust calibration can be evaluated on a held-out set; a map-aware baseline would clarify the contribution. The underlying architecture is interesting and the open-source implementation is an asset.\n\nMy recommendation: this deserves a serious referee, not a desk reject. The current version is not ready to be accepted, but the idea is novel enough that a good review cycle, pushing for a corrected likelihood and proper validation, could make this a solid contribution. I'd bring it to a reading group to discuss the KDE issue if nothing else.","headline":"A novel and promising neuro-symbolic filter whose central empirical claim is currently undermined by a self-referential likelihood construction and in-sample trust calibration, but the idea deserves a serious referee.","tokens_in":803,"tokens_out":1790,"would_cite":false,"duration_ms":36108,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new Bayesian filter, CoFi, multiplies the standard measurement update by the probability that an agent satisfies a neuro-symbolic Constitution, and on real marine data this improves tracking accuracy whenever the learned trust ratio is…","keywords":["constitutional filter","Bayesian estimation","neuro-symbolic systems","probabilistic logic programming","particle filter","statistical relational map","marine traffic tracking","trust calibration"],"falsifier":"Run CoFi on a trajectory with known ground truth where the agent systematically violates the Constitution while the process and measurement models are correct; if maximizing tracking accuracy ever selects $\\tau > 0$ and the resulting error exceeds the $\\tau = 0$ particle filter on that trajectory, the claim that CoFi guarantees baseline recovery is contradicted.","tokens_in":12414,"feed_emoji":"🚢","tokens_out":6098,"duration_ms":53919,"temperature":0.7,"pith_summary":"The paper argues that a Bayesian tracker can be made more accurate by consulting an explicit, human-interpretable model of the rules the tracked agent is expected to follow, which it calls a Constitution. It introduces the Constitutional Filter (CoFi), which multiplies the usual measurement likelihood by a constitutional likelihood: the probability that the agent's state and sensor reading satisfy rules expressed in probabilistic first-order logic. On real marine traffic data, CoFi tracks vessels more accurately than a standard particle filter whenever a learned trust ratio is positive, and it degrades exactly to the baseline when the trust ratio is zero. The practical payoff is that expert knowledge about laws, physical limits, and preferences can be injected into any recursive Bayesian estimator without losing the safety net of the rule-free filter.","feed_headline":"Trust-calibrated filter beats particle filter on ship data","feed_subtitle":"Neuro-symbolic 'Constitution' guides Bayesian tracking; when rules fail, τ=0 restores the baseline.","key_machinery":"The load-bearing object is the constitutional likelihood $p(c_t|x_t,z_t)$, obtained in three steps: (i) grounding a deep probabilistic first-order logic program (the Constitution, consisting of background knowledge, perception, and a StaR Map environment model) at a candidate state and measurement; (ii) computing the satisfaction probability via the sum-product of Eq. 11; and (iii) turning the discrete satisfaction values into a continuous density by kernel density estimation over samples from the filter's predictive prior. This density enters Eq. 15 as an additional multiplicative factor in the Bayes update, and Eq. 16 modulates it with the trust ratio $\\tau$ so that $\\tau = 0$ returns the standard filter.","core_discovery":"The central discovery is a belief update (Eq. 15) in which the posterior is proportional to the product of the measurement likelihood, the standard predictive prior, and a constitutional likelihood $p(c_t|x_t,z_t)$ computed from a neuro-symbolic program. The Constitution encodes the agent's expected compliance as probabilistic first-order logic clauses over spatial relations from a StaR Map, background knowledge, and perceptual features; exact sum-product inference yields the probability that a state-measurement pair satisfies it. Because this probability is a discrete number, CoFi converts it into a density by kernel density estimation over samples drawn from the filter's own prior, and then blends it with a uniform distribution through a trust ratio $\\tau$ (Eq. 16). The paper demonstrates on AIS vessel data that a trust-calibrated CoFi improves mean absolute tracking error as soon as $\\tau > 0$, while $\\tau = 0$ exactly recovers the unconstitutional particle filter, so an inaccurate Constitution cannot degrade performance.","pith_inferences":["The same construction should transfer to road traffic, pedestrian crowds, or airspace rules: any domain where a probabilistic first-order logic program can separate likely from unlikely states could show similar gains, with the StaR Map replaced by any uncertain semantic map.","The KDE step in Section IV-D samples from the filter's own predictive prior, so the resulting 'likelihood' is partly a function of the belief it is meant to correct; a testable consistency check would compare CoFi's posterior with one using an independent Monte Carlo estimate of $p(c_t|x_t,z_t)$.","Trust features could be learned online per agent instead of calibrated offline on historical data, letting CoFi adapt when an agent's compliance changes mid-trajectory, something the current time-invariant trust setting does not address.","If the Constitution is interpreted as a prior over states rather than a likelihood, Eq. 15 resembles a product-of-experts update; exploring that reading could clarify when the KDE approximation is valid and yield a more principled normalization."],"forward_implications":["Any Bayes filter, whether particle, Kalman, or unscented, can host CoFi, because the Constitution step only adds one extra likelihood factor to the update.","For the roughly 89% of vessels in the dataset whose optimal trust ratio is positive, CoFi reduces tracking error; the remaining vessels fall back to the particle filter baseline.","Precomputing the constitutional likelihood as a static scalar field brings CoFi's per-update runtime close to the baseline particle filter (0.007 s versus 0.004 s in the experiments).","The Constitution is a symbolic, human-readable artifact, so the reasoning that guides tracking can be inspected and edited by a domain expert.","A false or ill-matched Constitution cannot hurt tracking below baseline, because trust calibration is allowed to set $\\tau = 0$."],"supporting_citations":[{"why":"Defines the recursive Bayesian estimation framework that CoFi extends with its Constitution step.","marker":"[12]"},{"why":"Provides the StaR Map environment representation whose uncertain spatial relations supply the distribution parameters for the Constitution's ground atoms.","marker":"[35]"},{"why":"Supplies the hybrid probabilistic logic program formulation of a Constitution that CoFi adapts from mission planning to state estimation.","marker":"[2]"},{"why":"Exemplifies the neural-probabilistic logic programming paradigm that makes Constitutions learnable and end-to-end differentiable.","marker":"[7]"},{"why":"Supplies the kernel density estimation method used to convert discrete constitutional satisfaction probabilities into a likelihood density.","marker":"[41]"},{"why":"The knowledge compiler used to evaluate the sum-product inference in Eq. 11 efficiently.","marker":"[39]"},{"why":"Source of the real-world AIS vessel traffic data used to evaluate CoFi's tracking accuracy.","marker":"[43]"}],"fun_headline_variants":["CoFi: learns when to trust its rules for better ship tracking","Neuro-symbolic filter adapts rule trust to beat particle filter","Bayesian filter with Constitution recovers when rules are wrong","Trust-aware CoFi improves marine tracking, matches baseline when wrong"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The update treats a kernel density estimate built from the filter's own predictive samples as a genuine likelihood for an unobserved 'constitution satisfied' event, even though those samples come from the very distribution the update is meant to correct.","fun_headline_variants_meta":{"raw":{"variants":["CoFi: learns when to trust its rules for better ship tracking","Neuro-symbolic filter adapts rule trust to beat particle filter","Bayesian filter with Constitution recovers when rules are wrong","Trust-aware CoFi improves marine tracking, matches baseline when wrong"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00074,"raw_usage":{"total_tokens":3293,"prompt_tokens":923,"completion_tokens":2370,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":2298}},"tokens_in":539,"tokens_out":2370,"duration_ms":16784,"temperature":1.0,"reasoning_tokens":2298,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:48:00.435375+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CoFi on a trajectory with known ground truth where the agent systematically violates the Constitution while the process and measurement models are correct; if maximizing tracking accuracy ever selects $\\tau > 0$ and the resulting error exceeds the $\\tau = 0$ particle filter on that trajectory, the claim that CoFi guarantees baseline recovery is contradicted.","supporting_citations":[{"cited_title":"Thrun, W","cited_arxiv_id":null,"evidence_quote":"Defines the recursive Bayesian estimation framework that CoFi extends with its Constitution step."},{"cited_title":"Star maps: Unveiling uncertainty in geospatial relations,","cited_arxiv_id":null,"evidence_quote":"Provides the StaR Map environment representation whose uncertain spatial relations supply the distribution parameters for the Constitution's ground atoms."},{"cited_title":"Mission design for unmanned aerial vehicles using hybrid probabilistic logic programs,","cited_arxiv_id":null,"evidence_quote":"Supplies the hybrid probabilistic logic program formulation of a Constitution that CoFi adapts from mission planning to state estimation."},{"cited_title":"DeepProbLog: Neural probabilistic logic programming,","cited_arxiv_id":null,"evidence_quote":"Exemplifies the neural-probabilistic logic programming paradigm that makes Constitutions learnable and end-to-end differentiable."},{"cited_title":"D sharp: Fast d-DNNF compilation with sharpSAT,","cited_arxiv_id":null,"evidence_quote":"The knowledge compiler used to evaluate the sum-product inference in Eq. 11 efficiently."},{"cited_title":"Marine Cadastre Vessel Traffic,","cited_arxiv_id":null,"evidence_quote":"Source of the real-world AIS vessel traffic data used to evaluate CoFi's tracking accuracy."}],"review_version":1}