{"id":"0ee461ac-2cc2-4a47-a279-b9405b7c7cdc","arxiv_id":"2501.11896","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Rel-SAR, a vector-symbolic architecture with numeric, circular, and boolean vectors, improves accuracy on Raven's Progressive Matrices, particularly for position-based rules.","lead":"This paper describes a neuro-symbolic AI system called Rel-SAR that combines neural perception with high-dimensional vector math to solve Raven's Progressive Matrices, a classic visual reasoning test. It reports strong accuracy on the hardest puzzle configurations, although its performance relies on supervision from ground-truth rule labels and its out-of-distribution generalization is mixed.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The simplified logical relation function (Eq. 7) cannot express XOR, and the paper's unverified claim that RAVEN only uses Arithmetic+/Arithmetic- logical rules is load-bearing: if any Position rule is XOR, the reported 96.5%/98.0% accuracy would drop.","rationale":"The paper's central contributions are the diverse relation representations and the unified numerical/logical relation functions. The numerical function (Eq. 6) covers all additive RAVEN rules, and the experimental numbers are strong. However, the logical function is explicitly incomplete: the full version (Eq. 29) is described only in the appendix and is needed for XOR. The authors' justification is a single sentence claiming RAVEN contains only Arithmetic+ and Arithmetic- logical rules. This claim is empirical, easy to verify from the dataset's rule annotations, and not accompanied by any evidence in the paper. If the claim is false, the model would score near chance on every XOR-ruled problem, which would pull the reported averages down substantially, particularly on configurations like Out-InGrid and 2x2Grid where position rules occur. The concern is not about a disagreement with consensus; it is about an internally testable assumption about the benchmark itself. The test I propose settles it by counting rules and measuring per-rule accuracy. Given that the reader already conditioned on this assumption, my stress-test agrees and recommends keeping the CONDITIONAL verdict until the dataset statistics are checked.","tokens_in":22461,"tokens_out":7455,"duration_ms":73896,"concrete_test":"Download the official RAVEN and I-RAVEN datasets (or their generation code) and parse the per-sample rule label dictionaries; count how many samples use each logical operation (AND, OR, XOR, DIFF) for the Position/Existence attribute. If the count of XOR (or AND) is nonzero, run the released Rel-SAR code on that subset, or implement Equations 7-9 with ground-truth attribute vectors, and measure accuracy on just those problems. If accuracy is near chance (e.g., <10%) while the overall average is ~96%, the model does not cover the dataset's rule vocabulary, and the central claim is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix A states that the simplified logical relation function RLgc (Eq. 7, M=3) cannot express XOR, and asserts that 'the RAVEN dataset only involve Arithmetic+ and Arithmetic- logical rules,' so Table IV assigns only OR (Arithmetic+) and DIFF (Arithmetic-) to the Position attribute. This is the single load-bearing assumption for the reported accuracy: the reasoning backend either cannot represent XOR at all, or would need the full function (Eq. 29) that is never used in the experiments. The paper provides no per-rule-type accuracy, no dataset rule statistics, and no code, so there is no evidence that the logical rule vocabulary of RAVEN/I-RAVEN is limited to {OR, DIFF}. If an XOR Position rule exists in the 70,000 samples, the model's rule abduction cannot produce the correct rule and execution fails, directly contradicting both the systematicity claim and the headline accuracies. The reader's conditional verdict hinges on this same point; the concern is concrete and falsifiable by inspecting the dataset annotations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Rel-SAR, a neuro-symbolic model for Raven's Progressive Matrices (RPM) built on vector-symbolic architectures (VSA). It introduces four types of atomic high-dimensional vectors (random, numeric, circular, boolean), a structured high-dimensional representation (SHDR) for image panels and grid components, and numerical and logical relation functions that support rule abduction and execution. The model uses a ResNet-50 perception frontend to predict panel SHDRs and a reasoning backend that abducts rule parameters (operator powers OP and output r) via relation functions, executes them with inverse relation functions, and selects the answer with highest similarity. Experiments report 96.5% average accuracy on RAVEN and 98.0% on I-RAVEN under end-to-end training, as well as 99.2% on I-RAVEN when using attribute labels in a reasoning-only evaluation. The paper also reports attribute-value and OOD generalization experiments.","tokens_in":22710,"tokens_out":3605,"duration_ms":38769,"significance":"If the results hold, Rel-SAR is a meaningful step toward interpretable and systematic abstract visual reasoning: it substantially improves over prior VSA-based neuro-symbolic models on position-dependent configurations, and its perception frontend achieves near-perfect attribute extraction. The algebraic formulation of relation functions (Eqs. 6-9 and the full logical version in Appendix A) is internally consistent, and the experimental comparisons are broad. However, the central claim of systematic abductive reasoning is qualified by two issues: the main end-to-end results are obtained with ground-truth rule labels, and the simplified logical relation function cannot represent XOR, with no evidence that RAVEN lacks such rules. The reported OOD generalization is also less robust than the abstract suggests.","major_comments":[{"comment":"The main end-to-end comparison in Table VI uses rule labels through the losses Lop (Eq. 23) and Lr (Eq. 24), where the rule learners regress directly to ground-truth operator powers OP_gt and relation outputs r_gt. This supervision is disclosed in Section IV-F.1 but is not flagged in the abstract or in Section V-B, where the results are described simply as 'end-to-end learning.' Because the rule abduction module is therefore not autonomously deriving rules from the context panels alone, the high accuracy in Table VI largely reflects this extra supervision. Please report a variant trained without rule labels (for example, using only answer labels and the Lrs objective of Eq. 27) and clearly state in the abstract and results section what supervision each reported accuracy uses.","section":"IV-F.1, Eqs. (23)-(24), Table VI"},{"comment":"The simplified logical relation function RLgc (Eq. 7) cannot express XOR, and the paper asserts that 'the RAVEN dataset only involve Arithmetic+ and Arithmetic- logical rules' without providing per-rule statistics or dataset annotations. This assumption is load-bearing because the reported accuracy on position-dependent configurations (3x3Grid, Out-InGrid) would drop if any Position rule were XOR, as the reasoning backend could not represent or abduct it. The paper already defines the full logical relation function that can express XOR (Eq. 29) but never uses it in experiments. Please either provide dataset rule statistics demonstrating that XOR and other non-simplified rules are absent, or run the experiments with the full logical relation function.","section":"Appendix A, Eq. (7), Table IV"},{"comment":"The abstract claims 'robust out-of-distribution generalization,' but Table XI shows that Rel-SAR's OOD accuracy is markedly below that of Learn-VRF and ARLC on the Color attribute (e.g., Color Distribute Three: 70.7% vs. 100% and 99.8%). The paper itself acknowledges this limitation in the conclusion, but the abstract's unconditional claim is too strong. Please either temper the abstract's claim or add analysis explaining why Color fails and whether this is inherent to the relation-function representation or a property of the rule learner.","section":"Abstract; Section V-D, Table XI"}],"minor_comments":[{"comment":"There is a typo: 'high-dimentional' should be 'high-dimensional.'","section":"Abstract"},{"comment":"The word 'sytematic' in the final section should be 'systematic.'","section":"Conclusion"},{"comment":"The configuration name is inconsistently written as 'Out-InGrid' in the text and 'O-InGrid' in several table headers, figures, and the caption of Table X; please use one notation consistently.","section":"Throughout"},{"comment":"The notation v(◦opi)i is easy to misread as exponentiation; consider adding a sentence clarifying that the superscript denotes repeated binding (op_i times).","section":"IV-B.1, Eq. (6)"},{"comment":"The OR operation is defined using AND and XOR; a short derivation or reference would help readers verify that this implements Boolean OR in the vector domain.","section":"Table II"},{"comment":"No code or data are provided, which limits reproducibility of the reported numbers; the authors should consider releasing the implementation and dataset rule statistics.","section":"V"}],"recommendation":"major_revision","confidential_remarks":"The paper is not deceptive: Section IV-F.1 and the conclusion explicitly state that rule labels are used during training. The main issue is framing: the abstract and the comparison with other end-to-end methods obscure this supervision, making the central 'systematic abductive reasoning' claim appear stronger than the evidence supports. The XOR assumption is the other load-bearing point, and it is empirically checkable from the dataset. Also, no code or per-rule statistics are provided, which makes independent verification difficult. These concerns are fixable within the scope of a revision, so the recommendation is major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on Rel-SAR. The paper delivers a real advance in neuro-symbolic RPM: a unified VSA framework with numeric, circular, and boolean atomic vectors and two relation functions that cover the rule types in RAVEN. The headline numbers are strong—96.5% on RAVEN and 98.0% on I-RAVEN, with a 39-point jump over NVSA on 3x3Grid and 21 points on Out-InGrid. The architecture is clean, interpretable, and the perception frontend reaches 99.99% panel accuracy. The reasoning backend, when given attribute labels, hits 99.2% on I-RAVEN, beating ARLC. This is not just a repackaging of known primitives; the integration and the position-rule handling are genuinely new and effective.\n\nThe soft spots are mostly about the claims, not the mechanics. First, the rule abduction stage is trained with ground-truth rule labels via Lop and Lr (Eq. 23–24). The model is not abducting rules from the images; it is supervised to output the rule labels. That is fine as a system design, but the abstract's \"systematic abductive reasoning\" overstates what is demonstrated. Second, Appendix A admits the simplified logical relation function cannot express XOR and asserts RAVEN only uses Arithmetic+ and Arithmetic- for Position. That is plausible—I believe the RAVEN generator only uses union/difference for the Position Arithmetic rule—but the paper provides no dataset statistics or per-rule-type accuracy to back it. It is load-bearing: if any Position rule were XOR, the model would fail on it. This should be verified. Third, the OOD generalization tables (Table XI) show weak transfer to Color, so \"robust OOD generalization\" in the abstract is too strong; the conclusion actually acknowledges this. Fourth, no code or error bars, which is minor but limits verification.\n\nBottom line: this paper deserves a serious referee. The core framework is a legitimate contribution, and the position-rule gains are substantial. A revision should clarify the supervised nature of rule abduction, verify the rule-vocabulary assumption, temper the OOD claim, and release code. I'd bring it to the reading group and would cite it.","headline":"A genuinely better VSA-based RPM solver with a real win on position rules, but the 'systematic abduction' claim rests on supervised rule labels and an unverified assumption about RAVEN's rule vocabulary.","tokens_in":23224,"tokens_out":2918,"would_cite":true,"duration_ms":27749,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Rel-SAR shows that a vector-symbolic architecture using numeric, circular, and boolean high-dimensional codes can solve Raven's Progressive Matrices at 96.5% average accuracy while keeping abduction and execution interpretable.","keywords":["abstract visual reasoning","Raven's Progressive Matrices","vector-symbolic architecture","hyperdimensional computing","relation representation","rule abduction","neuro-symbolic reasoning","out-of-distribution generalization"],"falsifier":"Construct an RPM-style test where object existence follows the XOR rule ($v_3 = v_1 \\oplus v_2$) and train Rel-SAR end-to-end on it. The paper's own Appendix A states that the simplified logical relation function cannot represent XOR, so if the model fails on this rule, the claimed systematicity over logical rules is disproved.","tokens_in":22273,"feed_emoji":"🧩","tokens_out":7806,"duration_ms":78651,"temperature":0.7,"pith_summary":"This paper claims that abstract visual reasoning in Raven's Progressive Matrices can be carried out by a vector-symbolic architecture in which every attribute value is represented as a high-dimensional vector with explicit numeric, periodic, or logical semantics, and rules are abducted and executed by two general relation functions. The proposed model, Rel-SAR, reaches 96.5% average accuracy on RAVEN and 98.0% on I-RAVEN without attribute labels, on par with the best monolithic deep networks, while improving over the prior neuro-symbolic model NVSA by 8.8 and 9.9 points on average. The largest gains come on grid configurations where rules act on object positions, up 39.1 points on 3x3Grid and 20.8 points on Out-InGrid. If the approach is right, interpretable symbolic reasoning and high perceptual accuracy do not have to be traded off, and the same algebraically grounded vector code can support perception, rule finding, and rule execution in one mechanism.","feed_headline":"Vector-symbolic reasoner hits 96.5% on Raven puzzles","feed_subtitle":"Diverse numeric, circular, and boolean vector codes crack position rules that stumped earlier neuro-symbolic models.","key_machinery":"The load-bearing machinery is the pairing of four atomic vector types with two relation functions. Numeric vectors, built by fractional power encoding, make real addition correspond to vector binding; circular vectors add periodicity, which is exactly what cyclic grid shifts need; boolean vectors implement NOT, AND, OR, and (in the full version) XOR through binding and similarity operations. The structured high-dimensional representation (SHDR) is a role-filler binding of position vectors and attribute vectors that lets a whole image panel be encoded in one vector and then decomposed by unbinding. The numerical relation function $r_{\\mathrm{Num}} = \\bigcirc_{i=1}^{N} v_i^{\\circ \\mathrm{op}_i}$ and the logical relation function $r_{\\mathrm{Lgc}} = (\\mathrm{op}_1 v_1 \\wedge \\mathrm{op}_2 v_2) \\circ \\mathrm{op}_3 v_3$ take several attribute vectors and operator powers as input and return an output vector; rule abduction searches for one parameter setting that makes all relation outputs agree, while rule execution uses the inverse functions to predict the missing panel. A 5-operator full logical version in Appendix A covers XOR, though RAVEN itself only needs the simplified form for its Arithmetic+ and Arithmetic- position rules.","core_discovery":"Rel-SAR is a neuro-symbolic system that learns a structured high-dimensional representation (SHDR) of each image panel through VSA binding and bundling: a ResNet-50 predicts the SHDR, which is then decomposed into per-object attribute vectors by unbinding. A query-attention step re-embeds the noisy neural estimates into backend codebooks whose vectors carry algebraic semantics: numeric vectors for type, size, color, and number; circular vectors for grid positions; boolean vectors for object existence. Two relation functions are the rule representation: the numerical function $R_{\\mathrm{Num}}(v_{1:N},\\mathrm{OP}_{1:M}) = \\bigcirc_{i=1}^{N} v_i^{\\circ \\mathrm{op}_i}$ and the simplified logical function $R_{\\mathrm{Lgc}}(v_{1:N},\\mathrm{OP}_{1:M}) = (\\mathrm{op}_1 v_1 \\wedge \\mathrm{op}_2 v_2) \\circ \\mathrm{op}_3 v_3$. Rule abduction finds operator powers $\\mathrm{OP}_{1:M}$ that make all pairwise row and column relation outputs agree, and rule execution inverts the same functions to predict the missing panel. The paper reports that this unified scheme reaches 96.5% on RAVEN and 98.0% on I-RAVEN, and that with ground-truth attribute labels its reasoning backend scores 99.2% on I-RAVEN, 6.8 points above ARLC. The central claim is that systematicity comes from diverse relation representations rather than from a lookup table of per-rule templates.","pith_inferences":["A natural extension beyond RPM is to other relational reasoning benchmarks such as Bongard problems or visual analogy tasks, where the full logical relation function from Appendix A would be needed to handle XOR-style rules that RAVEN does not contain.","Because the numerical relation function only expresses additive relations, a testable extension is to represent attribute values in a logarithmic FPE basis so that vector binding encodes multiplication and ratios, which would let the same architecture handle proportion-style rules.","The reported limits on Color out-of-distribution transfer could be addressed by normalizing attribute-value ranges before FPE encoding, so that a rule learned on a narrow range like Type transfers to a wider range like Color.","The query-attention bridge between task-independent frontend codebooks and algebraically meaningful backend codebooks suggests a general recipe: train perception against random codebooks, then re-express the result in a structured codebook, a pattern that could apply to other neuro-symbolic systems."],"forward_implications":["Because the same relation functions are used across attributes, a rule learned for one attribute can transfer to another: the paper reports near-perfect out-of-distribution transfer for Type and Size rules on I-RAVEN.","Position-dependent configurations stop being the weak spot of vector-symbolic solvers: 3x3Grid rises from 57.1% to 96.2% and Out-InGrid from 65.4% to 86.2% on RAVEN compared with NVSA.","End-to-end training without attribute labels is sufficient: Rel-SAR's accuracy stays nearly identical when attribute labels are removed, whereas NVSA and ALANS degrade substantially.","With ground-truth attribute labels, the reasoning backend reaches 99.2% on I-RAVEN, surpassing prior VSA-based rule-learning models ARLC and Learn-VRF.","The abducted rules are explicit operator-power and output-vector pairs, so the final answer choice can be inspected as a series of vector operations."],"supporting_citations":[{"why":"Defines NVSA, the neuro-vector-symbolic baseline and direct inspiration whose rule templates and accuracy Rel-SAR is compared against.","marker":"[12]"},{"why":"Introduces the RAVEN dataset, including its seven configurations and the Constant, Progression, Arithmetic, and Distribute Three rules that the relation functions are built to cover.","marker":"[21]"},{"why":"Introduces I-RAVEN and its attribute bisection tree answer generation, the harder test set used for evaluation.","marker":"[8]"},{"why":"Supplies the FHRR and HRR vector models whose binding, bundling, inverse, and unbinding operations carry all the algebra.","marker":"[36]"},{"why":"Provides fractional power encoding, the construction that maps real numbers to numeric vectors so that addition becomes binding.","marker":"[19]"},{"why":"Gives the VSA survey that grounds the structured symbolic representations and role-filler binding used in the SHDR.","marker":"[20]"},{"why":"The ALANS learner is the closest neuro-symbolic comparison with learnable reasoning operators, and its attribute-label dependence is contrasted with Rel-SAR.","marker":"[13]"},{"why":"Learn-VRF supplies the VSA rule-learning baseline whose out-of-distribution rule-transfer setup Rel-SAR adopts and outperforms in reasoning-backend tests.","marker":"[31]"},{"why":"ARLC is the VSA rule-template model that Rel-SAR beats by 6.8 points on I-RAVEN with ground-truth attribute labels.","marker":"[14]"}],"fun_headline_variants":["Vector-symbolic AI cracks Raven puzzles with diverse relation codes","Interpretable neuro-symbolic model hits 96.5% on Raven's matrices","Diverse vector representations unlock systematic abductive reasoning","Rel-SAR: vector-symbolic reasoner reaches 98% on I-RAVEN","From panels to rules: Rel-SAR solves Raven with interpretable logic"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model only works if every rule in the puzzle can be expressed by adding together vectors of a few fixed types; the authors acknowledge in Appendix A that the simplified logical function cannot express XOR, so any rule of that form lies outside the model's reach.","fun_headline_variants_meta":{"raw":{"variants":["Vector-symbolic AI cracks Raven puzzles with diverse relation codes","Interpretable neuro-symbolic model hits 96.5% on Raven's matrices","Diverse vector representations unlock systematic abductive reasoning","Rel-SAR: vector-symbolic reasoner reaches 98% on I-RAVEN","From panels to rules: Rel-SAR solves Raven with interpretable logic"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000277,"raw_usage":{"total_tokens":1707,"prompt_tokens":1059,"completion_tokens":648,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":675,"completion_tokens_details":{"reasoning_tokens":551}},"tokens_in":675,"tokens_out":648,"duration_ms":6643,"temperature":1.0,"reasoning_tokens":551,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:44:26.330265+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct an RPM-style test where object existence follows the XOR rule ($v_3 = v_1 \\oplus v_2$) and train Rel-SAR end-to-end on it. The paper's own Appendix A states that the simplified logical relation function cannot represent XOR, so if the model fails on this rule, the claimed systematicity over logical rules is disproved.","supporting_citations":[{"cited_title":"A neuro-vector-symbolic architecture for solving raven’s progressive matrices,","cited_arxiv_id":null,"evidence_quote":"Defines NVSA, the neuro-vector-symbolic baseline and direct inspiration whose rule templates and accuracy Rel-SAR is compared against."},{"cited_title":"Raven: A dataset for relational and analogical visual reasoning,","cited_arxiv_id":null,"evidence_quote":"Introduces the RAVEN dataset, including its seven configurations and the Constant, Progression, Arithmetic, and Distribute Three rules that the relation functions are built to cover."},{"cited_title":"Stratified rule-aware network for abstract visual reasoning,","cited_arxiv_id":null,"evidence_quote":"Introduces I-RAVEN and its attribute bisection tree answer generation, the harder test set used for evaluation."},{"cited_title":"Com- puting on functions using randomized vector representations","cited_arxiv_id":null,"evidence_quote":"Provides fractional power encoding, the construction that maps real numbers to numeric vectors so that addition becomes binding."},{"cited_title":"A survey on hyperdimensional computing aka vector symbolic architectures, part i: Models and data transformations,","cited_arxiv_id":null,"evidence_quote":"Gives the VSA survey that grounds the structured symbolic representations and role-filler binding used in the SHDR."},{"cited_title":"Learning algebraic representation for systematic generalization in abstract reason- ing,","cited_arxiv_id":null,"evidence_quote":"The ALANS learner is the closest neuro-symbolic comparison with learnable reasoning operators, and its attribute-label dependence is contrasted with Rel-SAR."},{"cited_title":"Towards learning abductive reasoning using vsa distributed representations,","cited_arxiv_id":null,"evidence_quote":"ARLC is the VSA rule-template model that Rel-SAR beats by 6.8 points on I-RAVEN with ground-truth attribute labels."}],"review_version":1}