{"id":"976a3d32-e280-4165-a401-17654bab9ad9","arxiv_id":"2607.12711","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Adding MaxSAT-based conflict feedback to a vision-language model increases solved Sudoku instances across all tested models, most notably GPT-5.5 in full-board mode (45 to 73 solved).","lead":"The paper tests a hybrid setup where a vision-language model (VLM) proposes Sudoku moves and a MaxSAT solver checks which proposed moves can coexist without breaking Sudoku rules, feeding conflicts back to the model. Across three VLMs and 200 puzzles, this feedback raises the solve rate, most strongly when the model proposes an entire board at once.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Formal MaxSAT encoding omits puzzle givens and accepted placements as hard clauses, so 'formally verified' may not mean the solved board matches the puzzle.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing gap: the formal model does not specify that puzzle givens or accepted placements become hard clauses. This is the most serious issue because it directly threatens the validity of the 'solved' metric and the central claim of formal verification. Without givens as hard clauses, the MaxSAT oracle could certify a board unrelated to the presented puzzle. The concern is internal inconsistency/omission, not a disagreement with external consensus. It is addressable—the authors may have included givens in their implementation—but the manuscript as written does not support the claim. Because the issue is fixable and does not necessarily invalidate the empirical trend, the verdict should remain CONDITIONAL, matching the reader's assessment. I therefore recommend no change to the reader's verdict.","tokens_in":10742,"tokens_out":3478,"duration_ms":36304,"concrete_test":"Instrument the implementation (or, if code is unavailable, re-implement from the specification) and, for a small sample of puzzles, print the exact hard-clause set Φ_h. Check that for every given clue (r,c,d) the unit clause X_{r,c,d} is present in Φ_h, and that in iterative mode each accepted placement is added as a hard clause before the subsequent MaxSAT call. Then run a controlled comparison on 20 puzzles (10 easy, 10 hard) in full-board mode, once with givens hard-coded and once omitting them, and compare solved counts and average completeness. If the hard set lacks givens or the two runs differ materially, the published formalization is incomplete and the empirical claim is not established by the manuscript as written.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that MaxSAT feedback improves logical consistency, robustness, and solution quality—depends on the oracle verifying candidate placements against the actual Sudoku puzzle. However, the formalization in Sections 3–4 never states that the puzzle's given clues are encoded as hard clauses. Section 3 defines hard clauses only for cell, row, column, and subgrid constraints; Section 4 says Φ_h contains 'the hard clauses encoding all Sudoku constraints' and that each VLM placement is added as a soft unit clause. No mention is made of unit hard clauses for the givens. Consequently, the MaxSAT solver can return any valid 9×9 board—not necessarily the unique solution of the presented puzzle. In full-board mode, a board that satisfies generic Sudoku rules but disagrees with the givens would be counted as 'solved' if the implementation follows the text, inflating solve rates and making the ground-truth-based AC metric ambiguous. Additionally, iterative mode says accepted placements are 'permanently added to the board state' but does not specify that they become hard clauses; if they do not, later acceptances could contradict earlier ones, violating the stated guarantee that 'every accepted placement is formally verified.' This is not a mere presentation issue: if givens are not enforced, the empirical results do not demonstrate solving the given puzzles, and the central claim collapses.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a neuro-symbolic Sudoku-solving loop in which a VLM reads a Sudoku image and proposes placements, and a partial MaxSAT oracle treats those placements as soft unit clauses while Sudoku constraints are hard. The solver returns a largest mutually consistent subset; conflicts are translated into textual/visual feedback, and the loop repeats in either step-by-step (single placement per iteration) or full-board (complete board per iteration) mode. The authors evaluate GPT-5.5, Molmo2, and Qwen3-VL on 200 puzzles (100 difficulty 0 and 100 difficulty 1), comparing validity-only feedback with MaxSAT feedback. They report improvements in solved instances and average completeness, with the strongest gains for GPT-5.5 in full-board mode (45 to 73 solved; AC 72.0% to 80.7%). The central claim is that integrating partial MaxSAT into the VLM loop significantly improves logical consistency, robustness, and solution quality.","tokens_in":11015,"tokens_out":5659,"duration_ms":61722,"significance":"If the results hold, the paper provides a useful controlled demonstration of how a formal oracle can refine VLM-generated structured outputs, with a clean division of labor: the VLM proposes, the MaxSAT oracle validates and provides corrective feedback. The fixed dataset split, multiple VLMs, and comparison against validity-only feedback are strengths. The paper does not overclaim competitiveness with symbolic solvers. However, the formal verification guarantee is underspecified: the encoding as written omits puzzle givens and does not state that accepted placements become hard clauses, which directly affects whether the results measure solving the presented puzzles. In addition, the empirical claims are based on point estimates without uncertainty quantification. These issues are load-bearing but appear fixable in revision.","major_comments":[{"comment":"The formal encoding omits the puzzle givens. Section 3 defines hard clauses only for cell, row, column, and subgrid constraints; Section 4 states that Φ_h contains 'the hard clauses encoding all Sudoku constraints' but never states that each given digit is encoded as a unit hard clause. Without this, the MaxSAT oracle can certify a valid 9×9 Sudoku grid that is not the unique solution of the presented puzzle. Consequently, 'every accepted placement is formally verified' is only verification against generic Sudoku rules, not against the actual puzzle, and the solved/AC metrics in Table 1 become ambiguous. Please state explicitly, and ensure in the implementation, that the givens are hard clauses.","section":"§3, §4.2"},{"comment":"In iterative mode, an accepted placement is 'permanently added to the board state' but the manuscript never says it is added to Φ_h. If it is not, later MaxSAT optima may violate an earlier accepted placement, because soft unit clauses can be dropped when a new conflicting placement is proposed. The guarantee that 'every accepted placement is formally verified' is then only per-step and not cumulative. Specify whether accepted placements become hard clauses; if they do not, the formal verification claim is false.","section":"§4.1"},{"comment":"All conclusions rest on point estimates without uncertainty quantification. For example, the full-board MaxSAT vs validity-only comparison for Qwen3-VL is 13 vs 10 solved puzzles out of 200; the paper reports no confidence intervals, standard errors, or hypothesis tests. The phrase 'significantly improves' in Section 5.5 is therefore not supported statistically. Add confidence intervals or significance tests (e.g., McNemar's test for paired solve counts), or soften the causal claims to observed improvements.","section":"§5.4, Table 1"},{"comment":"The metric 'solved' is never defined in Section 5.4. If 'solved' means average completeness of 100%, then a MaxSAT output that satisfies only the generic Sudoku constraints but disagrees with the givens will not be counted as solved; however, the termination condition in Section 4.2 treats any 'complete and logically consistent solution' as success. This mismatch needs to be resolved, and the definition of 'solved' should be stated explicitly (e.g., full match to the ground-truth solution).","section":"§4.2, §5.4"}],"minor_comments":[{"comment":"The subgrid constraint formula appears to be missing summation notation or is poorly rendered; please fix the typesetting so the at-most-one/at-least-one encoding is unambiguous.","section":"§3"},{"comment":"Qwen3-VL is cited to reference [31], which is a Qwen3 Technical Report; please verify that this is the correct citation for the VLM variant used.","section":"§5.3"},{"comment":"The validity-only feedback prompt says 'Your placement is logically incorrect' but does not explain how validity is checked. If it uses the same MaxSAT oracle or a lighter local check, this should be stated to make the comparison meaningful.","section":"Appendix A"},{"comment":"The statement that the MaxSAT solver solves all instances 'in a few seconds' would benefit from a precise average or maximum runtime, especially since the paper does not report computational overhead of the feedback loop.","section":"§5.2"}],"recommendation":"major_revision","confidential_remarks":"The central formal gap is potentially fixable if the implementation already includes givens and accepted placements as hard clauses; the authors must state this explicitly and ideally provide an algorithm box or code. If the implementation does not enforce givens, the empirical results cannot be interpreted as solving the presented puzzles. The lack of statistical tests also needs attention. In its current form, the paper should not be accepted, but a careful revision addressing the formal encoding and uncertainty quantification could make it publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the quick take: the idea is a good one and the empirical gains look plausible, but the paper hasn't actually specified the formal model it relies on. The MaxSAT oracle is supposed to verify VLM proposals against the puzzle, yet the hard clauses are defined only for the generic Sudoku rules—cells, rows, columns, subgrids. The given clues never appear as hard units, and the iterative loop doesn't say whether accepted placements are added to the hard set. Without that, 'formally verified' can't mean what the paper says it means.\n\nWhat's new and worth noticing: the partial-MaxSAT-as-feedback setup, where the oracle returns the largest consistent subset of the VLM's proposed moves, is a clean template. It's not in the cited prior work, which mostly targets text-based LLM+solver pipelines or MaxSAT for code repair. The evaluation is systematic across three VLMs, two interaction modes, and 200 puzzles, and the pattern is consistent: every model improves on both solve rate and completeness with MaxSAT feedback. GPT-5.5 going from 45 to 73 solved in full-board mode is a big jump.\n\nThe soft spots are real. The biggest is the encoding gap I already mentioned. I think the solve-rate numbers are probably not inflated—since the final board is compared to ground truth, returning a different valid board would fail the AC check—but the feedback signals could be wrong if the oracle doesn't know the givens, and the paper's core guarantee is unverified. The authors need to state explicitly that givens are hard clauses and that accepted placements in step-by-step mode become hard units, or the claim should be softened. Second, the baseline is weak: validity-only feedback says 'your placement is logically incorrect' with no conflict information, while MaxSAT feedback names the conflicting cells and highlights them visually. Part of the gain could just be more informative feedback, not specifically the MaxSAT subset selection. A control with random or generic conflict hints would separate those. Third, there's no code or exact rendering details, so independent replication is hard. Temperature zero makes error bars unnecessary, but I'd still like a few details on image preprocessing and how the board state is represented.\n\nThis is written for people working on neuro-symbolic VLM systems who want a quick way to inject hard constraints into a perception loop. It's a reasonable contribution to that subfield.\n\nI'd send it to peer review, but I'd ask for a revised manuscript that fixes the formal encoding, adds the control, and releases the artifacts. The idea is useful enough that it deserves the feedback.","headline":"Strong empirical template with a core formal gap: the paper never says the puzzle's givens are hard clauses, so the 'formally verified' claim is not established.","tokens_in":11510,"tokens_out":5308,"would_cite":false,"duration_ms":51391,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A partial MaxSAT oracle that returns the largest mutually consistent subset of a vision-language model's Sudoku placements lifts solved puzzles from 45 to 73 for the strongest model, and improves every tested model.","keywords":["MaxSAT","Partial MaxSAT","Neuro-symbolic AI","Vision-language model","Sudoku","Logical consistency","Constraint satisfaction","Feedback refinement"],"falsifier":"Inspect the input to the MaxSAT solver for a specific puzzle: if the initial clues are not present as hard unit clauses, the solver will return a valid Sudoku board that can disagree with the image's givens; a single such accepted board would refute the claim that every accepted placement is formally verified. A re-run of the full-board experiment with the clues deliberately excluded from the hard clauses would show this directly.","tokens_in":10595,"feed_emoji":"🧩","tokens_out":8024,"duration_ms":71741,"temperature":0.7,"pith_summary":"The paper tries to establish that a formal constraint-optimization layer can repair the structured outputs of vision-language models. The authors encode Sudoku rules as hard clauses and each VLM-proposed digit placement as a soft clause in a partial MaxSAT instance; each round, the solver keeps the largest consistent portion of the model's placements and turns the rejected ones into textual and visual feedback for the next attempt. On 200 puzzles across three VLMs, every model improves on both solved instances and average completeness when feedback is MaxSAT-based rather than validity-only, with the strongest result being the full-board mode of the best-performing model, which rises from 45 to 73 solved puzzles and from 72.0% to 80.7% completeness. If this is right, a cheap symbolic oracle can turn near-miss neural predictions into formally certified solutions while keeping the VLM as the primary reasoning agent.","feed_headline":"MaxSAT feedback lifts Sudoku solves from 45 to 73","feed_subtitle":"Adding a MaxSAT consistency oracle improves every tested vision-language model's solved puzzles and completeness.","key_machinery":"The load-bearing object is the partial MaxSAT formulation. Hard clauses encode the generic Sudoku constraints (each cell exactly one digit, each digit once per row, column, and 3x3 subgrid), and soft unit clauses encode the VLM's proposed placements. The MaxSAT solver returns an assignment that satisfies all hard clauses and violates as few soft clauses as possible; the violated soft clauses point to a minimal set of conflicting placements, which become the corrective feedback. This turns a single binary validity check into an optimisation that identifies the most plausible consistent subset and tells the model where it went wrong.","core_discovery":"The central claim is that integrating partial MaxSAT into the VLM solving loop significantly improves logical consistency, robustness, and solution quality. The mechanism is a neuro-symbolic interaction loop: the VLM proposes placements, the MaxSAT oracle finds an assignment that satisfies all Sudoku constraints while maximising agreement with the proposals, and the placements it rejects are converted into structured feedback. In full-board mode, the oracle extracts the largest mutually consistent subset of the proposed complete board. The empirical evaluation shows that every tested VLM improves under MaxSAT feedback in both step-by-step and full-board modes, with the largest gain when the","pith_inferences":["The paper never states that the puzzle's given clues are encoded as hard clauses in the MaxSAT formulation; if they are not, the oracle could certify a valid Sudoku board that disagrees with the image. This is an omitted implementation detail that is load-bearing for the claim that every accepted placement is formally verified.","A testable extension is to feed the same loop different structured puzzles—Kakuro, KenKen, or nonograms—whose rules are also easily expressed as hard clauses; the predicted pattern is the same: large gains when the VLM is near-consistent, smaller gains when it is far off.","Because the symbolic layer is cheap, a natural next step is to ask the VLM for several alternative full-board proposals per round and let the MaxSAT oracle pick the best consistent subset, increasing the chance of finding a complete solution per API call."],"forward_implications":["The gain is achieved purely at inference time; no VLM retraining or fine-tuning is required, so the approach can be bolted onto any existing model that can output structured placements.","Full-board refinement outperforms step-by-step interaction, indicating that a globally structured but imperfect prediction is easier for MaxSAT to repair than a sequence of single placements.","MaxSAT feedback shrinks the difficulty gap: for the strongest model, difficulty-1 solved instances more than double (from 10 to 21 in full-board mode), and average completeness on hard puzzles rises from 62.2% to 76.7%.","The effectiveness of the oracle depends on the quality of the initial proposal: the proprietary model, which is already closer to global consistency, benefits far more than the open-source models, suggesting that symbolic refinement amplifies existing competence rather than replacing it."],"fun_headline_variants":["MaxSAT oracle sharpens VLM Sudoku accuracy","Neuro-symbolic boost: MaxSAT guides VLMs in Sudoku","VLM Sudoku gets logical consistency via MaxSAT feedback","Symbolic refiner steers VLMs to more solved Sudoku boards","Consistency checker improves vision-language Sudoku solving"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise, left unstated in Sections 3 and 4, is that the puzzle's given clues and any already-accepted placements are encoded as hard clauses in the MaxSAT formulation; the text only documents the generic cell/row/column/subgrid constraints as hard, so without the givens the oracle could certify a valid Sudoku board that does not match the puzzle.","fun_headline_variants_meta":{"raw":{"variants":["MaxSAT oracle sharpens VLM Sudoku accuracy","Neuro-symbolic boost: MaxSAT guides VLMs in Sudoku","VLM Sudoku gets logical consistency via MaxSAT feedback","Symbolic refiner steers VLMs to more solved Sudoku boards","Consistency checker improves vision-language Sudoku solving"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000609,"raw_usage":{"total_tokens":2651,"prompt_tokens":700,"completion_tokens":1951,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":444,"completion_tokens_details":{"reasoning_tokens":1881}},"tokens_in":444,"tokens_out":1951,"duration_ms":11589,"temperature":1.0,"reasoning_tokens":1881,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T06:20:33.479355+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the input to the MaxSAT solver for a specific puzzle: if the initial clues are not present as hard unit clauses, the solver will return a valid Sudoku board that can disagree with the image's givens; a single such accepted board would refute the claim that every accepted placement is formally verified. A re-run of the full-board experiment with the clues deliberately excluded from the hard clauses would show this directly.","supporting_citations":[],"review_version":2}