{"id":"1301ebc4-1814-417f-8277-34877a576895","arxiv_id":"2412.01937","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"The paper reports that A*, evolutionary, and reinforcement-learning search can solve 3D hypercube sliding puzzles, and that the stochastic methods find approximate solutions in dimensions 4 and 5 where A* times out.","lead":"This paper benchmarks three search methods, A*, an evolutionary algorithm, and reinforcement learning, on a higher-dimensional sliding puzzle played on a hypercube. It reports that A* solves puzzles in dimensions 3 and 4 but times out in dimension 5, while the stochastic methods return approximate solutions with far fewer moves.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's 'median less than 30' claim is contradicted by the paper's own Table 5: RL on d=5, k=4 reports median 161 moves.","rationale":"Read in good faith, the paper's contribution is an empirical benchmark showing approximate methods can handle hypercube sliding puzzles where exact A* fails. The quantitative anchor of that claim is the abstract's median-under-30 bound. Table 5 contradicts that bound for the only d=5, k=4 RL row, and Table 4 gives no EA solution for the same row; this is an internal inconsistency, not a matter of outside consensus. A separate mechanics-level issue exists: Eqs. (3)-(4) define h as a mismatch count, making fa(C,T)=1/(1+L-h) maximized away from the target and Algorithm 2's stopping criterion fa=1 unreachable for L>0. If the code uses the corrected objective, that issue is a text-level bug; the median contradiction, however, is in the reported experimental record. The reader's REJECT verdict remains appropriate, though the reader's weakest_assumption (EA objective) is not the same as the abstract-contradiction concern I identify, hence partial agreement.","tokens_in":19815,"tokens_out":6552,"duration_ms":257296,"concrete_test":"Compute the median of the raw d=5, k=4, level 0 RL move counts from the public repository (https://github.com/lemerleau/CubicPuzzle.git) or rerun the RL benchmark with the stated settings (P=100000 initial states, 1000 iterations, 150 runs). If the median remains about 161, the abstract's 'less than 30' statement is false as a general claim and must be qualified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that once A* fails at higher dimension, 'RL and EA methods can still provide a generally acceptable solution, i.e. a distribution of a number of moves with a median value of less than 30.' Appendix F, Table 5, d=5, l=26, k=4, level 0, reports RL success 100%, min 62, max 316, median 161. That is more than five times the advertised 30-move bound, so the headline generalization is internally contradicted by the authors' own data. Table 4 also reports EA success 0% for that same puzzle, so the 'RL and EA methods' plural cannot be supported at k=4. Because the contradiction is in the reported results rather than in a tunable assumption, it cannot be repaired by code cleanup or by clarifying the EA objective; the abstract needs either a narrower scope or a corrected number.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies a higher-dimensional sliding puzzle on the d-dimensional hypercube with the k-rule move constraint. It proposes three search methods—A*, an evolutionary algorithm (EA), and a reinforcement learning (RL) agent—and benchmarks them on puzzles of dimensions d=3, 4, and 5 across several difficulty levels. The main reported conclusions are that A* solves the d=3 and most d=4 puzzles optimally, while for larger dimensions EA and RL still return approximate solutions, with the EA being fastest and RL being more accurate. The paper also provides a CPU-time comparison and claims that the methods go beyond the theoretical reachability results of prior work.","tokens_in":19974,"tokens_out":6849,"duration_ms":69131,"significance":"If the claims held, the paper would provide a useful practical benchmark showing that stochastic search can handle hypercube sliding puzzles beyond the reach of exact search. The authors ship a public code repository, report experiments over multiple dimensions, face dimensions, and difficulty levels, and provide CPU-time comparisons; these are genuine strengths. However, the central empirical and algorithmic claims are not currently supported: the EA objective function as written is inconsistent with the reported stopping behavior, the RL pseudocode is not executable as printed, and the paper's headline claim about median moves under 30 is contradicted by its own Table 5. Because these issues affect the core results, the contribution cannot be accepted in its present form.","major_comments":[{"comment":"The fitness function in Eq. (3)-(4) is internally inconsistent with the stated stopping criterion. h(C,T) is defined as the number of mismatches (δ=1 when C_i≠T_i), so L−h is the number of matches. The fitness fa=1/(1+L−h) is therefore maximized when h=0, but its maximum value is 1/(1+L), not 1. Conversely, fa=1 only when h=L, i.e., when every ring is mismatched. This directly contradicts Algorithm 2 line 4, which stops when fa(C_b,T)=1, and Appendix A, which equates the maximum fitness with hamming distance 0. The EA results in Table 4 therefore cannot be produced by the algorithm as described unless the implemented objective differs from the printed one; the equations and the experimental code must be reconciled and the experiments rerun.","section":"§3.3, Eq. (3)-(4), Algorithm 2"},{"comment":"The RL pseudocode is not executable as written. Line 4 tests `n(T) < N`, but `n` is never defined and `N` is not an input parameter (the input is the branching parameter `P`). Line 17 contains the loop condition `c < 0`, which is never true because c starts at 0 and is only incremented; the surrounding text in §3.2 says episodes terminate when more steps have been taken than the best known path, which would require a condition such as `c < o` rather than `c < 0`. As printed, the algorithm either loops forever or terminates immediately, so the RL results in Table 5 are not reproducible from the stated method.","section":"Appendix D, Algorithm 4"},{"comment":"The abstract's central claim that, when the dimension increases, \"RL and EA methods can still provide a generally acceptable solution, i.e. a distribution of a number of moves with a median value of less than 30\" is directly contradicted by the paper's own data. Appendix F, Table 5, for d=5, l=26, k=4, level 0, reports RL success 100% with min 62, max 316, and median 161 moves. Table 4 reports an EA success rate of 0% for the same puzzle. Thus the reported median is more than five times the advertised bound, and the plural \"RL and EA methods\" is not supported for this case. The headline claim must be corrected or substantially narrowed.","section":"Abstract vs. Tables 4 and 5"},{"comment":"For d=4, k=1 and k=2, the EA performance table (Table 4) and the RL performance table (Table 5) report identical min, max, and median values for every difficulty level, including entries such as level 1: min 8, max 60, median 14, and level 4: min 12, max 28, median 16. Since these are independent stochastic algorithms run over 150 trials, identical summary statistics across all levels are not plausible unless the same data were used for both tables. This data-consistency problem must be resolved before the EA-vs-RL comparison in §4.2 can be trusted.","section":"Tables 4 and 5"}],"minor_comments":[{"comment":"The comments in Algorithm 2 refer to \"the length of the target RNA structure,\" \"base-pair distance,\" and a Lévy or Binomial distribution, which appear to be copied from an RNA-folding EA. These should be replaced with puzzle-specific terminology.","section":"Algorithm 2 comments"},{"comment":"The proof states the complexity is \"at least O((2^k−1)^n)\", but the following sentence says \"the complexity of A* search for a face dimension k=4 is at most O(7^n)\"; the latter should be \"at least\" or \"of order\" to be consistent.","section":"§4.3, Proposition 1"},{"comment":"The caption of Figure 3 says the puzzle parameters are k=2, d=3, l=4, but panel (b) shows d=4; the caption should describe both panels accurately.","section":"Figure 3 caption"},{"comment":"The sentence beginning \"Among the three algorithms ... EA and RL techniques are ˚a both choose\" contains a typographical error that obscures the intended meaning.","section":"§4.2"},{"comment":"The stopping criteria in Appendix A state that the hamming distance of the best agent to the target is 1, whereas the main text (§3.3) states the stopping condition is hamming distance 0; these should be made consistent.","section":"Appendix A"}],"recommendation":"reject","confidential_remarks":"The four major issues above are load-bearing: the EA objective is inconsistent with its stopping rule, the RL algorithm cannot run as printed, the abstract's headline median claim is contradicted by Table 5, and the EA and RL tables appear to contain identical data for several d=4 rows. The last point, in particular, warrants an editorial check against the provided repository. Should the authors correct the algorithm descriptions, reconcile the tables, and rerun the affected experiments, a resubmission could be considered, but the current manuscript does not support its central conclusions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: the benchmark idea is legitimate and new—nobody has run A*, EA, and RL on this [4] puzzle family before—and the comparison across d=3,4,5 with code on GitHub is potentially useful. The qualitative story (A* optimal but fails at scale; EA fast but poor at minimizing; RL slower but more accurate) is plausible. But as written the paper doesn't support its own conclusions.\n\nFirst, the EA objective is inverted. Eq. (3)-(4) define h as a count of mismatches and then set fa=1/(1+L-h). That function is maximized when h=L, i.e. when nothing matches, and fa=1 is unreachable for L>0. If the code follows the equations, the stopping condition can never fire; if it follows a corrected objective, then the experiments don't test the algorithm the paper describes.\n\nSecond, Algorithm 4 line 17 is not executable: 'while C != T and c < 0 and s(C) is not undefined'—c starts at 0, so the loop is never entered. The surrounding pseudocode also uses undefined functions. That makes the RL results unreproducible from the text.\n\nThird, the abstract's headline claim—RL and EA provide solutions with median under 30 when A* fails—is directly contradicted by Table 5: d=5, k=4, RL success 100% but median 161. And Table 4 shows EA success 0% for that same puzzle, so the plural claim fails too.\n\nFourth, the A* monotonicity argument in Section 3.1 is not a proof; saying 'since d(C,C') is precisely the number of moves' just restates the question. The heuristic is not shown to be monotone, so the optimality guarantee is unsupported.\n\nFifth, Tables 4 and 5 contain identical RL and EA rows for d=4, k=1 and k=2 at every level. That suggests a data-handling or table-generation error. The GitHub repository might clear this up, but the preprint alone doesn't.\n\nI don't see derivation-based circularity and the self-citation to [4] is appropriate. But these are load-bearing mechanical errors. I'd not desk-reject the subject matter: the puzzle benchmark could be a useful reference. But this version should not go to referees. I'd send it back to the authors with instructions to fix the objective, the pseudocode, the summary stats, and the table duplication, then resubmit.","headline":"A first benchmark of search methods on the hypercube k-rule sliding puzzle, but the preprint's own equations, pseudocode, and tables contradict the reported results.","tokens_in":20537,"tokens_out":4393,"would_cite":false,"duration_ms":44735,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On the paper's own account, exact A* search solves dimension-3 and most dimension-4 hypercube sliding puzzles, while evolutionary and reinforcement-learning searches still solve dimension-5 instances with median move counts usually below…","keywords":["higher-dimensional sliding puzzle","hypercube","k-rule","A* search","evolutionary algorithm","reinforcement learning","puzzle diameter","NP-hard search"],"falsifier":"Run the released EA with the objective exactly as printed in Eqs. (3)-(4): if h(C,T) is the number of mismatched ring-target pairs, then $f_a(C,T)=1/(1+L-h)$ reaches its maximum 1 at h=L, i.e. at a configuration where every ring is in the wrong place, so the loop 'while $f_a \\neq 1$' should never stop for any nontrivial puzzle. If the code still solves puzzles, the experiments are not testing the algorithm described; if it does not, the reported EA successes would not be reproducible.","tokens_in":19588,"feed_emoji":"🧩","tokens_out":10056,"duration_ms":85115,"temperature":0.7,"pith_summary":"The paper studies generalized sliding puzzles on a d-dimensional hypercube: colored rings occupy distinct vertices and may slide only across a k-dimensional face whose other vertices are empty. Its central claim is a computational frontier: in dimension three all three tested methods—A* search, an evolutionary algorithm, and reinforcement learning—solve the puzzles, and A* certifies optimal solutions; in dimensions four and five, exact A* search becomes infeasible for the harder levels, while the two stochastic methods still produce short, 'generally acceptable' solutions, with medians usually below thirty moves. The authors present this benchmark as evidence that approximately optimal search can cover a regime that exact search cannot reach for this NP-hard puzzle family. If the claim is right, the practical conclusion is that the high-dimensional cases are algorithmically tractable in an approximate sense, not that they become easy.","feed_headline":"Exact search gives out at dimension 5; stochastic search carries on","feed_subtitle":"When exact A* becomes infeasible, evolutionary and reinforcement searches still find solutions with median move counts usually under 30.","key_machinery":"The load-bearing object is the (d,k,l)-cubical sliding puzzle on the hypercube $Q_d$: $2^d-l$ colored rings occupy distinct vertices, and a k-move slides a ring to any vertex of a k-dimensional face that contains no other ring. The A* implementation depends on the admissible lower-bound heuristic $h_T(C)=\\sum_i \\lceil |C_i-T_i|/k\\rceil$, whose monotonicity is used to guarantee optimality. The EA carries the argument through a fitness $f_a(C,T)=1/(1+L-h(C,T))$ and a selection force that linearly combines that fitness with the inverse move count. The RL method carries its share through a reward function seeded by a breadth-first expansion of the target configuration, so that early random walks are biased toward configurations with known short paths to the goal.","core_discovery":"On the paper's own terms, the discovery is that the difficulty wall for exact search is not a wall for stochastic search. For d=3, A* yields provably optimal move counts between 4 and 10, and both RL and EA match or approach these optima. For d=4, A* still solves levels 0-3 optimally but fails on level 4 within the time limit; for d=5, A* finds no solution, while EA solves the k=1, k=2, and k=3 puzzles with median move counts of 12, 9, and 7, and RL solves them with medians of 8, 5, and 4. The paper's own tables record one sharp exception to the 'median below 30' slogan: for d=5 and k=4, RL has median 161 moves and EA never terminates successfully. The authors infer that EA is consistently the fastest method and RL the most accurate stochastic method, with A* remaining the method of choice whenever optimality is required and the dimension is small.","pith_inferences":["[Editorial inference] If the printed fitness in Eqs. (3)-(4) is taken literally—h counts mismatches—the EA as described cannot terminate, so the reported EA numbers must come from an unstated corrected objective; re-running with the stated equations would be a direct test of which algorithm the benchmark actually evaluates.","[Editorial inference] The target-seeded breadth-first weighting used for RL could be fused with A* as a bidirectional or windowed search, potentially pushing exact solutions into some d=5 instances where the current A* stalls.","[Editorial inference] The paper compares medians and CPU times, but the full distributions suggest a three-way trade-off among accuracy, variance, and speed; a Pareto-style summary would let practitioners choose a method by difficulty level rather than by dimension alone.","[Editorial inference] Because the difficulty levels appear to have been generated only for k=d-1, the reported monotone increase of move count with level and k may be an artifact of level generation; generating levels independently per k could separate puzzle hardness from algorithm behavior."],"forward_implications":["A* search certifies optimal solutions for d=3 and for most d=4 puzzles, with optimum move counts between 4 and 10, so for small dimensions exact search remains the reference method.","For d=5, A* times out, but EA solves k=1,2,3 puzzles with median moves 12, 9, and 7, and RL solves the same cases with medians 8, 5, and 4; approximate solutions remain available where optimal ones are not.","EA is consistently the fastest in CPU time but its success rate collapses for d=4, k=3 to under 5 percent, whereas RL keeps near-100 percent success across tested levels and face dimensions.","The A* branching factor on these puzzles is at least $2^k-1$, so exact search cost grows at least like $(2^k-1)^n$ in solution length $n$; this is the concrete combinatorial wall that the stochastic methods are claimed to bypass."],"supporting_citations":[{"why":"defines the higher-dimensional cubical sliding puzzle, the k-rule, and the connectivity results that the benchmark extends.","marker":"[4]"},{"why":"supplies the A* admissibility and monotonicity conditions that justify calling the exact search optimal.","marker":"[6]"},{"why":"establishes NP-hardness of the shortest-move-sequence problem for the graph-generalized 15-puzzle, motivating the approximate methods.","marker":"[5]"},{"why":"is the reference behind the k-rule move definition in the game setting.","marker":"[35]"},{"why":"gives the O(b^n) state-space complexity used to derive the lower bound on A*'s cost.","marker":"[36]"}],"fun_headline_variants":["Exact search hits dimension wall, stochastic keeps going","A* gives out at dimension 5; RL and EA still solve","When exact search times out, stochastic search finds a way","For higher-dim puzzles, exact search fails but stochastic wins"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The EA results presuppose that the implemented fitness function rewards rings that already match their target colours, but the printed equations define h as a mismatch count, so the printed fitness is maximized when zero rings match and the stopping condition $f_a=1$ can never be reached for $L>0$.","fun_headline_variants_meta":{"raw":{"variants":["Exact search hits dimension wall, stochastic keeps going","A* gives out at dimension 5; RL and EA still solve","When exact search times out, stochastic search finds a way","For higher-dim puzzles, exact search fails but stochastic wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00086,"raw_usage":{"total_tokens":3808,"prompt_tokens":1097,"completion_tokens":2711,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":713,"completion_tokens_details":{"reasoning_tokens":2642}},"tokens_in":713,"tokens_out":2711,"duration_ms":19629,"temperature":1.0,"reasoning_tokens":2642,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:00:36.363836+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released EA with the objective exactly as printed in Eqs. (3)-(4): if h(C,T) is the number of mismatched ring-target pairs, then $f_a(C,T)=1/(1+L-h)$ reaches its maximum 1 at h=L, i.e. at a configuration where every ring is in the wrong place, so the loop 'while $f_a \\neq 1$' should never stop for any nontrivial puzzle. If the code still solves puzzles, the experiments are not testing the algorithm described; if it does not, the reported EA successes would not be reproducible.","supporting_citations":[{"cited_title":"Higher-dimensional cubical sliding puzzles","cited_arxiv_id":"2307.14143","evidence_quote":"defines the higher-dimensional cubical sliding puzzle, the k-rule, and the connectivity results that the benchmark extends."},{"cited_title":"A formal basis for the heuristic determination of minimum cost paths","cited_arxiv_id":null,"evidence_quote":"supplies the A* admissibility and monotonicity conditions that justify calling the exact search optimal."},{"cited_title":"Finding the shortest move-sequence in the graph-generalized 15-puzzle is NP-hard","cited_arxiv_id":null,"evidence_quote":"establishes NP-hardness of the shortest-move-sequence problem for the graph-generalized 15-puzzle, motivating the approximate methods."},{"cited_title":"Graph puzzles, homotopy, and the alternating group","cited_arxiv_id":null,"evidence_quote":"is the reference behind the k-rule move definition in the game setting."},{"cited_title":"Artificial Intelligence: A Modern Approach (2nd ed.)","cited_arxiv_id":null,"evidence_quote":"gives the O(b^n) state-space complexity used to derive the lower bound on A*'s cost."}],"review_version":1}