{"id":"2fa6f00c-8d9b-4efd-9791-dbd73bdd7cc6","arxiv_id":"2608.01528","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Projecting a codebook into its own M-dimensional orthonormal basis does not compress it when the basis is stored, so the claimed 15.75x memory reduction is not supported.","lead":"This paper proposes rewriting VSA codebook vectors in an orthonormal basis to reduce GPU memory and speed up inference. The math preserves dot products, but the basis itself takes as much memory as the codebook, so the headline compression claim does not hold.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gram-Space's claimed compression omits the D×k basis U; for full-rank codebooks k=M, storing U and T costs O(DM+M^2), which exceeds the original O(DM), so the reported memory reductions are an accounting artifact.","rationale":"The paper's formal correctness analysis is valid: orthonormal basis transforms preserve inner products and linear combinations in exact arithmetic. But correctness is not the issue. The stated contribution is memory-efficient compression, and that claim fails on its own storage model. §3.1 counts storage as O(Dk+Mk) but ignores that U must be kept to perform reconstruction in §3.3 (x=Ut). In the benchmark regime (D≥256, M=40, random full-rank codebook, k=M), this yields O(DM+M^2), which is asymptotically and concretely larger than the original O(DM). The only way the claimed savings appear is by omitting U from the accounting. The empirical numbers, including the 15.75× memory reduction and the 51× 'compression ratio' in Table 4, are inconsistent with a storage model that includes U. A direct memory measurement with U included would settle this immediately. The reader's weakest_assumption identifies the same issue, and the verdict of REJECT is justified.","tokens_in":8087,"tokens_out":2735,"duration_ms":34913,"concrete_test":"Re-run the memory profiling for NVSA (or ARLC) with Gram-Space, registering U as a persistent buffer on the same device with the same dtype as W, and measure peak GPU memory during inference. Compare this peak to the original codebook W's peak memory. If including U does not yield a memory reduction—or if the 15.75× figure disappears—the compression claim is refuted. Also recompute the theoretical storage ratio counting both U and T: (Dk+Mk)/(DM) = k/M + k/D; for k=M this equals 1+M/D, which is >1.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that Gram-Space compresses VSA codebooks, with storage O(Dk+Mk) instead of O(DM). However, the model must store both the coefficient matrix T (k×M) and the orthonormal basis U (D×k), because reconstruction x=Ut (§3.3) requires U. For the VSA codebooks used here, columns are random high-dimensional vectors with D≥M, hence almost surely full column rank, so k=M. Storage becomes O(DM+M^2), strictly larger than the original O(DM). §3.1 claims 'net memory reduction when k≪D', but the experiments set k=M ('maximal compression') with M=40, D≥256, so k is not ≪D; the stated ratio D/k≥6.4 is not a storage reduction ratio. The reported 15.75× model-level memory reduction (Fig. 5b, Table 1) and the '51× compression ratio' in Table 4 appear to exclude U or to compare D/M rather than total parameter counts. If U must be retained, the codebook is not compressed; if U is dropped, §3.3's reconstruction is impossible. This is the load-bearing weakness in the paper's main contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Gram-Space, a compression framework for vector-symbolic-architecture (VSA) codebooks used in neuro-symbolic reasoning. It constructs an orthonormal basis U for the column space of the codebook W, projects the codebook into coefficients T = U^T W, and runs matrix-space operators (similarity, probability vectorization, attention) directly in the coefficient space. Vectors are reconstructed via x = Ut before component-wise symbolic operators such as binding/unbinding. The paper proves an inner-product invariance result, claims storage of O(Dk+Mk) versus the original O(DM), and reports memory reductions up to 15.75x and latency speedups up to 3.62x on NVSA, LearnVRF, and ARLC.","tokens_in":8368,"tokens_out":4592,"duration_ms":55535,"significance":"The invariance statement, Eq. (2), is correct: for any A,B in span(U), <A,B> = <U^T A, U^T B>. This is a standard property of orthonormal bases and is proved directly from definitions, so the mathematical core is sound. If the storage claim were valid, the method would be a useful lossless, retraining-free compression for matrix-space VSA operators. However, the central storage claim does not survive accounting for the basis U. The reported empirical reductions appear to measure intermediate-tensor allocations rather than persistent codebook storage. The paper's main contribution, codebook compression, is therefore unsupported as stated.","major_comments":[{"comment":"The claimed storage O(Dk+Mk) omits that the representation is (U,T), not just T. With k=M, which is the setting used in all experiments, storage is O(DM+M^2), strictly larger than the original O(DM). The paper's condition 'k<<D' is not met: experiments set k=M=40 with D≥256, so D/k≥6.4 is a dimension ratio, not a storage compression ratio. If U is kept on-GPU for reconstruction, the codebook is not compressed; if U is dropped, the reconstruction x=Ut in §3.3 is impossible. The claimed memory reductions are therefore not supported as codebook compression.","section":"§3.1, Eq. (1)"},{"comment":"The empirical reductions appear to conflate transient GPU memory with persistent codebook storage. Table 1 reports reductions in Empty&copy, MatMul, and Concat memory, which are consistent with performing matrix operations in a lower-dimensional coefficient space; that is a computational optimization, not a reduction in the codebook footprint. The abstract's 15.75x model-level memory reduction and Table 4's 51x compression ratio need a precise parameter-count statement. As written, they do not establish the central claim of codebook compression.","section":"§4.2, Fig. 5, Tables 1 and 4"},{"comment":"The sentence 'the codebook is iteratively re-initialized to become fully orthogonal before basis construction' is a substantive modification of the codebook, not a compression step. No algorithm, ablation, or theoretical argument is given to show that this re-initialization preserves reasoning accuracy. Since VSA binding and unbinding rely on the statistical properties of random high-dimensional vectors, replacing them with an exactly orthogonal set may change rule semantics. This is load-bearing for the lossless-accuracy claim and needs to be addressed explicitly.","section":"§3.3"}],"minor_comments":[{"comment":"The definition of k is inconsistent: it is called the algebraic rank of the codebook in §3.2, but set to M in §3.1. Clarify whether k is rank or a configurable subspace dimension and how the two relate when D≥M.","section":"§3.1, §3.2"},{"comment":"The 'Accuracy after reconstruction' in Table 4 appears to be a similarity score between original and reconstructed codebook, not task accuracy. The text claims no model accuracy loss on RAVEN and I-RAVEN, but no task-accuracy numbers are reported. Please provide them or qualify the claim.","section":"Table 4"},{"comment":"The compression ratio in Fig. 5(a) should be clearly defined. As drawn, it appears to plot D/k rather than an actual storage ratio; these are not the same when U must be stored.","section":"Fig. 5(a)"},{"comment":"The phrase 'we set k=M for maximal compression' is misleading. At k=M the coefficient matrix T has the same column dimension as W but the basis U adds M extra columns per original vector, so there is no coefficient-space dimension reduction relative to the full codebook.","section":"§3.1"}],"recommendation":"reject","confidential_remarks":"The paper's central contribution rests on a storage-accounting error: the D×k basis U is part of the representation, and with k=M it makes the compressed codebook larger than the original. The invariance theorem is textbook linear algebra and likely correct, and the latency improvements from lower-dimensional matmuls may be real, but these do not constitute codebook compression. Reframing the contribution as intermediate-tensor optimization would not match the title or the stated claims. I see no path to acceptance without changing the core contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know that the paper's main claim—15.75x memory reduction from codebook compression—doesn't hold up under a simple storage count. The authors store both the D×k basis U and the k×M coefficient matrix T. For the VSA codebooks they use, k=M (full column rank), so the total is O(DM+M^2), which is larger than the original O(DM). Their own §3.1 gives the formula O(Dk+Mk) but then sets k=M, and the numbers in Fig 5 and Table 4 appear to exclude U or compare D/M rather than parameter counts. That's the load-bearing flaw.\n\nWhat the paper does well: the orthonormal-basis invariance argument (Eq. 2) is correctly stated, and the operator-space classification—matrix-space vs component-wise—is a sensible way to think about which VSA stages can be reparameterized. The profiling data (Table 1, roofline) are also plausibly collected, and they suggest that much of the observed speedup comes from shrinking intermediate tensors in matrix multiply, not from compressing the codebook itself. If reframed as 'running matrix-space ops in a low-dimensional coefficient space,' the systems work might have some value.\n\nSoft spots beyond the storage error: no baseline against a simple QR or SVD projection of the codebook; the '51× compression ratio' in Table 4 is just D/M, not a storage ratio; the codebook re-initialization step in §3.3 is unexplained and sounds like it could change the codebook; and no code or data is provided to reproduce the latency/memory numbers.\n\nBottom line: the theoretical correctness is not the issue—the issue is that the paper's own accounting contradicts its headline claim. This is for readers who want hands-on practice spotting compressed-representation fallacies. I would not cite it as a successful compression method. But the underlying idea of operator-aware reparameterization is worth a serious referee's time if the authors are willing to correct the accounting and reframe the contribution honestly; as it stands, I'd recommend a major revision or reject with an invitation to resubmit.\n\nMy recommendation: send it to peer review, because the flaw is identifiable and fixable, but the reviewers should force a real comparison and a correct memory model.","headline":"The invariance proof is fine, but the compression claim dissolves once you count the basis matrix U.","tokens_in":8823,"tokens_out":2998,"would_cite":false,"duration_ms":37336,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a VSA codebook can be compressed losslessly into an orthonormal basis and coefficient matrix—cutting GPU memory up to 15.75x and latency up to 3.62x—because inner-product-based operations are invariant under orthonorm","keywords":["vector symbolic architectures","neuro-symbolic AI","codebook compression","Gram-Schmidt orthogonalization","orthonormal basis","GPU memory","inference latency","lossless compression"],"falsifier":"For the paper's own setting—M = 40, D = 256, k = M—count stored scalars: Dk + Mk = 11840 versus DM = 10240, so storing both U and T is larger. Measure peak GPU memory during inference with (U, T) both resident and compare it to the original W resident; if (U, T) does not reduce peak memory, the central compression claim is refuted.","tokens_in":7957,"feed_emoji":"📉","tokens_out":7170,"duration_ms":65848,"temperature":0.7,"pith_summary":"This paper argues that a VSA codebook—a large matrix of near-orthogonal high-dimensional vectors used as a lexicon in neuro-symbolic reasoning—can be replaced by a much smaller pair of matrices without changing the results of the pipeline's matrix-space computations. The idea is to compute an orthonormal basis U for the column space of the codebook and store each vector by its coordinates T = U^T W; because inner products are unchanged by orthonormal change of basis, similarity scoring, probability vectorization, and attention can run in this compact 'Gram-Space' and only vectors needed by component-wise symbolic operators (binding, unbinding) are reconstructed as x = U t. If correct, this would cut codebook-associated GPU memory by up to 15.75x and inference latency by up to 3.62x on three representative VSA-based models, with no retraining and no accuracy loss.","feed_headline":"Orthonormal basis cuts VSA codebook memory 15.75x","feed_subtitle":"Similarity and attention run in the compact coefficient space; vectors are rebuilt only for binding and unbinding.","key_machinery":"The Gram-Space representation (U, T): an orthonormal basis U of the codebook's column space together with the coefficient matrix T = U^T W. It carries the argument because orthonormal basis transforms preserve inner products exactly in exact arithmetic, so every matrix-space operator—similarity, probability vectorization, attention score—can be executed on the k-dimensional coordinates; only component-wise symbolic operators require the reconstruction x = U t, applied per-batch to active vectors. The paper also uses Householder QR rather than classical Gram-Schmidt to keep the basis numerically stable.","core_discovery":"The central claim is that Gram-Space achieves lossless structure-preserving compression: for a codebook matrix W ∈ $R^{{D×M}}$, construct an orthonormal basis U ∈ $R^{{D×k}}$ spanning the column space of W and coefficients T = U^T W ∈ $R^{{k×M}}$. Since every vector and linear combination produced by matrix-space operators lies in span(U), ⟨A,B⟩ = ⟨U^T A, U^T B⟩, so dot-product similarity, probability-to-vector transforms, and attention computations are numerically equivalent in the compressed coordinates. Component-wise operations such as softmax and binding/unbinding are not invariant, so those stages reconstruct the few active vectors via x = U t. The paper reports that this reduces model-level GPU m","pith_inferences":["The same lossless change-of-basis argument applies to any frozen embedding table or codebook whose downstream operations are inner-product based—similarity search, attention key/value lookups, or nearest-neighbor classification could run in coefficient space and reconstruct only at nonlinear boundaries.","A direct way to push the idea further is to exploit true low rank: if some codebook entries are linear combinations of others, k can be made smaller than M, and the storage tradeoff Dk + Mk versus DM improves beyond the full-rank case.","The paper's own profiling suggests the headline savings come mainly from shrinking intermediate tensors in matrix-heavy stages rather than from the static codebook; caching and reusing reconstructed vectors across batches would target the remaining memory-bound reconstruction stage."],"forward_implications":["Model-level GPU memory usage falls by up to 15.75x across NVSA, LearnVRF, and ARLC, reducing the minimum GPU memory needed to run these workloads.","End-to-end inference latency improves by 1.68x to 3.62x, with ARLC benefiting most because it repeatedly executes matrix-based codebook operations during rule exploration.","Reconstructed codebooks match the originals exactly (similarity score 100%) and model accuracy on RAVEN and I-RAVEN is unchanged, so no retraining is needed.","Matrix-space operators—similarity, probability vectorization, attention—produce numerically equivalent results in the compressed coordinates, not approximations.","After compression, the bottleneck shifts from allocation-heavy tensor materialization to matrix multiplication, so further gains would require reducing reconstruction-stage memory traffic."],"supporting_citations":[{"why":"Provides the NVSA baseline model that Gram-Space compresses and profiles.","marker":"Hersche et al. (2023)"},{"why":"Provides the LearnVRF baseline model, the learnable-rule variant.","marker":"Hersche et al. (2024)"},{"why":"Provides the ARLC baseline model used for rule-exploration experiments.","marker":"Camposampiero et al. (2024)"},{"why":"Documents errors in FFT-based compression for bipolar/binary codebooks, the baseline Gram-Space positions against.","marker":"Ganesan et al. (2021)"},{"why":"Defines the attention-score operator that Gram-Space preserves in matrix space.","marker":"Vaswani et al. (2017)"},{"why":"Characterizes neuro-symbolic workloads and motivates the memory bottleneck that Gram-Space addresses.","marker":"Wan et al. (2024)"}],"fun_headline_variants":["Gram-Space: 15.75x memory cut for VSA reasoning","Lossless codebook compression shrinks memory 15.75x","Orthonormal basis trims VSA memory 15.75x","Gram-Space: 3.62x speedup, 15.75x memory cut"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The savings argument assumes the orthonormal basis U can be stored without offsetting the compression: with k = M the pair (U, T) costs O(DM + $M^{2}$), which is no smaller than storing the original codebook W, so the reported memory reduction must come from intermediate tensors or from excluding U from the count.","fun_headline_variants_meta":{"raw":{"variants":["Gram-Space: 15.75x memory cut for VSA reasoning","Lossless codebook compression shrinks memory 15.75x","Orthonormal basis trims VSA memory 15.75x","Gram-Space: 3.62x speedup, 15.75x memory cut"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00017,"raw_usage":{"total_tokens":1102,"prompt_tokens":741,"completion_tokens":361,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":277}},"tokens_in":485,"tokens_out":361,"duration_ms":4127,"temperature":1.0,"reasoning_tokens":277,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T00:04:11.493655+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For the paper's own setting—M = 40, D = 256, k = M—count stored scalars: Dk + Mk = 11840 versus DM = 10240, so storing both U and T is larger. Measure peak GPU memory during inference with (U, T) both resident and compare it to the original W resident; if (U, T) does not reduce peak memory, the central compression claim is refuted.","supporting_citations":[],"review_version":1}