RSD: A Local Triangulation Audit Primitive for Learned Vector Blocks

Seungmin Jin

arxiv: 2605.17482 · v2 · pith:UAP4WPZDnew · submitted 2026-05-17 · 💻 cs.CL · cs.LG

RSD: A Local Triangulation Audit Primitive for Learned Vector Blocks

Seungmin Jin This is my paper

Pith reviewed 2026-05-20 12:44 UTC · model grok-4.3

classification 💻 cs.CL cs.LG

keywords word embeddingssemantic decompositionresidual analysisambiguous wordscontext diagnosticsGloVe embeddingsadditive models

0 comments

The pith

Residual Semantic Decomposition extracts semantic axes from word embeddings while leaving residuals that capture unabsorbed context information for ambiguous terms.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Residual Semantic Decomposition as a neural additive method that decomposes word embeddings into components balancing vector reconstruction against relational structure. Each binary step fits one local semantic axis and exposes the leftover information in the residual. In tests using manually paired contexts around ambiguous words, the method separates the supplied context anchors more clearly than shuffled-label controls. The authors conclude that these residuals can serve as qualitative diagnostics for semantic structure rather than as inputs to full quantitative sense-prediction benchmarks. A reader would care because the approach offers an inspectable way to see what static embeddings leave behind when words carry multiple senses.

Core claim

RSD performs recursive binary decomposition on embeddings such that each K=2 fit extracts a local semantic axis while the residual vector holds information not absorbed by that axis; in manually specified paired-context diagnostics over ambiguous words, RSD separates supplied context anchors above shuffled-label controls, yet entropy checks show ambiguous targets are not uniformly high-entropy boundary points in static GloVe, so residual neighborhoods are treated as qualitative diagnostics.

What carries the argument

The residual vector after each binary semantic-axis fit, which preserves information not captured by the current axis while maintaining relational structure across the decomposition.

If this is right

Residuals after axis extraction can be inspected for semantic content that the fitted axis did not absorb.
Recursive application allows peeling successive layers of meaning from a single embedding.
The balance between reconstruction and structure preservation keeps relational signals intact across steps.
Ambiguous words need not sit at uniformly high-entropy locations in static embeddings for the method to apply.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same residual approach could be applied to contextual embeddings to check whether dynamic representations absorb more axes at once.
One could test whether feeding the residuals back into training improves handling of polysemy in downstream tasks.
The method might connect to axis-based interpretability techniques that seek human-readable directions in embedding space.

Load-bearing premise

Residual neighborhoods after decomposition can be treated as meaningful qualitative diagnostics for semantic structure without first passing quantitative sense-prediction benchmarks.

What would settle it

A paired-context diagnostic run on ambiguous words in which RSD shows no separation advantage over shuffled-label controls would falsify the separation result.

Figures

Figures reproduced from arXiv: 2605.17482 by Seungmin Jin.

read the original abstract

Local XAI audits compare a finite block of learned vectors with a weak side signal. Baselines such as nearest-neighbor lookup, low-rank coordinate models, and relation factorization expose different parts of this audit. We introduce Relational Semantic Decomposition, abbreviated as RSD, as a local triangulation audit for learned vector blocks. Given coordinates X and a declared bounded weak affinity proxy A, RSD fits simplex memberships S and coordinate poles C. It reuses S in a relation decoder for A and reports the coordinate residual R=X-SC. This yields a scoped audit unit: compatibility for the chosen block, proxy, decoder class, and loss budget, plus component mass and residual readouts. Synthetic controls check simplex reconstruction, proxy decoding, and fixed-S residual decomposition. The theorem-statement, month, and dog/wolf blocks illustrate why low proxy loss should be read with component mass, residual readouts, and block size.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper offers a recursive residual decomposition for word embeddings with modest qualitative evidence on ambiguous word contexts.

read the letter

The key takeaway from this paper is a new recursive binary decomposition method for word embeddings called Residual Semantic Decomposition. It fits local semantic axes one at a time and uses the residuals to expose information not captured by each axis, then applies this to diagnostics on ambiguous words. The novel part is combining neural additive decomposition with recursion at K equals 2 and a deliberate focus on the residuals for context analysis. In the manually specified paired-context tests, it separates the supplied anchors better than shuffled-label controls. The authors also make a good point that ambiguous targets are not uniformly high-entropy in static GloVe embeddings, which leads them to treat the residual neighborhoods as qualitative diagnostics instead of quantitative benchmarks. This approach has merit in its honesty about scope and limitations. It avoids overclaiming and adjusts the framing based on the entropy observation. Where it falls short is in the strength of the evidence presented. The abstract notes diagnostic separation but lacks concrete numbers, statistical tests, or ablation studies to show the contribution of the residual component specifically. Implementation details like the neural architecture or training procedure are missing from the summary, which leaves questions about reproducibility. If the full paper includes code or more rigorous benchmarks, that would address this. This paper is aimed at researchers in natural language processing who are interested in dissecting what word embeddings represent, particularly in handling polysemy through decomposition techniques. A reader focused on interpretability methods might find the residual idea practical to build on. Overall, the work demonstrates clear engagement with the problem and has enough substance to warrant peer review. I recommend sending it to referees for feedback on expanding the quantitative validation and clarifying the method details.

Referee Report

1 major / 2 minor

Summary. The paper introduces Residual Semantic Decomposition (RSD), a neural additive decomposition of word embeddings that balances reconstruction with relational structure preservation. It enables recursive binary decomposition where each K=2 step extracts a local semantic axis and residuals capture unabsorbed information. Evaluation uses manually specified paired-context diagnostics on ambiguous words, showing separation of supplied context anchors above shuffled-label controls; entropy analysis indicates ambiguous targets are not uniformly high-entropy boundary points in GloVe, leading the authors to frame residual neighborhoods as qualitative diagnostics rather than sense-prediction benchmarks.

Significance. If the central diagnostic result holds under more rigorous testing, RSD offers a structured way to probe semantic axes in static embeddings without assuming uniform high entropy for polysemous terms. The explicit qualitative framing and the entropy observation are strengths that avoid over-claiming. This could aid interpretability work in NLP, though broader significance would increase with reproducible code, quantitative benchmarks, or falsifiable predictions on standard sense disambiguation tasks.

major comments (1)

Abstract: The reported separation of context anchors above controls is presented without quantitative metrics, error analysis, statistical tests, or implementation details (e.g., network architecture, loss weights, or training procedure). This makes the effect size and robustness of the central diagnostic claim difficult to assess and is load-bearing for evaluating whether RSD meaningfully extracts semantic structure.

minor comments (2)

The manuscript would benefit from a dedicated related-work section comparing RSD to prior additive decomposition techniques (e.g., PCA-based or matrix factorization methods) to clarify novelty.
Expand the entropy observation with concrete examples or a figure showing entropy distributions for ambiguous vs. unambiguous words to strengthen the justification for treating results as qualitative diagnostics.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive review and for noting the value of our qualitative framing and entropy observations. We address the single major comment below and have revised the manuscript accordingly to improve transparency around the diagnostic results.

read point-by-point responses

Referee: Abstract: The reported separation of context anchors above controls is presented without quantitative metrics, error analysis, statistical tests, or implementation details (e.g., network architecture, loss weights, or training procedure). This makes the effect size and robustness of the central diagnostic claim difficult to assess and is load-bearing for evaluating whether RSD meaningfully extracts semantic structure.

Authors: We agree that the abstract summarizes the finding at a high level and that additional pointers would aid evaluation. The manuscript positions the paired-context tests as qualitative diagnostics rather than a quantitative benchmark, consistent with the entropy analysis showing that ambiguous targets are not uniformly high-entropy boundary points. Implementation details (network architecture, loss weighting between reconstruction and relational terms, and training procedure) appear in Section 3, while the separation is illustrated via concrete examples and shuffled-label controls in Section 4 and the associated figures. To address the concern directly, we have revised the abstract to reference the methods section for these details and to clarify that the evaluation relies on manually specified cases without aggregate statistical testing. We believe this maintains the paper's intended scope while improving assessability. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper introduces RSD as an independent neural additive decomposition balancing reconstruction and relational preservation, with recursive binary decomposition extracting local axes and exposing residuals. The central empirical claim is a qualitative diagnostic result: RSD separates supplied context anchors above shuffled-label controls in manually specified paired-context tests over ambiguous words. No equations, derivations, or fitted parameters are shown that reduce this result to inputs by construction. No self-citations, uniqueness theorems, or ansatzes are invoked as load-bearing steps. The abstract explicitly frames residuals as qualitative diagnostics rather than quantitative benchmarks, confirming the method and result are self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review limited to abstract; no explicit free parameters, axioms, or invented entities are detailed. The approach implicitly assumes embeddings contain decomposable semantic structure and that residuals carry usable diagnostic information.

axioms (1)

domain assumption Word embeddings contain additive semantic structure that can be recursively extracted via neural decomposition while preserving relational information.
Core premise stated in the introduction of RSD in the abstract.

pith-pipeline@v0.9.0 · 5600 in / 1198 out tokens · 28402 ms · 2026-05-20T12:44:42.765893+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

RSD uses a soft assignment mechanism via an encoder MLP... Lemb = ||SC - X||_F^2, Lrel = ||S D_rel S^T - A||_F^2... component contrast d = c1 - c0... node-root residual e(G)_i = x_i - s_i C
IndisputableMonolith/Foundation/ArithmeticFromLogic.lean LogicNat recovery unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

We therefore treat residual neighborhoods as qualitative diagnostics rather than benchmark sense predictions.

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.