{"id":"4110ef46-7864-4d84-a44a-2954cfc7fb1f","arxiv_id":"2607.18004","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A pairwise-adaptive Mahalanobis distance, parameterized as a positive-definite quadratic form, improves representation learning and control performance when plugged into bisimulation-based visual RL algorithms.","lead":"This paper introduces a new type of distance metric for comparing visual states in reinforcement learning, which adapts to each pair of states while retaining a structured mathematical form. It shows that swapping this metric into existing bisimulation-based RL algorithms improves their control performance on several visual MuJoCo tasks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DBC-family gains rest on DBC-Det, a modified baseline whose comparability to original probabilistic DBC is questionable; Hopper Hop shows a large gap.","rationale":"The central claim has two empirical pillars: the DBC family and the independently-coupled (MICo/SimSR) family. The independently-coupled comparison is well controlled: within the SimSR pipeline, only the latent comparator is replaced (cosine by PAMD), and the gains are consistent across tasks. The residual-fitting diagnostic and the diagonal/trace ablations further support the mechanism. The DBC pillar, however, depends on a modified baseline. The paper explicitly changes the transition model to make PAMD applicable, then compares against DBC-Det rather than the original probabilistic DBC. The paper's own data show that DBC-Det and DBC are not reliably interchangeable: Hopper Hop differs by roughly a factor of two. Because the headline DBC numbers (e.g., DBC-Det 350±60 to PAMD 524±14) are against this stand-in, the magnitude of the claimed DBC-family improvement is uncertain, even though the direction is plausible. This is a load-bearing concern because it undermines the generality of the plug-in claim: PAMD cannot currently be plugged into one of the two operator families it claims to improve. It is not grounds for rejection, because the independently-coupled results and the diagnostic provide independent support for the central qualitative conclusion. Therefore the reader's CONDITIONAL verdict stands unchanged, but the rebuttal should supply the requested equivalence evidence or a proper DBC-PAMD implementation.","tokens_in":19357,"tokens_out":9161,"duration_ms":107749,"concrete_test":"Run original probabilistic DBC with PAMD by replacing the closed-form Gaussian Wasserstein term with a sample-based Sinkhorn or dual-form Wasserstein using d_theta as ground cost (or, if impossible, use an ensemble of Gaussian transitions and Monte-Carlo estimate of W_dtheta), keeping all other DBC hyperparameters. Compare DBC-PAMD vs original DBC and vs DBC-Det+PAMD on Hopper Hop and Cheetah Run. If DBC-PAMD does not beat DBC by a margin comparable to DBC-Det+PAMD vs DBC-Det, the DBC-family claim is not supported. A cheaper intermediate check: increase DBC-Det seeds to at least 20 on Hopper Hop and test statistical equivalence with DBC; if DBC-Det is not comparable, the 33->181 gain cannot be attributed to the metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's DBC-family evidence is not a clean metric plug-in. Section 5.1 admits that PAMD cannot be inserted into the original probabilistic DBC loss because the closed-form Gaussian Wasserstein term is tied to a Euclidean ground metric; all DBC+PAMD runs use a deterministic-transition variant (DBC-Det). The controlled comparison is therefore DBC-Det vs. DBC-Det+PAMD, not DBC vs. DBC+PAMD. For that comparison to support the headline that DBC is substantially improved, DBC-Det must be a faithful stand-in for DBC. Table 1 shows this is doubtful: on Hopper Hop DBC-Det (33±29) is roughly half of DBC (67±16), and on Cheetah Run the means differ by 40 (350 vs 310) with overlapping error bars. The paper calls these 'broadly comparable' without a statistical test. If DBC-Det is a weaker or differently shaped baseline, the metric-replacement gain (e.g., 350->524 on Cheetah, 33->181 on Hopper) may overstate what PAMD would give inside the original DBC. The direct DBC vs. DBC-Det+PAMD comparison is confounded by the simultaneous transition-model change. Thus the DBC-family portion of the central claim is not yet demonstrated in the original algorithm; the independently-coupled (SimSR/MICo) results are better controlled and provide the strongest support.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PAMD (Pairwise Adaptive Mahalanobis Distance), a plug-in latent dissimilarity for bisimulation-style representation learning in visual RL. PAMD replaces fixed global norms (e.g., ℓ1, cosine) with a pair-conditioned positive-definite quadratic form dθ(z,z') = sqrt((z−z')ᵀ G̃θ(z,z')(z−z') + ε), where G̃θ is a symmetrized, trace-normalized Cholesky-parameterized matrix. The authors argue that this structured adaptive geometry is more expressive than fixed norms while avoiding the degenerate solutions of an unconstrained pairwise MLP distance. They evaluate PAMD in two operator families: a deterministic-transition DBC variant and an independently-coupled MICo/SimSR style operator, reporting improved final returns on several pixel-based DMControl tasks. They also provide a residual-fitting diagnostic showing that a parameter-matched unstructured MLP distance can reduce the Bellman residual with a frozen encoder, whereas PAMD requires a trainable encoder, which they interpret as evidence that PAMD preserves learning pressure on the representation.","tokens_in":19735,"tokens_out":6333,"duration_ms":71398,"significance":"The core claim is consequential: if correct, the choice of latent distance geometry is a first-order design factor in bisimulation-based representation learning, not a minor implementation detail. The paper's strengths include hyperparameter-controlled comparisons, re-run baselines, a parameter-matched MLP-vs-PD diagnostic, and a clean conceptual framework connecting metric embedding to behavioral distances. The residual-fitting experiment is a genuinely useful tool. However, the current evidence is incomplete: the DBC-family comparison is confounded by the switch to a deterministic-transition variant, and the independently-coupled comparison suffers from ambiguity about which baseline pipeline the PAMD curve actually uses. These issues affect the two main empirical pillars of the paper, so the central claim is defensible but not yet fully demonstrated.","major_comments":[{"comment":"The controlled DBC-family comparison is DBC-Det vs. DBC-Det+PAMD, not original DBC vs. DBC+PAMD, because PAMD cannot be inserted into the closed-form Gaussian Wasserstein term of probabilistic DBC. The paper bridges this with the assertion that DBC-Det and DBC are 'broadly comparable,' but the Hopper Hop row contradicts this: DBC-Det is 33±29 vs. DBC at 67±16, a factor-of-two gap, and Cheetah Run shows only overlapping error bars (350±60 vs. 310±32). No statistical test is provided. Since the DBC-family results are used to claim improvement over a standard bisimulation algorithm, this is a load-bearing confound. Please either (a) report DBC+PAMD using a PAMD-compatible transition-discrepancy estimator matched to the probabilistic baseline, or (b) provide a per-seed paired analysis and additional tasks demonstrating that DBC-Det is a faithful stand-in, and restrict the DBC claims accordin","section":"§5.1, Table 1"},{"comment":"The single curve 'Indep.-coupled (MICo/SimSR)+PAMD' cannot preserve the same training pipeline as both MICo and SimSR simultaneously, because Appendix D.5 states that MICo uses no learned transition model while SimSR uses an ensemble probabilistic transition model. If the PAMD curve is SimSR-based, the comparison to MICo is not controlled; if it is a third implementation, the exact pipeline is unspecified. This matters because the independently-coupled family is the best-controlled part of the evidence. Please report SimSR+PAMD and MICo+PAMD separately, each matched to its corresponding baseline pipeline, or clearly specify which baseline the 'Indep.-coupled+PAMD' curve extends.","section":"§5.1, Fig. 2 bottom row and Table 3"},{"comment":"The statement that PAMD 'preserves the contraction property of standard behavioral distance operators' is only valid for the fixed target distance, not for the learned proxy dθ. The training objective in Eq. (15) is a TD-style regression whose target contains dθ evaluated on next states; no convergence or contraction guarantee is proven for the learned comparator, and the residual diagnostic in §5.2 shows that the learned distance does not fully reach the fixed point under a frozen encoder. Please soften this claim to apply to the target operator only, or provide a formal statement and proof for the learned distance.","section":"§6 and Remark 4.2"}],"minor_comments":[{"comment":"The upper bound in Lemma 4.1 is dθ(z,z') ≤ sqrt(||Δ||₂² + ε), not dθ ≤ ||Δ||₂ as suggested by the prose 'dθ ≲ ||Δ||₂'. Please align the notation and the statement.","section":"§4.2, Eq. (12)–(14)"},{"comment":"The metric loss as written uses Tdθ in the target without indicating that the target is detached. The detachment is only described in Appendix D.1. Please state this in the main text to avoid apparent circularity.","section":"§4.2, Eq. (15)"},{"comment":"The DBC column for Cheetah Run reports 350±49, while Table 1 reports DBC (Prob.) = 310±32 and DBC-Det = 350±60. Please clarify whether Table 3 reports probabilistic DBC or DBC-Det, and reconcile the numbers.","section":"Table 3"},{"comment":"The residual-fitting diagnostic would benefit from reporting the actual residual values after convergence for both frozen and trainable encoders, rather than only the learning curves, to support the claim that PAMD 'plateaus above a nonzero level' when the encoder is frozen.","section":"§5.2 / Appendix E"},{"comment":"The auxiliary anisotropy coefficient λ_aux = 1e−3 is introduced but not ablated. Since it directly encourages deviation from isotropy, a small sensitivity study (e.g., λ_aux ∈ {0, 1e−4, 1e−2}) would strengthen the claim that the benefits come from the structured form rather than this regularizer.","section":"Appendix D.1"}],"recommendation":"major_revision","confidential_remarks":"The DBC-Det issue is the main substantive concern: the paper's own Table 1 shows a large discrepancy on Hopper Hop, and the 'broadly comparable' phrasing is not backed by a statistical test. This is fixable with additional experiments or a more cautious claim, but it currently prevents the DBC-family result from being taken as evidence about the original DBC algorithm. The independently-coupled evidence is promising but requires pipeline clarification. I do not see grounds for rejection, as the structured-distance idea and the residual diagnostic are valuable and the central mechanism is plausibly correct."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing you should know: this paper makes a clean, modular contribution, and its main empirical claim mostly holds up, but the DBC side of the story is weaker than the abstract suggests. The independently-coupled (MICo/SimSR) results are the real evidence; the DBC results are against a modified baseline.\n\nWhat is actually new: replacing the fixed latent distance in bisimulation-based RL with a pair-conditioned positive-definite quadratic form is a sensible middle ground between an ℓₚ norm and an unconstrained pairwise MLP. The structured form matters because it prevents the distance head from absorbing all supervision while leaving the encoder under-trained. The residual-fitting diagnostic (frozen vs trainable encoder) is a nice, directly interpretable tool for this failure mode, and the parameter-matched comparison (MLP vs PD, ~194k params each) gives it teeth. The ablations on diagonal variants and trace normalization are also honest and informative.\n\nWhere it is soft: the DBC-family gains (e.g., Cheetah 350→524, Hopper 33→181) are not DBC vs DBC+PAMD. They are DBC-Det vs DBC-Det+PAMD. The paper asserts DBC-Det and DBC are \"broadly comparable,\" but Table 1 shows Hopper Hop at 33±29 vs 67±16 — that is a 2× gap, not comparable. So the controlled comparison is really between two deterministic-transition variants, and whatever the metric replacement buys, it is not yet shown to transfer to the original probabilistic DBC. The reference comparison to TACO/DrQ etc. is appropriately flagged as uncontrolled, so I don't hold that against them, but the DBC baseline substitution is a load-bearing gap that needs a direct test or a careful argument. Also, several stabilization constants (λ, ε, λ_aux, α_MICo, β) are stated, but some appear only in the appendix and the sensitivity to them is not explored. Minor, but for a method whose benefit could be tuning-sensitive, it matters.\n\nOverall: the paper is a competent plug-in method with a meaningful diagnostic and consistent gains in the better-controlled family. The central idea is worth referee time. I would not desk-reject it, but I would ask for DBC-Det-vs-DBC equivalence evidence and at least one fully controlled comparison against a strong recent baseline before accepting.\n\nRecommendation: send to peer review, with a request for revision focused on the DBC comparison and hyperparameter sensitivity.","headline":"PAMD is a genuinely useful plug-in distance for bisimulation RL, with a sharp diagnostic, but the DBC-family headline gains rest on a modified baseline whose comparability to original DBC is shaky.","tokens_in":20231,"tokens_out":1111,"would_cite":true,"duration_ms":14572,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing the fixed latent distance in bisimulation-based visual RL with a pair-conditioned positive-definite quadratic form—PAMD—improves final returns and keeps the encoder from being under-trained.","keywords":["bisimulation metrics","representation learning","visual reinforcement learning","metric learning","Mahalanobis distance","state similarity","deep reinforcement learning","positive-definite quadratic form"],"falsifier":"Train a parameter-matched unconstrained pairwise MLP that is forced to depend only on the displacement Δ = z − z′ (e.g., by symmetrizing its inputs and subtracting its value on identical pairs), and run the same frozen-encoder residual diagnostic and DMC tasks. If that displacement-tied MLP drives the frozen residual to the same plateau as PAMD and matches its downstream returns, the paper's claim that the positive-definite quadratic form specifically prevents distance-dominated solutions is falsified; a simpler displacement-tied head would suffice.","tokens_in":19192,"feed_emoji":"📐","tokens_out":6779,"duration_ms":62504,"temperature":0.7,"pith_summary":"The paper argues that the latent-space distance used to embed behavioral state similarity is a first-order design choice in bisimulation-based visual reinforcement learning, not an incidental detail. Fixed global norms such as ℓ1 are too rigid, while an unconstrained pairwise network can drive the metric loss to zero on its own, leaving the encoder weakly shaped. The paper introduces PAMD, a pairwise-conditioned Mahalanobis distance built as the square root of a positive-definite quadratic form on the latent displacement, and shows it occupies the useful middle: expressive per pair, yet structured so the fixed-point objective can only be satisfied by actually changing the representation. Plugging PAMD into existing DBC and MICo/SimSR pipelines consistently improves final returns on pixel-based DMControl tasks, and a frozen-encoder diagnostic attributes this to preserved learning pressure on the encoder. A careful reader would care because it pinpoints the latent geometry—not just the behavioral target—as a controllable lever in representation learning.","feed_headline":"A learned pair-wise metric lifts visual-RL returns","feed_subtitle":"Bisimulation methods improve when the latent distance adapts per state pair yet still forces the encoder to do the learning.","key_machinery":"The central object is the pairwise-conditioned positive-definite quadratic form d_θ(z,z′) = √(Δᵀ G̃_θ(z,z′) Δ + ε). G̃_θ(z,z′) is constructed by an MLP that takes [z;z′], outputs the lower-triangular entries of a Cholesky factor L, forms G₀ = LLᵀ, symmetrizes as G = G₀(z,z′) + G₀(z′,z), adds a ridge λI, and applies per-pair trace normalization. This single construction guarantees symmetry, nonnegativity, and self-zero by design, and it ties the distance to the relative displacement rather than to the absolute positions of the pair. The work it does is to keep the distance expressive—pair-dependent anisotropy, including off-diagonal cross-coordinate terms—while making the metric loss solvable","core_discovery":"PAMD parameterizes the latent dissimilarity as d_θ(z,z′) = √(Δᵀ G̃_θ(z,z′) Δ + ε) with Δ = z − z′. The matrix G̃ is built by an MLP that maps the pair to the lower-triangular entries of a Cholesky factor, forming a positive-definite matrix, symmetrizing it over the pair order, adding a ridge, and trace-normalizing per pair to fix scale. The claim is that this displacement-tied, positive-definite structure prevents the distance head from absorbing the bisimulation supervision by itself: with the encoder frozen, the residual of the one-step fixed-point equation plateaus above zero, whereas an unconstrained MLP distance of matched size drives it near zero; with a trainable encoder, PAMD's resid","pith_inferences":["The same frozen-encoder diagnostic could be used as a cheap architectural screening test for any similarity-based representation objective: measure how much residual an architecture can fit without changing the encoder, before committing to a full RL run.","If the mechanism is correct, part of the performance differences among existing bisimulation methods may be attributable to their latent comparators rather than their behavioral targets; re-running them under a common adaptive comparator would separate target choice from embedding geometry.","The dense Cholesky construction scales as O(d³) with latent dimension, so the observed gains may shrink in higher-dimensional latent spaces; low-rank or structured-sparse variants would test whether the essential benefit is the displacement-tied PD form or full per-pair anisotropy."],"forward_implications":["The same fixed-point targets used by DBC, MICo, and SimSR become reachable with higher final returns when the latent comparator is PAMD, so any bisimulation-style objective can adopt the module without re-deriving its target distance.","The frozen-encoder residual test offers a mechanistic explanation of why an unstructured distance head underperforms: low residual with a frozen encoder means the distance alone satisfies the objective, weakening gradient pressure on the encoder.","Removing trace normalization degrades performance, indicating that per-pair scale control is part of what prevents degenerate solutions in learned quadratic-form distances.","Diagonal and global Mahalanobis variants recover only part of PAMD's gains, suggesting that pair-dependent off-diagonal structure is an active ingredient, not a byproduct.","The contractivity, fixed-point, and value-function properties of the underlying behavioral distances are inherited unchanged, since PAMD modifies only the proxy latent comparator."],"fun_headline_variants":["PAMD: Pairwise adaptive distance boosts visual RL performance","Pairwise adaptive distance improves bisimulation RL agents","Structured adaptive distance lifts visual-RL returns","Plug-in adaptive distance elevates bisimulation RL","Pairwise adaptive distance for bisimulation RL gains"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The controlled DBC comparison assumes that DBC-Det, the deterministic-transition variant that can accept PAMD, is an adequate stand-in for the original probabilistic DBC—because PAMD cannot be inserted into the Gaussian-Wasserstein term without changing the transition-discrepancy estimator—so the reported DBC-family gains rest on the two baselines being broadly comparable.","fun_headline_variants_meta":{"raw":{"variants":["PAMD: Pairwise adaptive distance boosts visual RL performance","Pairwise adaptive distance improves bisimulation RL agents","Structured adaptive distance lifts visual-RL returns","Plug-in adaptive distance elevates bisimulation RL","Pairwise adaptive distance for bisimulation RL gains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000708,"raw_usage":{"total_tokens":3018,"prompt_tokens":728,"completion_tokens":2290,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":2217}},"tokens_in":472,"tokens_out":2290,"duration_ms":16351,"temperature":1.0,"reasoning_tokens":2217,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T16:21:11.354510+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a parameter-matched unconstrained pairwise MLP that is forced to depend only on the displacement Δ = z − z′ (e.g., by symmetrizing its inputs and subtracting its value on identical pairs), and run the same frozen-encoder residual diagnostic and DMC tasks. If that displacement-tied MLP drives the frozen residual to the same plateau as PAMD and matches its downstream returns, the paper's claim that the positive-definite quadratic form specifically prevents distance-dominated solutions is falsified; a simpler displacement-tied head would suffice.","supporting_citations":[],"review_version":1}