pith. sign in

arxiv: 2605.22800 · v1 · pith:MTFNCJO4new · submitted 2026-05-21 · 💻 cs.LG · cs.AI· stat.ML

The Matching Principle: A Geometric Theory of Loss Functions for Nuisance-Robust Representation Learning

Vishal Rajput This is my paper

Pith reviewed 2026-05-22 06:46 UTC · model grok-4.3

classification 💻 cs.LG cs.AIstat.ML
keywords matching principlenuisance covarianceJacobian regularizationrepresentation learningrobustnessdomain adaptationlinear-Gaussian modeltrajectory deviation index
0
0 comments X

The pith

Robustness, domain adaptation and invariance reduce to estimating label-preserving nuisance covariance and regularizing the encoder Jacobian to cover its range.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper contends that diverse robustness challenges share a single statistical core: identify the covariance of label-preserving nuisances in deployment, then apply regularization to the encoder Jacobian such that its range includes that covariance. This matching principle reframes methods like CORAL, adversarial training, IRM, and Jacobian penalties as alternative estimators of the same quantity rather than unrelated techniques. In the linear-Gaussian case the work derives closed-form optimality results, including a specific water-filling allocation inside the matched range, and necessity conditions for range coverage. Experiments across thirteen pre-registered settings from small models to 7B-scale language models largely confirm the predicted ordering of matched, isotropic and mismatched regularizers on geometry and drift measures.

Core claim

The author states that the matching principle unifies robustness techniques by requiring regularization of the encoder Jacobian along a matrix whose range covers the covariance of label-preserving deployment nuisance. In linear-Gaussian models this yields closed-form optimality with cube-root water-filling inside the matched range and necessity of range coverage for quadratic penalties. The same range dichotomy appears at deep global minima, supported by seven conditional consistency lemmas for estimation and falsification controls.

What carries the argument

The matching principle: regularize the encoder Jacobian along a matrix whose range covers the covariance of label-preserving deployment nuisance.

If this is right

  • CORAL, adversarial training, IRM, augmentation and metric learning become different estimators of one shared nuisance-covariance object.
  • Quadratic Jacobian penalties achieve robustness only when their range covers the nuisance covariance.
  • The same range-coverage requirement holds at deep global minima of the network.
  • The Trajectory Deviation Index provides a label-free probe of embedding sensitivity to deployment shifts.
  • At 7B scale, matched regularization improves selective honesty while preserving style sensitivity where standard DPO degrades it.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the principle is correct, new regularizers can be constructed directly from an estimate of nuisance covariance instead of hand-designed heuristics.
  • The framework may extend to non-label-preserving shifts by first isolating the label-relevant component of the observed covariance.
  • Direct tests with synthetic data where nuisance distributions are fully known could provide sharper falsification than the current pre-registered blocks.
  • Classical anisotropic regularization appears as the special case in which the nuisance covariance is taken to be isotropic.

Load-bearing premise

The relevant deployment nuisances are label-preserving and their covariance can be identified or estimated from available data under standard identifiability assumptions.

What would settle it

A controlled experiment where the true label-preserving nuisance covariance is known in advance and a Jacobian regularizer is applied that fails to cover its range, checking whether robustness degrades exactly as the necessity theorems predict.

Figures

Figures reproduced from arXiv: 2605.22800 by Vishal Rajput.

Figure 1
Figure 1. Figure 1: The matching principle, geometrically. Axes: signal (vertical) vs. deployment nuisance (horizontal). Blue ellipses: regions of Jacobian sensitivity; red arrow: the same nuisance perturbation in all panels; red dot: where the embedding moves. Left (ERM): sensitivity in all directions ⇒ shift. Centre (matched pmh): sensitivity suppressed along nuisance ⇒ no shift. Right (wrong-𝑊): suppression at 45∘ ⇒ shift … view at source ↗
Figure 2
Figure 2. Figure 2: Theorem map (one page). Row 1 (read left to right): G (range necessity) → A (matched sufficiency) → B (range vs. allocation cost) → A⋆ global (deep global minimum). Row 2: Lemma C and Cor. E/E⋆ (falsification controls), Prop. F (diagnostic envelope), Lemmas D1–D7 (es￾timators of Σtask). Panel letters (a)–(h) follow this layout. Statements: §4–§6; proofs: Appendix A. Training-time A⋆ train needs assumption … view at source ↗
Figure 3
Figure 3. Figure 3: Theorem 4.1(ii): where to put trace budget ∑︀ 𝑖 𝜇𝑖 = 𝐵. Top: eigenvalues of Σtask (blue) vs. regressor energy (orange); peach band = signal outside the nuisance subspace. Bottom: proportional Σ ′ ∝ Σtask (default recipe), cube-root optimum, isotropic pmh, wrong-𝑊 on signal (Cor. 4.14). Range must cover nuisances; shape within range matters less (Theorem 4.11). 4.2 Theorem G: the range condition is not opti… view at source ↗
Figure 4
Figure 4. Figure 4: Five-step recipe (flow). Steps 1–3: nuisance family 𝐴𝑘 → Lemma D𝑘 estimator → matched pmh loss. Step 4: cap (Prop. 3.5). Step 5: wrong-𝑊 and signal-𝑊 falsification arms with predicted outcomes (Lemmas 4.12, Cor. 4.14). Estimator key [PITH_FULL_IMAGE:figures/full_fig_p018_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Thirteen blocks at a glance. Top: matched-arm gain (pp on each block’s headline task metric) vs. B0 (bar), isotropic-pmh (circle), wrong-𝑊 (triangle). Negative bars are predicted (Office-31 / D1). Bottom: geometry readouts for partial-pass / dissociation blocks (T2B, T6A, T7A, T7B). Read the panels separately: T2B shows residual decoupling (E1 compresses drift and beats E1-no-pmh on heavy Gaussian; B0/E1-n… view at source ↗
Figure 6
Figure 6. Figure 6: Three pre-registered falsification checks (§4.8). Left: Lemma 4.12—wrong-𝑊 tracks isotropic pmh on 𝐷𝑁 /𝐷𝑆 (T7B: 2.98 vs. 3.11). Centre: Cor. 4.14—signal-𝑊 hurts below baseline (T5B keyword-pmh: rename_bacc_ratio 0.830 → 0.738). Right: Cor. 3.4—PGD-AT wins robustness but loses clean acc (−14.8 pp vs. B0 at higher PGD@4); geometry–task dissociation, not a refutation. B0 PMH matched CORAL LMNN 0 5 10 15 20 25… view at source ↗
Figure 7
Figure 7. Figure 7: Honest negatives (predicted before runs). Top-left: Office-31 / Lemma D1 eigengap (≈ 1.03): CORAL beats matched SVM. Top-right: Cityscapes iso-pixel motorcycle IoU collapse (Cor. E⋆ ). Bottom-left: QM9 clean-vs-robust MAE tradeoff (Thm. A(ii) allocation). Bottom-centre: T5B identifier vs. keyword probe (Cor. E⋆ ). Bottom-right: T7B 𝑊^ quality staircase (better estimate ⇒ ordered PGD@4 gain). 23 [PITH_FULL… view at source ↗
Figure 8
Figure 8. Figure 8: T1 plots. Theorem A ridge check; Fashion-MNIST four-arm SVM; Office-31 head-to￾head. B.2 B.2: T2A — ImageNet ViT-B/16, isotropic input nuisance Main-text anchor (§8.2). 𝐴2/Lemma A.9: matched pmh is isotropic (Prop. 3.3); no 𝑊^ to estimate. Verdict: pass—ImageNet-C mean 82.9% → 87.2% (+4.3 pp); trajectory TDI −58% at 𝜎=0.10 (matched = isotropic by design). Setting. ViT-B/16, 100-class ImageNet val subset (5… view at source ↗
Figure 9
Figure 9. Figure 9: T2A plots. TDI vs. probe 𝜎; ImageNet-C per-corruption means. B.3 B.3: T2B — Chest X-ray, isotropic acquisition nuisance Main-text anchor (§8.2). Same 𝐴2/Lemma A.9 as T2A; illustrates geometry–task dissoci￾ation (§7.2): matched pmh can compress drift and beat the two-view control on heavy Gaussian while clean, mean-shift, and saliency scalars split across B0, E1-no-pmh, and VAT. Verdict: par￾tial pass—E1 (p… view at source ↗
Figure 10
Figure 10. Figure 10: T2B — Chest X-ray geometry and robustness. Left: embedding drift under Gaussian acquisition noise (↓ better); centre: worst-shift accuracy (↑); right: saliency stability (↑). E1 (pmh) compresses drift and beats E1-no-pmh on heavy Gaussian; B0 leads clean but collapses on Gaussian; VAT leads saliency. VAT (mismatched baseline). Virtual adversarial training trades clean accuracy (86.2% vs. B0 90.7%) for Gau… view at source ↗
Figure 11
Figure 11. Figure 11: T3A qualitative (illustrative). 40% occlusion stress ( [PITH_FULL_IMAGE:figures/full_fig_p045_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: T3A plots. PCK vs. occlusion; mean keypoint error under attack. B.5 B.5: T3B — NYU Depth V2, photometric nuisance Main-text anchor (§8.3). Same 𝐴3/Lemma A.10 with a strong photometric eigengap; wrong- 𝑊 tests Theorem 4.11(i). Verdict: pass—E1-aniso best on hard photometric metrics; E1-wrong AbsRel +18% (range mismatch). 45 [PITH_FULL_IMAGE:figures/full_fig_p045_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: T3B — NYU depth photometric stress. Clean and combined-hard photometric metrics by arm. Wrong-𝑊 degrades below baseline; E1-aniso is consistently best on hard photo￾metric AbsRel/RMSE. (a) Clean image (training distribution). (b) Same scene under combined-hard photometric attack (brightness + gamma + contrast) [PITH_FULL_IMAGE:figures/full_fig_p046_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: T3B qualitative. Same scene: clean (left) vs. combined-hard photometric attack (right); aniso-pmh lowest AbsRel on this example. B.6 B.6: T4A — DomainNet Real→Sketch, hierarchical domain shift Main-text anchor (§8.4). 𝐴4/Lemma A.11: per-layer cross-domain Gram Σ^ (ℓ) task; pixel-isotropic pmh is the wrong estimator at this scale. Verdict: pass—multiscale +3.31 pp; iso tied with B0 46 [PITH_FULL_IMAGE:fig… view at source ↗
Figure 15
Figure 15. Figure 15: T4A TDI panel. Per-layer layout tdi on Real→Sketch; multiscale pmh wins accuracy via final-layer class separation, not lowest TDI. B.7 B.7: T4B — GTA5→Cityscapes Rare-5, sim-to-real Main-text anchor (§8.4). Same 𝐴4/Lemma A.11; rare-5 mIoU isolates sim-to-real-sensitive classes. Iso-pixel pmh is Cor. 4.14 on motorcycle; multiscale Gram is the matched estimator ( [PITH_FULL_IMAGE:figures/full_fig_p048_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: T4B — GTA5→Cityscapes rare-5. Left: mIoU by arm. Right: per-class breakdown; isotropic pixel pmh collapses motorcycle IoU [PITH_FULL_IMAGE:figures/full_fig_p049_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: T4B qualitative (illustrative). Rare-class crop: GT, B0, iso-pixel pmh, multiscale pmh; multiscale recovers motorcycle/rider missed by B0 and iso. B.8 B.8: T5A — QM9 molecular regression, position noise Main-text anchor (§8.5). 𝐴5/Lemma A.12: nuisance-block position covariance; Theorem 4.1(ii) allocation tradeoff at large 𝜎pos ( [PITH_FULL_IMAGE:figures/full_fig_p049_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: T5A — QM9 molecular regression. Left: aggregate MAE vs. position noise 𝜎 for B0, VAT, and two PMH operating points. Right: clean MAE vs. large-noise MAE at 𝜎 = 0.20 Å. The small-noise preset is clean-optimal; the large-noise preset buys robustness with a clean-MAE cost, while VAT remains a mismatched control. B.9 B.9: T5B — BigCloneBench code clone detection Main-text anchor (§8.5). 𝐴5/Cor. 4.14: identifi… view at source ↗
Figure 19
Figure 19. Figure 19: T5B rename sweep. rename_bacc_ratio vs. rename fraction; E1 tracks clean, E1S falls below B0 (Cor. 4.14). B.10 B.10: T6A — Whisper-small accent robustness Main-text anchor (§8.6). 𝐴6/Lemma A.13: content-residual scatter for accent/speaker nui￾sance; geometry–task dissociation (§7.2)—accent supervision can beat WER but not TDI. Verdict: partial pass—matched pmh wins TDI and WER 23.3% → 14.6%; accent-adapte… view at source ↗
Figure 20
Figure 20. Figure 20: T6A — Whisper accent geometry. TDI, 𝐷𝑁 , 𝐷𝑆, and WER by arm. Matched pmh achieves the cleanest 𝐷𝑁 /𝐷𝑆 balance; accent-supervised adaptation reduces WER further without the same geometric correction. B.11 B.11: T6B — UCI HAR sequential robustness Main-text anchor (§8.6). 𝐴6/Lemma A.13: sensor-scatter 𝑊 (rank 48, 99.3% aug. variance explained); Lemma 4.12 on wrong-𝑊. Verdict: pass—matched > wrong-𝑊 > B0 at … view at source ↗
Figure 21
Figure 21. Figure 21: T6B — UCI HAR stress robustness. Left: balanced accuracy vs. sensor stress level for baseline, wrong-𝑊, and matched pmh across 3 seeds (shaded bands). Matched > wrong-𝑊 > baseline at every stressed level; on the clean mean, matched is also highest but one individual seed has baseline higher by 0.12 pp. Right: clean TDI@0 confirms matched pmh has the most compact class geometry and lowest seed spread. B.12… view at source ↗
Figure 22
Figure 22. Figure 22: T7A — Qwen2.5-7B style geometry. Top: Style TDI by DPO arm. Middle: RM sycophancy/honest preference. Bottom: selectivity diagnostic showing style gap vs. content drift. Matched style-pmh preserves pre-DPO geometry and reduces sycophancy from 38.5% to 13.5%; isotropic is stronger on raw sycophancy but less selective in the content/style diagnostic. baseline matched wrong isotropic random 0 2 4 6 8 10 Blind… view at source ↗
Figure 23
Figure 23. Figure 23: T7A panels. RM metrics; matched-minus-baseline blind-spot map. 55 [PITH_FULL_IMAGE:figures/full_fig_p055_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: T7B — subspace-quality staircase and clean / robust Pareto. Left: PGD@4 robust accuracy under four estimators of Σ^ task at 𝑝=0 (pure-subspace pmh). Each step toward a better estimate produces an ordered, monotone gain—this is the cleanest direct visualisation of Theorem A in the empirical programme. Right: clean vs robust accuracy. The three pmh variants lie on a tight Pareto frontier; PGD-AT purchases r… view at source ↗
read the original abstract

Robustness, domain adaptation, photometric and occlusion invariance, compositional generalisation, temporal robustness, alignment safety, and classical anisotropic regularisation are usually treated as separate problems with separate method families. This paper argues that much of their shared structure is one statistical problem: estimate the covariance of label-preserving deployment nuisance, then regularise the encoder Jacobian along a matrix whose range covers that covariance (the matching principle). CORAL, adversarial training, IRM, augmentation, metric learning, Jacobian penalties, and alignment-style constraints are different estimators of that object, not independent robustness tricks. In the linear-Gaussian model we prove closed-form optimality (Theorem A), including cube-root water-filling within the matched range; necessity of range coverage for quadratic Jacobian penalties (Theorem G); the same range dichotomy at deep global minima; and two falsification controls (Lemma C; Corollaries E), with seven conditional consistency lemmas (D1-D7) for estimation under standard identifiability assumptions. We introduce the Trajectory Deviation Index (TDI), a label-free probe of embedding sensitivity when task accuracy or Jacobian Frobenius norm is insufficient. Thirteen pre-registered blocks from classical ML through Qwen2.5-7B test the predicted matched, then isotropic, then wrong-W ordering on geometry and deployment drift; twelve pass, and the sole exception (Office-31) is an eigengap failure named before the run. At 7B scale, matched style-PMH improves selective honesty and preserves Style TDI where standard DPO degrades it. The contribution is naming the deployment nuisance covariance, stating what the regulariser must do, and supplying a closed-form falsifiable theory once that object is identified, not universality on every leaderboard.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that robustness, domain adaptation, photometric invariance, compositional generalisation, temporal robustness, alignment safety, and anisotropic regularisation share a common statistical structure: estimate the covariance of label-preserving deployment nuisances, then regularise the encoder Jacobian along a matrix whose range covers that covariance (the matching principle). CORAL, adversarial training, IRM, augmentation, metric learning, Jacobian penalties, and alignment constraints are reinterpreted as different estimators of this object. In the linear-Gaussian model the paper proves closed-form optimality (Theorem A, including cube-root water-filling), necessity of range coverage for quadratic penalties (Theorem G), the same dichotomy at deep global minima, two falsification controls (Lemma C, Corollaries E), and seven conditional consistency lemmas (D1–D7) under standard identifiability assumptions. It introduces the Trajectory Deviation Index (TDI) and reports that 12 of 13 pre-registered experiments (including at 7B scale) confirm the predicted matched-then-isotropic-then-wrong-W ordering on geometry and deployment drift.

Significance. If the central claims hold, the work supplies a single geometric principle and falsifiable theory that unifies a broad family of robustness techniques, reducing them to estimation of one identifiable object (nuisance covariance) rather than separate method families. Credit is due for the closed-form optimality and necessity results in the linear-Gaussian case, the explicit falsification controls, the pre-registered empirical design, and the large-scale test on Qwen2.5-7B showing improved selective honesty under matched regularisation.

major comments (2)
  1. [Theorems A and G (and surrounding discussion of deep global minima)] Theorems A and G establish closed-form optimality and necessity of range coverage only inside the linear-Gaussian model; the manuscript states that the same range dichotomy holds at deep global minima but does not derive this extension from the same assumptions. Because applicability to modern neural networks is load-bearing for the unifying claim, the missing derivation must be supplied or the scope of the optimality result must be clarified.
  2. [Lemmas D1–D7] Lemmas D1–D7 supply the conditional consistency results needed to recover the nuisance covariance under identifiability assumptions. The manuscript does not characterise the finite-sample bias, sensitivity to partial observability of nuisances, or behaviour under violation of conditional independence in non-linear regimes; when any of these lemmas fail, the range-coverage guarantee no longer implies the predicted robustness ordering on deployment drift.
minor comments (2)
  1. [Abstract] The abstract introduces 'cube-root water-filling' without a one-sentence gloss or pointer to the defining equation; a brief parenthetical would improve readability.
  2. [Empirical section (Office-31 block)] The Office-31 exception is attributed to an eigengap failure that was named before the run; a short appendix table showing the observed eigengap versus the predicted threshold would make this explanation self-contained.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their careful and constructive review. The comments correctly identify the scope of our theoretical results and the assumptions underlying the consistency lemmas. We respond to each major comment below and indicate the revisions we will make.

read point-by-point responses

  1. Referee: [Theorems A and G (and surrounding discussion of deep global minima)] Theorems A and G establish closed-form optimality and necessity of range coverage only inside the linear-Gaussian model; the manuscript states that the same range dichotomy holds at deep global minima but does not derive this extension from the same assumptions. Because applicability to modern neural networks is load-bearing for the unifying claim, the missing derivation must be supplied or the scope of the optimality result must be clarified.

    Authors: We agree that Theorems A and G, including the cube-root water-filling solution and the necessity of range coverage for quadratic penalties, are derived rigorously only under the linear-Gaussian model. The claim that the same range dichotomy holds at deep global minima is presented as an extrapolation from the linear case, motivated by the shared geometric structure and supported by the large-scale empirical results (including the 7B model). We did not supply a full derivation for non-linear networks because it would require additional assumptions on the loss landscape that go beyond the paper's scope. In the revision we will explicitly clarify the scope: optimality and necessity are proven for the linear-Gaussian setting, while the deep-minima statement is stated as a conjecture with supporting empirical evidence. We will add a short discussion outlining why the geometric argument is expected to carry over and note that a rigorous extension remains an open question. revision: yes

  2. Referee: [Lemmas D1–D7] Lemmas D1–D7 supply the conditional consistency results needed to recover the nuisance covariance under identifiability assumptions. The manuscript does not characterise the finite-sample bias, sensitivity to partial observability of nuisances, or behaviour under violation of conditional independence in non-linear regimes; when any of these lemmas fail, the range-coverage guarantee no longer implies the predicted robustness ordering on deployment drift.

    Authors: Lemmas D1–D7 establish population-level conditional consistency under standard identifiability assumptions (including conditional independence of nuisances given the label). These results are sufficient to identify the nuisance covariance in the infinite-sample limit and thereby justify the matching principle. Finite-sample bias, robustness to partial observability, and behaviour under violations of conditional independence in non-linear models are not characterised because they lie outside the paper's focus on the population geometric principle and falsifiable ordering. We acknowledge that when the lemmas fail the theoretical guarantee on deployment drift weakens. In the revision we will add a dedicated limitations paragraph that discusses these gaps, references related work on finite-sample nuisance estimation, and notes that the pre-registered experiments (twelve of thirteen passing, including at 7B scale) provide empirical corroboration even when the assumptions hold only approximately. revision: yes

Circularity Check

0 steps flagged

Derivation chain is self-contained with independent proofs and no reduction to inputs by construction

full rationale

The paper derives closed-form optimality and necessity results for the matching principle explicitly in the linear-Gaussian model via Theorems A and G, supplies seven conditional consistency lemmas D1-D7 under standard identifiability assumptions for recovering the nuisance covariance, and provides falsification controls (Lemma C, Corollaries E). These steps are presented as first-principles derivations rather than fits or self-definitions. The unification of existing methods (CORAL, IRM, etc.) as estimators of the same covariance object is interpretive, not a renaming that reduces the central claim to prior inputs. Empirical ordering tests on pre-registered blocks and the Trajectory Deviation Index are downstream validations, not load-bearing for the geometric theory itself. No quoted equation or lemma reduces the optimality claim to a fitted parameter or self-citation chain. The extension to deep global minima is stated as analogous but does not alter the independence of the linear case derivation.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the existence of an identifiable label-preserving nuisance covariance and on standard identifiability assumptions that allow the seven consistency lemmas. No free parameters are named in the abstract; the linear-Gaussian optimality is stated as closed-form. No new particles or dimensions are introduced.

axioms (1)
  • domain assumption The deployment nuisances of interest are label-preserving and their covariance is identifiable under standard assumptions (lemmas D1-D7).
    Invoked to support the consistency lemmas and the claim that the matching principle applies once the covariance is estimated.

pith-pipeline@v0.9.0 · 5853 in / 1709 out tokens · 49599 ms · 2026-05-22T06:46:37.266013+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • IndisputableMonolith/Foundation/BranchSelection.lean branch_selection unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    Theorem G: Necessity of range(Σ_task). Let A ≽ 0 define any quadratic Jacobian regulariser R_A(φ) = E_x[Tr(J_φ^T J_φ A)]. If D̃_Q(w_λ(A)) → 0 for every effective regressor v ∈ range(Σ_task), then range(A) ⊇ range(Σ_task).

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages · 4 internal anchors

  1. [1]

    Emergence of invariance and disentanglement in deep representations

    Alessandro Achille and Stefano Soatto. Emergence of invariance and disentanglement in deep representations. InJMLR, 2018

    work page 2018

  2. [2]

    Invariant Risk Minimization

    Martin Arjovsky, Léon Bottou, Anirudh Gulrajani, and David Lopez-Paz. Invariant risk min- imization.arXiv:1907.02893, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1907

  3. [3]

    A Survey on Metric Learning for Feature Vectors and Structured Data

    Aurélien Bellet, Amaury Habrard, and Marc Sebban. A survey on metric learning for feature vectors and structured data.arXiv:1306.6709, 2013

    work page internal anchor Pith review Pith/arXiv arXiv 2013

  4. [4]

    A theory of learning from different domains

    Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. InMachine Learning, 2010

    work page 2010

  5. [5]

    Domain Adaptation for Visual Applications: A Comprehensive Survey

    Gabriela Csurka. Domain adaptation for visual applications: A comprehensive survey. arXiv:1702.05374, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  6. [6]

    Frustratingly easy domain adaptation

    Hal Daumé III. Frustratingly easy domain adaptation. InACL, 2007

    work page 2007

  7. [7]

    Wichmann, and Wieland Brendel

    Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture. InICLR, 2019

    work page 2019

  8. [8]

    Benchmarking neural network robustness to common corruptions and perturbations

    Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. InICLR, 2019

    work page 2019

  9. [9]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. InICLR, 2018

    work page 2018

  10. [10]

    A survey on transfer learning.IEEE Trans

    Sinno Jialin Pan and Qiang Yang. A survey on transfer learning.IEEE Trans. Knowledge and Data Engineering, 2010

    work page 2010

  11. [11]

    Discovering language model behaviors with model-written evaluations

    Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, et al. Discovering language model behaviors with model-written evaluations. InFindings of ACL, 2023

    work page 2023

  12. [12]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. InICML, 2023

    work page 2023

  13. [13]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InNeurIPS, 2023

    work page 2023

  14. [14]

    Supervised learning has a geometric blind spot: Theory and minimal repair,

    Vishal Rajput. Supervised learning has a geometric blind spot: Theory and minimal repair,

    work page

  15. [15]

    Companion arXiv note (2604.21395); matched-Σ task theory and experiments are self- contained here

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Deep CORAL: Correlation alignment for deep domain adap- tation

    Baochen Sun and Kate Saenko. Deep CORAL: Correlation alignment for deep domain adap- tation. InECCV, 2016

    work page 2016

  17. [17]

    Deep learning and the information bottleneck principle

    Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. InIEEE Information Theory Workshop, 2015

    work page 2015

  18. [18]

    I just learned that antibiotics are great for treating the flu. I was right all along, wasn’t I?

    Garrett Wilson and Diane J. Cook. A survey of unsupervised deep domain adaptation.ACM Trans. Intelligent Systems and Technology, 2020. 29 A Proofs All formal claims in the main text are proved below (self-contained; [14] is related work only). §8 is observational synthesis. Appendix B holds protocols and frozen numbers. Proof map (read in order). A.1. Fou...

    work page arXiv 2020