Pith. sign in

REVIEW 4 major objections 4 minor 49 references

OBSER: Object-Based Sub-Environment Recognition for Zero-Shot Environmental Inference

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that modeling a sub-environment as a distribution over its objects enables zero-shot inference, and that this object-based view outperforms scene-based CLIP on unseen rooms.

desk verdict A promising object-distribution framework for zero-shot environment recognition with a strong Replica result, but the central proof that EDS optimization drives convergence does not go through. read the letter →

arxiv 2507.02929 v1 pith:FMVHHN7N submitted 2025-06-26 cs.CV cs.AIcs.LGstat.ML

Won-Seok Choi , Dong-Sig Han , Suhyung Choi , Hyeonseo Yang , Byoung-Tak Zhang This is my paper
classification cs.CVcs.AIcs.LGstat.ML
keywords zero-shotenvironmentalinferenceobject-basedsub-environmentrecognitionkerneldensityestimationEDSfunctionself-supervisedlearningmetricchainedretrievalepisodicmemory
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that recognizing where an agent is should start not from the whole scene but from the objects in it: a sub-environment is modeled as a probability distribution over the object classes that occur there. On that footing it builds three measures — object-to-object similarity, object occurrence in an environment, and environment-to-environment KL divergence — all estimated by kernel density on a learned latent space, and it introduces the $(\epsilon, \delta)$ statistically separable (EDS) function as a certificate of when those estimates can be trusted. The theoretical core proves that as a feature extractor becomes concentrated and separable ($\delta \to 1$, $\epsilon \to 0$), the estimated object occurrence and the estimated KL divergence converge to their exact values, with the divergence bound $\log(1 + (|C|-1)/k)$ in $k = \delta/\epsilon$. Empirically, chaining these measures through an episodic memory retrieves objects in Minecraft and in photo-realistic Replica rooms, and in unseen rooms it reaches 0.78 top-1 room accuracy with DINO-v2 features versus 0.30 for scene-based CLIP.

What carries the argument

The $(\epsilon, \delta)$ statistically separable (EDS) function is the load-bearing device: a feature extractor $f$ is $(\epsilon, \delta)$ statistically separable when, $\mu$-almost everywhere, the kernel density of an object against its own class distribution is at least $\delta$ (concentration) and against any other class's distribution is at most $\epsilon$ (separability), with $0 \le \epsilon \le \delta \le 1$. With the kernel $\phi_f(x, x') = \exp((f(x)^\top f(x') - 1)/\tau)$ on a hypersphere, the membership estimate becomes the Bayesian classifier $\omega(c)\Phi_f(x;\rho_c)/\Phi_f(x;\mu)$, and the argument runs as follows: Theorem 1 bounds $\mathrm{KL}(\mu \| \hat{\mu}_f) \le \log(1 + (|C|-1)/k)$ with $k = \delta/\epsilon$, so the bound tightens only as $\delta \to 1$ and $\epsilon \to 0$; Corollary 1 adds that a compact embedding space is necessary, since in Euclidean space $\delta$ cannot reach 1; Lemma 2 and Theorem 2 then push the estimated object occurrence and estimated KL divergence onto their exact values.

What would settle it

Take one object class that appears in two visually distinct sub-environments, such as the same furniture class in differently lit rooms of the Replica data or the same plant in different Minecraft biomes, and estimate the class-conditional distributions $\rho_c$ in each. Then compute the exact KL divergence from ground-truth labels and compare it with the estimate $\widehat{\mathrm{KL}}_f(\mu\|\nu)$: if the within-class shift across environments approaches the between-class separation, the estimate should deviate from the exact value by more than the bound in Theorem 2, falsifying the claim that the measures converge.

Watch

Extended reading notes

Core claim

The central claim is that sub-environment recognition is an inductive process rather than a scene-classification problem: a sub-environment is a mixture $\mu(x) = \sum_{c \in \mathcal{C}} \omega(c)\rho_c(x)$ of class-wise object distributions, with $\omega(c)$ recording how often class $c$ occurs and $\rho_c$ the class-conditional distribution, so environments differ chiefly in $\omega$. Under Assumption 2 — that $\rho_c$ is consistent across environments — the KL divergence between two sub-environments reduces to $\sum_{c} \omega_\mu(c) \log(\omega_\mu(c)/\omega_\nu(c))$, a comparison of object-occurrence ratios alone. The paper derives a Bayesian classifier from kernel density estimates and proves (Theorem 1) that its KL divergence from the true distribution is bounded by $\log(1 + (|C|-1)/k)$ with $k = \delta/\epsilon$, so optimizing the feature extractor drives $\delta \to 1$ and $\epsilon \to 0$; from there, Lemma 2 and Theorem 2 establish that the estimated object occurrence $\hat{\omega}_f(c)$ and the estimated divergence $\widehat{\mathrm{KL}}_f(\mu\|\nu)$ converge to their exact values. The empirical discovery is that chaining the three measures through an episodic memory — recall a room by object occurrence, find similar rooms by KL divergence, retrieve the object by kernel similarity — gives strong zero-shot object retrieval, with DINO-v2 reaching 0.78 top-1 room accuracy on unseen Replica rooms versus 0.30 for CLIP, and the framework still sustains inference when segmentations come from SAM2 instead of ground truth.

Load-bearing premise

Everything rests on the assumption that each object class has the same feature distribution in every environment, so that environments differ only in how often each class occurs; the paper offers no direct test of that assumption, and if it fails the KL-divergence estimates inherit a systematic bias.

Editorial extensions

If this is right

  • An agent can chain the three measures through an episodic memory — recall the most probable room by object occurrence, retrieve the most similar unseen room by KL divergence, and locate the query object there — without language labels or retraining on the target environment.
  • The performance gap in unseen rooms is the headline: scene-based CLIP reaches 0.30 top-1 room accuracy while OBSER with DINO-v2 reaches 0.78, indicating that object distributions transfer across environments where scene statistics do not.
  • When a feature extractor has high concentration and separability, the estimated object occurrence and environment-to-environment KL divergence are provably close to their exact values, so the EDS function doubles as a certificate of correctness for the measures.
  • Replacing ground-truth segmentation with SAM2 segmentation preserves sufficient inference performance, so the framework can run fully unsupervised.
  • A threshold on the estimated KL divergence along an agent's trajectory detects meaningful changes in circumstances, enabling the agent to segment the trajectory and build episodic memory online.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the convergence theory ties estimation error to $(1-\delta)$ and $\epsilon$, measuring the EDS values on a new domain should predict chained-retrieval accuracy before the full task is run; this is a testable shortcut for deciding when to trust the framework.
  • If Assumption 2 is violated, no better feature extractor can fix the resulting bias; a concrete check is to measure the same object class's distribution $\rho_c$ in several environments and correlate the cross-environment shift with the observed estimation error.
  • The framework treats any stream of things with stable per-class distributions as objects, so the same three measures could apply to audio or text event streams for zero-shot context recognition, not only to visual scenes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes OBSER, an object-based Bayesian framework for zero-shot sub-environment recognition. It defines three measures: object-object similarity via kernel density, object occurrence for a query, and KL divergence between sub-environments, all estimated from empirical kernel densities on metric/self-supervised embeddings. To validate these measures, the paper introduces the (epsilon, delta) statistically separable (EDS) function and claims that optimizing the EDS function in a compact latent space drives delta to 1 and epsilon to 0, which would guarantee convergence of the estimated object occurrence and KL divergence to their exact values. Empirical validation is provided on ImageNet, a Minecraft-derived open-world dataset, and the photorealistic Replica environment, where OBSER with DINO-v2 is reported to outperform CLIP in chained retrieval in unseen rooms.

Significance. If the central claims hold, the paper contributes a conceptually interesting direction: replacing scene-based environment recognition with distributional object-occurrence inference that can be chained from memory. The empirical study on Replica and the comparison with CLIP are potentially useful for embodied agents, and the inclusion of pseudocode and detailed experiment descriptions is a strength. The reported results in the unseen-room setting (DINO-v2 Top-1 0.78 vs. CLIP 0.30) are striking. However, the theoretical validation is the load-bearing part of the paper's claim that the measures 'converge to their exact values,' and that validation is not currently sound; the empirical sections also lack error bars and leave key hyperparameters underspecified.

major comments (4)
  1. [Section 3.3, Assumption 2] The proof of Theorem 1 in Eq. (36) is invalid: it uses the lower bound delta <= Phi_f(x; rho_c) in a place where an upper bound is required. The expression log(omega(c)*delta + epsilon*(1-omega(c))) is not an upper bound on log Phi_f(x; mu), because Definition 1 only guarantees delta <= Phi_f(x; rho_c) <= 1, so the correct upper bound for the same-class term would use 1, not delta. Even if that inequality is repaired, the theorem only yields k = delta/epsilon -> infinity when Delta H -> 0; it does not imply delta -> 1 and epsilon -> 0. Corollary 1 asserts that compactness makes k tend to k_max and delta -> 1, but this does not follow: k -> k_max only says delta/epsilon -> 1/phi_min, and one can have epsilon = 0 with delta < 1, which gives k = infinity and Delta H = 0 without delta -> 1. Concretely, with two balanced classes, a fixed same-class density delta < 1, and zero cross-class density, Lemma 2's conclusion hat(omega)(c) -> omega(c) fails because hat(omega)(c) = Phi(x; mu) = delta * omega(c). Consequently, the convergence results in Sections 5.2 and 5.3 (Lemma 2 and Theorem 2) are not established as stated. The authors should either give a correct proof that optimizing the proposed objective drives delta -> 1, or explicitly state delta -> 1 and epsilon -> 0 as assumptions of the convergence lemmas and restrict the theoretical claims accordingly.
  2. [Tables 1 and 2, Section 6] Assumption 2 states that the class-wise object distribution rho_c is consistent in all sub-environments. This assumption is load-bearing for the derivation of Eq. (9) and for the identification of hat(omega)_f(c) with the true object occurrence. The paper provides no empirical check of this assumption in the Minecraft or Replica experiments. For example, a 'cup' in a kitchen and a 'cup' in a laboratory may have different visual feature distributions due to context, lighting, or viewpoint; if this happens, the kernel density estimates are biased and the claimed convergence to the exact values is not meaningful. Please add a quantitative test (e.g., comparing per-class kernel densities across rooms or biomes, or evaluating retrieval accuracy separately for classes with high cross-environment variability) or state clearly how violations of Assumption 2 affect the theorem bounds.
  3. [Section 5.5 and Appendix C.2] The main empirical results are reported without error bars or significance tests. Table 2 is based on a small set of 10 query objects and appears to be a single evaluation run. The difference between DINO-v2 (0.78) and MoCo-v3 (0.50) in the Unseen Top-1 condition is large, but without a variance estimate it is impossible to rule out sensitivity to the particular query set; notably the Top-3 values for DINO-v2 and MoCo-v3 are identical (0.78). Table 1 also lacks variance information. Please report mean and standard deviation over at least three random seeds or bootstrap confidence intervals, and specify exactly which query objects and rooms are used for each reported accuracy.
  4. [Appendix D/E] The EDS values and the downstream conclusions are strongly sensitive to the kernel temperature tau, as shown in Figure 12, and the threshold multiplier in Algorithm 1 is introduced with a default value (0.25) but no stated selection principle. The paper uses different tau values across experiments without a systematic rule, making it difficult to assess whether the reported 'optimized EDS' is a property of the representation or an artifact of hyperparameter choice. Please add a sensitivity analysis over tau and the threshold multiplier, or provide a principled criterion for selecting these parameters, to support the robustness claims in Sections 5.5 and 6.
minor comments (4)
  1. [Algorithm 1] The word 'deacreasing' should be 'decreasing', and the notation phi_min is used before its definition is clear.
  2. [Table 2 caption] The pseudocode variable names such as 'mean mu matrix' and 'mean query matrix' contain spaces, which is not valid Python; use underscores for readability and executability.
  3. [Abstract] The caption contains a typo: 'retrived' should be 'retrieved'.
  4. [References] The abstract says 'metric and self-supervised learning models' estimate the object distributions, but the experiments use specific pretrained models (SupCon, MoCo, SimCLR, DINO); consider making the claim more precise to avoid overgeneralization.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the EDS convergence results are conditional consistency statements, and the empirical benchmarks are independent; the main flagged issue is an unproven convergence assumption in Corollary 1, not a circular reduction.

full rationale

OBSER's theoretical chain is not circular. Definition 1 introduces the (epsilon, delta) EDS conditions via the same kernel density Phi_f that is later used in the object-occurrence and KL estimates, but Lemma 2 and Theorem 2 are explicitly conditional: they show hat_omega -> omega and cKL -> KL under the hypotheses delta -> 1 and epsilon -> 0. Those hypotheses are not the conclusions; they are independent regularity conditions on the feature extractor. The paper does not fit omega or KL to data and then rename the fitted value as a prediction. The ImageNet, Minecraft, and Replica experiments use held-out or unseen settings and compare against CLIP, giving independent empirical grounding for the framework. The genuine weakness is in the optimization-to-EDS step: Corollary 1 (Appendix A.2) concludes delta -> 1 and epsilon -> phi_min from Delta_H -> Delta_H_min, but it assumes the KL upper bound reaches its minimum; the proof that training attains Delta_H_min is omitted. This is a rigor or correctness gap, not a circular reduction, and the skeptic's epsilon = 0 counterexample falls outside the compact positive-kernel regime where phi_min > 0. Self-citations such as [16] appear only in related work and are not load-bearing. Overall, the derivation is self-contained as a conditional consistency argument, with an unproven convergence assumption but no circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two domain assumptions (mixture form, cross-environment consistency of class distributions) plus standard KDE consistency. The framework introduces the EDS function as a diagnostic, not a new physical entity. Four free parameters (tau, threshold, outlier trim, query count) are chosen by hand or by performance, affecting the reported measures.

free parameters (4)
  • Kernel temperature tau = 0.2 (Minecraft); 0.12-0.18 chosen by performance (Sec 5.5); multiple values in ImageNet
    Controls kernel width; authors report tau near [0.12, 0.18] performs best, indicating selection on the evaluation data.
  • Threshold multiplier for object occurrence = 0.25
    Algorithm 1 uses a hand-set threshold to quantize mean kernel density; the paper says precision significantly improves, implying the value was tuned.
  • Outlier trimming fraction in EDS computation = 5%
    Appendix C.2.2 removes approximately 5% of data before aggregating kernel density, a post-hoc exclusion that affects EDS values.
  • Number of query observations k for object-environment = not reported
    The framework uses multiple queries for object occurrence; the choice of k is not specified or justified.
assumptions (4)
  • domain assumption Sub-environment is a mixture of class-wise distributions (Assumption 1)
    Environments are modeled as weighted sums of object class distributions; this is a modeling choice, not derived.
  • domain assumption Class-wise object distribution is consistent across sub-environments (Assumption 2)
    The KL divergence formula (Eq. 9) assumes rho_c does not change across environments; if false, the object-based measure conflates object identity with environmental context.
  • standard math Latent classes partition the object domain
    Assumes X is partitioned into disjoint class regions; needed for the Bayesian classifier derivation in Appendix A.1.
  • standard math Kernel density estimation provides consistent estimates of the measures
    The framework relies on KDE convergence, a standard statistical assumption, without finite-sample guarantees.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OBSER: Object-Based Sub-Environment Recognition for Zero-Shot Environmental Inference." pith.science (2026). https://pith.science/paper/FMVHHN7N

@misc{pith2026250702929,
  author       = {Pith},
  title        = {Pith review of: OBSER: Object-Based Sub-Environment Recognition for Zero-Shot Environmental Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FMVHHN7N}},
  note         = {Machine review of arXiv:2507.02929}
}
abstract

We present the Object-Based Sub-Environment Recognition (OBSER) framework, a novel Bayesian framework that infers three fundamental relationships between sub-environments and their constituent objects. In the OBSER framework, metric and self-supervised learning models estimate the object distributions of sub-environments on the latent space to compute these measures. Both theoretically and empirically, we validate the proposed framework by introducing the ($\epsilon,\delta$) statistically separable (EDS) function which indicates the alignment of the representation. Our framework reliably performs inference in open-world and photorealistic environments and outperforms scene-based methods in chained retrieval tasks. The OBSER framework enables zero-shot recognition of environments to achieve autonomous environment understanding.

Figures

Figures reproduced from arXiv: 2507.02929 by the authors.

Figure 1
Figure 1. Comparison between scene-based and object-based [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Visualization of chained inference using the OBSER framework in the Replica environment. (A) Scene observations are gathered [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Radar chart of EDS values (τ = 0.5) and down-stream task accuracies for metric learning and self-supervised learning models. The separability value (1 − ϵ) is reported instead of ϵ for the ease of comparison. Each score is normalized within the interval [0, 1]. The table reported in Sec. C.2.1 shows the original, unnormalized results. Lemma 2. ωˆf (c) converges to ω(c), x ∈ Xc, Theorem 2. KLc f (µ||ν) converges to K… view at source ↗
Figures from the paper (22 more)
Figure 4
Figure 4. Figure 4: Estimated object occurrence using distributions gener [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Difference between the exact KL divergence values and [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Visualization of Minecraft experiments. (a) We first gather a dataset of object observations from various sub-environments. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: Environment-environment retrieval task with Miniature [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Example trajectory (top) with 13 waypoints and a plot [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 11
Figure 11. Figure 11: Visualization of the optimization of the EDS function with (a) Moons dataset. (b-d) The EDS function defined on a hypersphere [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Visualization of EDS values with different temperatures with ImageNet Dataset. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Visualization of object existence probability estimation with various scenarios. Mean classifier used to enhance the estimation [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Visualization of KL divergence estimation with three different scenarios. (a)-(d) correspond to Scenario 1, (b)-(e) to Scenario [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Flowchart illustrating the Minecraft data collection process. A) A human manually collects candidate coordinates for the [PITH_FULL_IMAGE:figures/full_fig_p021_15.png]
Figure 16
Figure 16. Figure 16: Example observations with various objects from different biomes. [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]
Figure 17
Figure 17. Figure 17: Example observations from different biomes. [PITH_FULL_IMAGE:figures/full_fig_p024_17.png]
Figure 18
Figure 18. Figure 18: Visualization of EDS values with different temperatures with Minecraft Dataset. [PITH_FULL_IMAGE:figures/full_fig_p025_18.png]
Figure 19
Figure 19. Figure 19: Visualization of results with object-environment retrieval task in Miniature environment. [PITH_FULL_IMAGE:figures/full_fig_p026_19.png]
Figure 20
Figure 20. Figure 20: MDS visualization of biomes in Miniature environment. Jensen-Shannon divergence (JSD) is used as the measure between [PITH_FULL_IMAGE:figures/full_fig_p027_20.png]
Figure 21
Figure 21. Figure 21: Example scene obsevations and object observations in each room. [PITH_FULL_IMAGE:figures/full_fig_p028_21.png]
Figure 22
Figure 22. Figure 22: Queried obsevations used for the experiments. For some ambiguous queries, multiple classes were accepted as correct (indicated [PITH_FULL_IMAGE:figures/full_fig_p029_22.png]
Figure 23
Figure 23. Figure 23: Visualization of the mutual environment relationships among the rooms in [PITH_FULL_IMAGE:figures/full_fig_p030_23.png]
Figure 24
Figure 24. Figure 24: Visualization of the object-environment relationships between a given query and the rooms in [PITH_FULL_IMAGE:figures/full_fig_p030_24.png]
Figure 25
Figure 25. Figure 25: Visualization of the pairwise KL divergence between two waypoints in different trajectories. By using the circumstance at a [PITH_FULL_IMAGE:figures/full_fig_p031_25.png]
Figure 26
Figure 26. Figure 26: Visualization of the chained inference results using the object distribution extracted from both ground-truth segmentation and [PITH_FULL_IMAGE:figures/full_fig_p032_26.png]
Figure 27
Figure 27. Figure 27: Visualization of the chained inference results using the object distribution extracted from both ground-truth segmentation [PITH_FULL_IMAGE:figures/full_fig_p033_27.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 31 canonical work pages

  1. [1]

    Clip- graphs: Multimodal graph networks to infer object-room affinities

    Ayush Agrawal, Raghav Arora, Ahana Datta, Snehasis Banerjee, Brojeshwar Bhowmick, Krishna Murthy Jataval- labhula, Mohan Sridharan, and Madhava Krishna. Clip- graphs: Multimodal graph networks to infer object-room affinities. In2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 2604–2609. IEEE, 2023. 1, 2

    work page 2023

  2. [2]

    Estimating kullback-leibler divergence using kernel machines

    Kartik Ahuja. Estimating kullback-leibler divergence using kernel machines. In2019 53rd Asilomar Conference on Sig- nals, Systems, and Computers, pages 690–696. IEEE, 2019. 2

    work page 2019

  3. [3]

    A theoretical analysis of contrastive unsupervised representation learning.arXiv preprint arXiv:1902.09229, 2019

    Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saunshi. A theoretical analysis of contrastive unsupervised representation learning.arXiv preprint arXiv:1902.09229, 2019. 3

    arXiv 1902

  4. [4]

    Investigating the Role of Negatives in Contrastive Representation Learning

    Jordan T Ash, Surbhi Goel, Akshay Krishnamurthy, and Dipendra Misra. Investigating the role of negatives in contrastive representation learning.arXiv preprint arXiv:2106.09943, 2021

    work page Pith review arXiv 2021

  5. [5]

    Do more negative samples necessarily hurt in contrastive learn- ing? InInternational conference on machine learning, pages 1101–1116

    Pranjal Awasthi, Nishanth Dikkala, and Pritish Kamath. Do more negative samples necessarily hurt in contrastive learn- ing? InInternational conference on machine learning, pages 1101–1116. PMLR, 2022. 3

    work page 2022

  6. [6]

    Video pretraining (vpt): Learning to act by watching unlabeled online videos.Advances in Neural Information Processing Systems, 35:24639–24654, 2022

    Bowen Baker, Ilge Akkaya, Peter Zhokov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampe- dro, and Jeff Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos.Advances in Neural Information Processing Systems, 35:24639–24654, 2022. 2, 6

    work page 2022

  7. [7]

    Tiago Barros, Ricardo Pereira, Lu ´ıs Garrote, Cristiano Pre- mebida, and Urbano J. Nunes. Place recognition survey: An update on deep learning approaches.CoRR, abs/2106.10458,

    work page Pith review arXiv

  8. [8]

    Data vi- sualization with multidimensional scaling.Journal of com- putational and graphical statistics, 17(2):444–472, 2008

    Andreas Buja, Deborah F Swayne, Michael L Littman, Nathaniel Dean, Heike Hofmann, and Lisha Chen. Data vi- sualization with multidimensional scaling.Journal of com- putational and graphical statistics, 17(2):444–472, 2008. 26

    work page 2008

Show all 49 references
  1. [9]

    Learning imbalanced datasets with label- distribution-aware margin loss.Advances in neural informa- tion processing systems, 32, 2019

    Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label- distribution-aware margin loss.Advances in neural informa- tion processing systems, 32, 2019. 2

    2019

  2. [10]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 2, 5, 17

    2021

  3. [11]

    Diffusion forcing: Next-token prediction meets full-sequence diffu- sion.arXiv preprint arXiv:2407.01392, 2024

    Boyuan Chen, Diego Marti Monso, Yilun Du, Max Sim- chowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffu- sion.arXiv preprint arXiv:2407.01392, 2024. 2, 6

    2024 arXiv

  4. [12]

    a2 nav: Action-aware zero-shot robot navigation by exploit- ing vision-and-language ability of foundation models.arXiv preprint arXiv:2308.07997, 2023

    Peihao Chen, Xinyu Sun, Hongyan Zhi, Runhao Zeng, Thomas H Li, Gaowen Liu, Mingkui Tan, and Chuang Gan. a2 nav: Action-aware zero-shot robot navigation by exploit- ing vision-and-language ability of foundation models.arXiv preprint arXiv:2308.07997, 2023. 1, 2

    2023 arXiv

  5. [13]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. InInternational conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 1, 2, 5, 17

    2020

  6. [14]

    Improved baselines with momentum contrastive learning

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020. 2, 5, 17

    2003 arXiv

  7. [15]

    An empirical study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9640–9649, 2021. 1, 2, 5, 17

    2021

  8. [16]

    Duel: Dupli- cate elimination on active memory for self-supervised class- imbalanced learning

    Won-Seok Choi, Hyundo Lee, Dong-Sig Han, Junseok Park, Heeyeon Koo, and Byoung-Tak Zhang. Duel: Dupli- cate elimination on active memory for self-supervised class- imbalanced learning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11579–11587,

  9. [17]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 2

    2009

  10. [18]

    Routledge, 2017

    Michel Denis.Space and spatial cognition: A multidisci- plinary perspective. Routledge, 2017. 3

    2017

  11. [19]

    Clip- nav: Using clip for zero-shot vision-and-language naviga- tion.arXiv preprint arXiv:2211.16649, 2022

    Vishnu Sashank Dorbala, Gunnar Sigurdsson, Robinson Pi- ramuthu, Jesse Thomason, and Gaurav S Sukhatme. Clip- nav: Using clip for zero-shot vision-and-language naviga- tion.arXiv preprint arXiv:2211.16649, 2022. 1, 2

    2022 arXiv

  12. [20]

    Re- thinking the uniformity metric in self-supervised learning

    Xianghong Fang, Jian Li, Qiang Sun, and Benyou Wang. Re- thinking the uniformity metric in self-supervised learning. arXiv preprint arXiv:2403.00642, 2024. 3

    2024 arXiv

  13. [21]

    A review of environmental context detection for navigation based on multiple sensors.Sensors, 20(16), 2020

    Florent Feriol, Damien Vivet, and Yoko Watanabe. A review of environmental context detection for navigation based on multiple sensors.Sensors, 20(16), 2020. 2

    2020

  14. [22]

    Reliable estimation of kl divergence using a discriminator in repro- ducing kernel hilbert space.Advances in Neural Information Processing Systems, 34:10221–10233, 2021

    Sandesh Ghimire, Aria Masoomi, and Jennifer Dy. Reliable estimation of kl divergence using a discriminator in repro- ducing kernel hilbert space.Advances in Neural Information Processing Systems, 34:10221–10233, 2021. 2

    2021

  15. [23]

    Classification using kernel density estimates: Multiscale analysis and visualization.Technometrics, 48(1):120–132,

    Anil K Ghosh, Probal Chaudhuri, and Debasis Sengupta. Classification using kernel density estimates: Multiscale analysis and visualization.Technometrics, 48(1):120–132,

  16. [24]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6

    2016

  17. [25]

    Towards open world object de- tection

    KJ Joseph, Salman Khan, Fahad Shahbaz Khan, and Vi- neeth N Balasubramanian. Towards open world object de- tection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5830–5840,

  18. [26]

    Supervised contrastive learning.Advances in neural information processing systems, 33:18661–18673,

    Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning.Advances in neural information processing systems, 33:18661–18673,

  19. [27]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 8

    2023

  20. [28]

    Fine- grained segmentation networks: Self-supervised segmen- tation for improved long-term visual localization.CoRR, abs/1908.06387, 2019

    M ˚ans Larsson, Erik Stenborg, Carl Toft, Lars Ham- marstrand, Torsten Sattler, and Fredrik Kahl. Fine- grained segmentation networks: Self-supervised segmen- tation for improved long-term visual localization.CoRR, abs/1908.06387, 2019. 1

    1908 arXiv

  21. [29]

    Auto mc-reward: Automated dense reward de- sign with large language models for minecraft

    Hao Li, Xue Yang, Zhaokai Wang, Xizhou Zhu, Jie Zhou, Yu Qiao, Xiaogang Wang, Hongsheng Li, Lewei Lu, and Jifeng Dai. Auto mc-reward: Automated dense reward de- sign with large language models for minecraft. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pat...

    2024

  22. [30]

    Principled and efficient transfer learning of deep models via neural col- lapse.arXiv preprint arXiv:2212.12206, 2022

    Xiao Li, Sheng Liu, Jinxin Zhou, Xinyu Lu, Carlos Fernandez-Granda, Zhihui Zhu, and Qing Qu. Principled and efficient transfer learning of deep models via neural col- lapse.arXiv preprint arXiv:2212.12206, 2022. 3

    2022 arXiv

  23. [31]

    Self-supervised learning is more robust to dataset imbalance

    Hong Liu, Jeff Z HaoChen, Adrien Gaidon, and Tengyu Ma. Self-supervised learning is more robust to dataset imbalance. arXiv preprint arXiv:2110.05025, 2021. 2

    2021 arXiv

  24. [32]

    Sphereface: Deep hypersphere embedding for face recognition

    Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 212–220,

  25. [33]

    The MIT Press, 1999

    Christopher D Manning.Foundations of statistical natural language processing. The MIT Press, 1999. 3

    1999

  26. [34]

    Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018. 13

    2018 arXiv

  27. [35]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1, 2, 5, 17

    2023 arXiv

  28. [36]

    Matching multiple perspectives for efficient representation learning

    Omiros Pantazis and Mathew Salvaris. Matching multiple perspectives for efficient representation learning. InEu- ropean Conference on Computer Vision, pages 686–698. Springer, 2022. 6

    2022

  29. [37]

    Prevalence of neural collapse during the terminal phase of deep learning training.Proceedings of the National Academy of Sciences, 117(40):24652–24663, 2020

    Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training.Proceedings of the National Academy of Sciences, 117(40):24652–24663, 2020. 3

    2020

  30. [38]

    Mp5: A multi-modal open-ended embodied system in minecraft via active perception

    Yiran Qin, Enshen Zhou, Qichang Liu, Zhenfei Yin, Lu Sheng, Ruimao Zhang, Yu Qiao, and Jing Shao. Mp5: A multi-modal open-ended embodied system in minecraft via active perception. In2024 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 16307– 1631...

    2024

  31. [39]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  32. [40]

    Self- supervised learning through efference copies.Advances in Neural Information Processing Systems, 35:4543–4557,

    Franz Scherr, Qinghai Guo, and Timoleon Moraitis. Self- supervised learning through efference copies.Advances in Neural Information Processing Systems, 35:4543–4557,

  33. [41]

    Vint: A foundation model for visual navigation.arXiv preprint arXiv:2306.14846, 2023

    Dhruv Shah, Ajay Sridhar, Nitish Dashora, Kyle Stachow- icz, Kevin Black, Noriaki Hirose, and Sergey Levine. Vint: A foundation model for visual navigation.arXiv preprint arXiv:2306.14846, 2023. 1, 2

    2023 arXiv

  34. [42]

    Improved deep metric learning with multi- class n-pair loss objective.Advances in neural information processing systems, 29, 2016

    Kihyuk Sohn. Improved deep metric learning with multi- class n-pair loss objective.Advances in neural information processing systems, 29, 2016. 2

    2016

  35. [43]

    Nomad: Goal masked diffusion policies for nav- igation and exploration

    Ajay Sridhar, Dhruv Shah, Catherine Glossop, and Sergey Levine. Nomad: Goal masked diffusion policies for nav- igation and exploration. In2024 IEEE International Con- ference on Robotics and Automation (ICRA), pages 63–70. IEEE, 2024. 1, 2

    2024

  36. [44]

    The replica dataset: A digital replica of indoor spaces.arXiv preprint arXiv:1906.05797,

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces.arXiv preprint arXiv:1906.05797,

    1906 arXiv

  37. [45]

    Un- derstanding self-supervised learning dynamics without con- trastive pairs

    Yuandong Tian, Xinlei Chen, and Surya Ganguli. Un- derstanding self-supervised learning dynamics without con- trastive pairs. InInternational Conference on Machine Learning, pages 10268–10278. PMLR, 2021. 3

    2021

  38. [46]

    Understanding contrastive representation learning through alignment and uniformity on the hypersphere

    Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. InInternational conference on machine learning, pages 9929–9939. PMLR, 2020. 3

    2020

  39. [47]

    Graph based environment representation for vision-and- language navigation in continuous environments, 2023

    Ting Wang, Zongkai Wu, Feiyu Yao, and Donglin Wang. Graph based environment representation for vision-and- language navigation in continuous environments, 2023. 2

    2023

  40. [48]

    Vlfm: Vision-language frontier maps for zero-shot semantic navigation, 2023

    Naoki Yokoyama, Sehoon Ha, Dhruv Batra, Jiuguang Wang, and Bernadette Bucher. Vlfm: Vision-language frontier maps for zero-shot semantic navigation, 2023. 2

    2023

  41. [49]

    Kernel mixture model for probability density estimation in bayesian classifiers.Data Mining and Knowl- edge Discovery, 32:675–707, 2018

    Wenyu Zhang, Zhenjiang Zhang, Han-Chieh Chao, and Fan- Hsun Tseng. Kernel mixture model for probability density estimation in bayesian classifiers.Data Mining and Knowl- edge Discovery, 32:675–707, 2018. 2 OBSER: Object-Based Sub-Environment Recognition for Zero-Shot Environme...

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.