{"id":"23c6e12b-dbbf-48eb-ac76-c802906367be","arxiv_id":"2412.19920","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Vision foundation models show predictable feature instability at accidental viewpoints, while their handling of out-of-distribution viewpoints varies widely across models.","lead":"This paper tests how nine popular vision models react to small changes in viewing angle. It finds that features become unstable at accidental viewpoints, where an object's 3D shape is hidden, and that this instability degrades classification, visual question answering, and 3D reconstruction.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Instability labels from Eq. 1 may be confounded by uncontrolled lighting in ABO/CO3D; the accidental/OOD claims need a lighting-controlled check.","rationale":"The reader's weakest-assumption concern about τ and r is valid: since τ is set to the 97th percentile per model and dataset, and r is never reported, the instability labels may be sensitive to those choices. However, I find an even more direct threat to the central claim: Eq. 1 is intended to isolate viewpoint changes, but the ABO dataset explicitly varies lighting across renderings while dv does not include lighting. The same issue affects CO3D, where real-world videos contain motion blur, occlusion, and lighting shifts that are not separated from camera pose. If these factors contribute to high instability scores, the paper's causal language about 'accidental viewpoints' and OOD viewpoints being the cause of downstream failures is not supported. This does not invalidate the empirical observation that features are unstable at certain views, nor the useful engineering contribution of predicting instability from features; it does mean the specific geometric interpretation needs a controlled experiment. The reader's conditional verdict is appropriate, and this additional confound strengthens, rather than changes, the need for revision. I therefore leave the verdict unchanged rather than moving to reject, because the central phenomenon may well survive a lighting-controlled rerun.","tokens_in":12601,"tokens_out":5872,"duration_ms":69199,"concrete_test":"Re-run the ABO analysis on lighting-controlled renderings: for the same set of objects and the same 72 azimuth/elevation viewpoints, render each view under a fixed lighting condition (or include lighting identity in dv and condition it out). Then recompute Eq. 1 instability labels, the SVM classification accuracies, the accidental/OOD k-means assignment, and the downstream performance gaps. If the unstable/accidental label sets differ from the original by IoU < 0.8, or if the downstream gaps shrink materially, the viewpoint-specific interpretation is confounded by lighting variation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on Eq. 1 measuring feature changes caused by small camera-viewpoint perturbations. But in Sec. 3.1.1 the ABO Spins renderings are described as having 'randomized elevation angles and varying lighting conditions,' while dv in Eq. 2 (Sec. 3.2) encodes only 3D position and angle, not illumination. Consequently, viewpoints that are neighbors in camera space can differ in lighting, and the cosine distances in Eq. 1 mix lighting-induced feature changes with viewpoint-induced ones. Since the 97th-percentile threshold τ is fit per model and dataset, these confounded scores determine every downstream label: the SVM stable/unstable classification, the k-means accidental/OOD split, the IoU agreement numbers, and the classification/VQA/reconstruction performance gaps. CO3D introduces additional uncontrolled factors (occlusion, blur, lighting changes) that are likewise not separated from viewpoint. If lighting or other non-viewpoint factors drive a substantial share of the high instability scores, then the paper's conclusions about 'accidental viewpoints' and OOD viewpoints being encoded by foundation models would be an artifact of the image-acquisition setup rather than a property of viewpoint geometry. Missing details, such as the neighborhood radius r and SVM hyperparameters, make this hard to rule out from the paper alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a definition of viewpoint instability for vision foundation models, computed as the mean cosine-distance change in feature embeddings over small camera-perturbation neighborhoods (Eq. 1). It labels viewpoints as stable or unstable using a per-model 97th-percentile threshold, clusters the unstable points into accidental and OOD subtypes via k-means and silhouette scores, and evaluates the resulting labels in SVM prediction experiments and three downstream tasks (zero-shot and linear-probe classification, VQA, and monocular 3D reconstruction) on the ABO and CO3D datasets. The central findings are that unstable viewpoints are largely separable from stable ones in feature space, that models agree on accidental viewpoints but disagree on OOD viewpoints, and that downstream performance degrades at both unstable subtypes.","tokens_in":115,"tokens_out":3750,"duration_ms":83402,"significance":"The paper is useful in scope and breadth: it studies nine featurizers, two datasets, and three downstream tasks, and it includes a human-annotation sanity check for accidental viewpoints. The proposed instability score is simple and reproducible, and the finding that accidental viewpoints produce consistent feature collapse across architectures, while OOD viewpoints are model-specific, is potentially valuable for model auditing and for designing viewpoint-robust representations. However, the strength of these conclusions is currently limited by unresolved sensitivity to the definition's free parameters and by confounding factors, so the paper is a promising framework rather than a definitive characterization.","major_comments":[{"comment":"The neighborhood radius r in Eq. 1 is never reported, and the instability threshold τ is set to the 97th percentile of per-model instability scores in Sec. 3.3. Since every subsequent experiment reuses these binary labels, the SVM accuracies, k-means subtypes, IoU agreement numbers, and downstream performance gaps all inherit whatever sensitivity Eq. 1 has to r and τ. The paper should report r, justify the choice of τ, and include an ablation that varies r and τ and recomputes Table 1 and the IoU statistics.","section":"Sec. 3.2, Eq. 1; Sec. 3.3"},{"comment":"The ABO Spins renderings are described as having varying lighting conditions, but the viewpoint distance dv in Eq. 1 encodes only 3D position and angle. Cosine distances in Eq. 1 therefore mix lighting-induced feature changes with viewpoint-induced changes, and because τ is fit per model and dataset, the unstable labels may partly reflect illumination rather than viewpoint geometry. The authors should either constrain the neighbor set to renderings with fixed lighting, include lighting in dv, or provide a lighting-controlled control experiment to show that the instability scores are not dominated by non-viewpoint factors.","section":"Sec. 3.1.1 vs. Sec. 3.2"},{"comment":"The SVM prediction experiment is partly circular: the ground-truth stable/unstable labels are defined by feature distances in f (Eq. 1), and the same f provides the SVM input features. Figure 4 shows that unstable points form clusters in that space, so high accuracies in Table 1 may largely reflect the label-generating geometry rather than a semantically meaningful property. A stronger test would train the SVM on labels from one featurizer and evaluate on another featurizer's input features, or use the human-annotated labels as ground truth.","section":"Sec. 3.3, Table 1"},{"comment":"The accidental/OOD split is obtained by k-means with k=2 and by assigning the cluster with the highest silhouette score to accidental; this heuristic is not anchored to any geometric definition of accidental viewpoints. The human-overlap result (78.3%) is only moderate, and treating all unstable views as accidental raises agreement to 89.8%, which weakens the claim that the subtype separation is meaningful. The paper should validate the clustering against an independent criterion, such as camera elevation aligned with an object's degenerate dimension, rather than relying on silhouette alone.","section":"Sec. 3.3, Fig. 4"}],"minor_comments":[{"comment":"There is a typo in 'paramters' and a duplicated 'parameters.' in the SAM entry; these should be corrected.","section":"Sec. 3.1.2"},{"comment":"The text defines Nv(vi, r) but the following sentence refers to N(vi, τ); the notation should be made consistent.","section":"Sec. 3.2"},{"comment":"The caption reads 'datasetWe' and is missing a period; please fix the spacing and punctuation.","section":"Fig. 4 caption"},{"comment":"The text refers to 'Figures 8' while there is only one Figure 8, and the model name is inconsistently written as 'Stable Fast3D' versus the reference 'SF3D'; please harmonize.","section":"Sec. 3.4.4"},{"comment":"MAE reports the same Stable accuracy of 93.97% on both ABO and CO3D; please verify whether this is a copy/paste error.","section":"Table 3"},{"comment":"The SVM RBF hyperparameters (C, gamma), the non-maximal suppression details, and the exact number of scenes used per dataset are not reported, which prevents replication.","section":"Sec. 3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of a computer vision venue and addresses an interesting and timely question, but the central instability labels are under-specified and partially confounded by lighting and by circularity in the prediction experiment. I would not reject because the framework and downstream results are suggestive; however, the authors should be asked to provide the missing hyperparameters, a sensitivity analysis, and a deconfounding or cross-featurizer validation before the claims can be accepted at the stated strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something useful: it extends viewpoint-robustness analysis from two VLMs to nine heterogeneous foundation models on two datasets, and it shows the feature-space signal is real enough to separate stable from unstable views at better-than-chance rates. The downstream experiments on classification, VQA, and monocular reconstruction are a reasonable addition, and the human-annotation check on the accidental/OOD split (78% overlap) is a legitimate external anchor. The authors also deserve credit for not papering over CO3D's messiness; they explicitly note backgrounds, lighting, occlusion, and acquisition factors. As an empirical characterization, this is a solid contribution to the 3D-awareness literature.\n\nThe soft spots are real and need addressing before the headline claim can be trusted. First, the instability label in Eq. 1 depends on an unrecorded neighborhood radius r and a threshold tau fixed as the 97th percentile per model/dataset. Every subsequent experiment uses those labels, so any sensitivity in r or tau propagates into the SVM accuracies, the k-means split, and the downstream gaps. Second, the SVM prediction is circular in an important sense: the ground-truth labels are feature distances in the same space the SVM sees, so high accuracies mostly demonstrate that unstable points cluster in feature space. The scene split mitigates but does not remove the issue. Third, the stress-test concern is on point. ABO Spins renderings have randomized elevation angles and varying lighting, while dv encodes only 3D position and angle. Neighboring views can therefore differ in lighting, and the cosine distances in Eq. 1 blend lighting-induced feature changes with viewpoint-induced ones. The paper does not control for this, so the accidental/OOD clusters and the downstream performance gaps could be partly driven by non-viewpoint factors. This is not fatal, because the qualitative examples are plausibly geometric, but it is exactly the kind of confound that needs an ablation or a fixed-lighting control before the 'consistent encoding' claim is accepted.\n\nThis is a good paper for a workshop or a solid conference submission after revision. The core finding — that foundation models have measurable viewpoint instabilities and that these affect downstream tasks — will survive; the subtyping and the predictive claims are what need tightening. Send it to peer review, but with a clear request for a lighting-controlled analysis and full hyperparameter reporting.","headline":"A genuinely useful expansion of viewpoint-instability analysis to nine foundation models, but the label definition is confounded by uncontrolled lighting and the prediction claim is more circular than it looks.","tokens_in":13366,"tokens_out":2217,"would_cite":false,"duration_ms":24203,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Vision foundation models all share a weakness at accidental viewpoints, and failures there are predictable from features alone.","keywords":["viewpoint stability","vision foundation models","accidental viewpoints","out-of-distribution viewpoints","feature robustness","3D reasoning","CLIP","DINOv2"],"falsifier":"Recompute the instability labels while varying the neighborhood radius $r$ over a plausible range and setting the threshold at the 90th and 99th percentiles rather than the 97th; if the SVM stable/unstable accuracies and the accidental-OOD cluster assignments in Tables 1-3 change materially, the feature-only predictability result is an artifact of the chosen cutoffs, and if they barely change, the claim is robust to the hyperparameters.","tokens_in":12466,"feed_emoji":"🎥","tokens_out":7020,"duration_ms":62984,"temperature":0.7,"pith_summary":"This study asks whether the vision foundation models now used to embed images are stable under changes in camera viewpoint, and finds that they are not uniformly stable. It defines instability as a large change in a model's feature vector when the viewing angle is perturbed slightly, and shows that across nine models the same geometric 'accidental' viewpoints—camera angles that hide an object's true 3D structure—tend to be unstable, while views that are merely out-of-distribution differ from model to model. The paper proposes that unstable views can be recognized and classified from the feature vector alone, without looking at the image, and demonstrates that classification, visual question answering, and monocular 3D reconstruction all lose accuracy at such viewpoints. The care a reader should take from this is that viewpoint-dependent failures are not random noise: they follow geometry and model bias, and can be anticipated in advance.","feed_headline":"Accidental viewpoints fool every vision foundation model tested","feed_subtitle":"Nine models all stumble on geometry-hiding camera angles, and the failures show up in classification, VQA, and 3D reconstruction.","key_machinery":"The load-bearing object is the viewpoint-instability score of Eq. 1: for a view $v_i$, take the average cosine distance in feature space between $f(v_i)$ and the features of all camera views within a small neighbourhood radius $r$, and call the view unstable when this average exceeds a threshold $\\tau$ (set to the 97th percentile of per-model scores). On top of that score the paper builds two tools: an RBF kernel SVM that predicts the stable/unstable label from a single image's feature vector without seeing neighbouring views, and a $k=2$ clustering of unstable features (assigned by silhouette score) that separates accidental from OOD viewpoints. Accidental viewpoints are the key geometric concept: camera orientations that collapse an object to a 1D or 2D silhouette and thereby obscure its true structure.","core_discovery":"The paper's central claim is that viewpoint instability in vision foundation models is real, predictable, and structured. Using nine featurizers (CLIP, DINO, DINOv2, ConvNeXt, DeiT III, DreamSim, MAE, SAM, SigLip) on two multi-view datasets (ABO and CO3D), it labels views as unstable when small camera perturbations produce large feature changes, and then shows that a classifier can separate stable from unstable views using only the feature vector, with accuracies of 81-96% on ABO and 71-77% on CO3D. The unstable views split into two subtypes: accidental viewpoints, where the camera angle hides the object's true 3D structure, and out-of-distribution (OOD) viewpoints, which are rare or unseen in training. The paper finds that all nine models largely agree on which views are accidental, but agree almost not at all on which views are OOD (mean IoU 0.12), and that downstream accuracy falls off sharply at both kinds of unstable views—most dramatically at accidental viewpoints.","pith_inferences":["Editorial inference: the same feature-only instability detector could be repurposed as a confidence measure for high-stakes uses of vision-language models, since a single forward pass would already reveal whether a view is likely to mislead the model.","Editorial inference: the tight clustering of accidental viewpoints across models hints that a small set of geometric augmentations targeting those orientations—rather than exhaustive viewpoint augmentation—might close most of the stability gap.","Editorial inference: the central comparisons all depend on the 97th-percentile threshold and the unreported neighbourhood radius $r$; a reader wanting to rely on the quantitative claims should test whether the reported accuracies are stable when these two hyperparameters are varied."],"forward_implications":["A deployed system could flag a view as untrustworthy at inference time using only the model's own feature vector, before committing to a classification, caption, or reconstruction.","Because accidental viewpoints are shared across featurizers, failures at these views should be expected in any current vision foundation model, not treated as a quirk of one architecture.","Because OOD viewpoints are largely model-specific, robustness claims about unusual views cannot be transferred from one model to another and need per-model evaluation.","Downstream models that rely on a backbone (VQA and monocular reconstruction in this study) inherit the backbone's viewpoint instability, so improving the backbone's feature stability should improve the downstream task.","The feature-only separability of stable and unstable views means instability can be studied without storing or transmitting the input images, which matters for privacy-sensitive applications."],"supporting_citations":[{"why":"CLIP is the primary featurizer used throughout; its embeddings also drive zero-shot classification and LLaVA's VQA backbone.","marker":"[37]"},{"why":"DINOv2 supplies self-supervised features used in the instability analysis and is the backbone of the Stable Fast3D reconstruction model.","marker":"[34]"},{"why":"ABO is the synthetic multi-view dataset with 590,400 viewpoints from which most instability labels are computed.","marker":"[10]"},{"why":"CO3D provides real-world in-the-wild multi-view images used to test whether instability findings hold outside synthetic renderings.","marker":"[38]"},{"why":"Witkin and Tenenbaum define accidental viewpoints, the geometric concept the paper operationalizes and measures across models.","marker":"[52]"},{"why":"OmniView is the closest prior study of OOD viewpoint robustness in vision-language models, which this work extends to nine featurizers and to accidental viewpoints.","marker":"[41]"},{"why":"DINO is one of the nine featurizers and provides a representative self-supervised contrast for the stability analysis.","marker":"[4]"}],"fun_headline_variants":["Accidental viewpoints trip up all 9 vision models tested","Nine vision models all stumble on hiding camera angles","Viewpoint blind spot: 9 models fooled by accidental views","All 9 vision models fail on geometry-hiding viewpoints"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole analysis inherits its definition of 'unstable' from Eq. 1, which depends on a neighborhood radius $r$ that is never reported and on a threshold fixed at the 97th percentile of each model's instability scores; if those two choices were changed, the set of views labeled unstable would change, and every classification and downstream result would move with them.","fun_headline_variants_meta":{"raw":{"variants":["Accidental viewpoints trip up all 9 vision models tested","Nine vision models all stumble on hiding camera angles","Viewpoint blind spot: 9 models fooled by accidental views","All 9 vision models fail on geometry-hiding viewpoints"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000166,"raw_usage":{"total_tokens":1249,"prompt_tokens":935,"completion_tokens":314,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":247}},"tokens_in":551,"tokens_out":314,"duration_ms":3889,"temperature":1.0,"reasoning_tokens":247,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:46:51.165793+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the instability labels while varying the neighborhood radius $r$ over a plausible range and setting the threshold at the 90th and 99th percentiles rather than the 97th; if the SVM stable/unstable accuracies and the accidental-OOD cluster assignments in Tables 1-3 change materially, the feature-only predictability result is an artifact of the chosen cutoffs, and if they barely change, the claim is robust to the hyperparameters.","supporting_citations":[{"cited_title":"Learning transferable visual models from natural language supervi- sion","cited_arxiv_id":null,"evidence_quote":"CLIP is the primary featurizer used throughout; its embeddings also drive zero-shot classification and LLaVA's VQA backbone."},{"cited_title":"Abo: Dataset and benchmarks for real-world 3d object un- derstanding","cited_arxiv_id":null,"evidence_quote":"ABO is the synthetic multi-view dataset with 590,400 viewpoints from which most instability labels are computed."},{"cited_title":"Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction","cited_arxiv_id":null,"evidence_quote":"CO3D provides real-world in-the-wild multi-view images used to test whether instability findings hold outside synthetic renderings."},{"cited_title":"On the role of structure in vision","cited_arxiv_id":null,"evidence_quote":"Witkin and Tenenbaum define accidental viewpoints, the geometric concept the paper operationalizes and measures across models."},{"cited_title":"Omniview-Tuning: Boosting Viewpoint Invariance of Vision-Language Pre-training Models","cited_arxiv_id":"2404.12139","evidence_quote":"OmniView is the closest prior study of OOD viewpoint robustness in vision-language models, which this work extends to nine featurizers and to accidental viewpoints."}],"review_version":1}