Pith. sign in

REVIEW 3 major objections 7 minor 84 references

What Images Cannot Say: Language-Guided Olfactory Representation Learning

T0 review · 3 major / 7 minor · reviewed 2026-07-08 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VLM-inferred smell descriptors bridge vision and olfaction

desk verdict First paper to align e-nose data with both vision and language. The core retrieval gains are real but modest, and the SOTA claim is complicated by a baseline reproduction that falls short of originally reported numbers. read the letter →

arxiv 2607.06402 v1 pith:OG4AHFUJ submitted 2026-07-07 cs.CV cs.AIcs.LG

Eleftherios Tsonis , Xi Wang , Vicky Kalogeiton This is my paper
classification cs.CVcs.AIcs.LG
keywords olfactorysmellcontextualenvironmentalimageslearningrepresentationsscene
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that language generated by Vision-Language Models can serve as a semantic bridge between electronic-nose sensor readings and visual scenes, compensating for the fact that cameras capture only a partial view of what an e-nose actually measures. The authors introduce SCENT, a framework that prompts a VLM to produce structured textual descriptors of a scene's objects, environmental context, and plausible ambient smells, then uses these descriptors to train a smell encoder aligned with both CLIP image and text embedding spaces. The core mechanism is a three-stream contrastive alignment: the olfactory signal is projected through two heads, one matching visual embeddings and one matching textual embeddings, so that the smell representation is jointly shaped by what is visible and what is semantically implied. A second contribution is a language-guided latent decomposition that splits the olfactory embedding into an object-specific component and a contextual component, each contrastively aligned to its corresponding text descriptor and held together by a reconstruction constraint. On the New York Smells dataset, SCENT reportedly improves smell-to-image retrieval from 20.0 to 23.0 R@5 and smell-to-text retrieval from 8.1 to 11.9 R@5 over a vision-only baseline, and the decomposition is validated by a zero-shot recombination task where decoded synthetic smell fingerprints retrieve unseen object-context pairings better than raw signal concatenation.

What carries the argument

The load-bearing machinery is the three-stream contrastive alignment between a Transformer-based smell encoder, a CLIP image encoder, and a frozen CLIP text encoder, supplemented by the language-guided latent decomposition that splits the olfactory embedding into object and context sub-vectors each aligned to VLM-generated text descriptors.

What would settle it

If the VLM-generated smell descriptors are plausible but physically absent hallucinations, then the text supervision would align the smell encoder to semantic labels with no basis in the actual sensor signal, and the retrieval gains would reflect the text encoder's prior knowledge rather than a genuine smell-language alignment. A decisive test would be to collect human olfactory judgments or gas chromatography measurements for a subset of scenes and check whether the VLM-inferred smells correlate with physically present odorants.

Watch

Extended reading notes

Core claim

The central finding is that VLM-generated textual descriptions of scenes, including inferred ambient smells not directly visible in the image, provide supervision that meaningfully improves the alignment of electronic-nose signals with both visual and textual embedding spaces. The paper demonstrates this through a controlled ablation: aligning smell with only object labels yields poor retrieval, but progressively adding environmental context and VLM-inferred smell descriptors monotonically improves performance across smell-to-image, smell-to-text, and joint retrieval tasks. The language-guided decomposition further shows that the learned olfactory representation can be split into object and.

Load-bearing premise

The paper assumes that VLM-generated smell descriptors correspond to actual chemical signals present in the e-nose measurements, but validates this only indirectly through a second VLM judging plausibility and a held-out view confirming 32.2% of inferences, without any chemical or human-olfactory ground truth verifying physical presence.

Editorial extensions

If this is right

  • If VLM-inferred smells genuinely correspond to chemical signals in the e-nose data, then scaling the dataset and VLM reasoning quality should continue to improve olfactory retrieval, potentially enabling practical applications like environmental monitoring or food quality assessment from sensor data alone.
  • The latent decomposition, if it truly separates object from context, could allow targeted manipulation of olfactory representations, such as subtracting background odors to isolate a target scent, which would matter for safety or diagnostic applications.
  • The finding that a two-stage LLM pipeline (caption then reason) underperforms direct VLM grounding suggests that spatial and visual context is important for olfactory inference, not just common-sense priors, which constrains how far decoupled language reasoning can go.
  • If the decomposition generalizes, the same principle of language-guided factorization could apply to other mixed-signal modalities where sensor readings blend multiple sources, such as audio scenes or tactile measurements.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper introduces SCENT, a multimodal framework for learning olfactory representations from electronic nose signals, using Vision-Language Models (VLMs) to generate textual scene descriptors (object, context, and inferred ambient smells) as a semantic bridge between vision and olfaction. The method trains a smell encoder via contrastive learning to align with both CLIP visual and textual embeddings, and introduces a language-guided latent decomposition separating object-specific from contextual odor components. Experiments on the New York Smells (NYS) dataset show improvements over a reproduced NYS baseline on smell-to-image (S2I), smell-to-text (S2T), and joint (S2IT) retrieval tasks, along with a zero-shot recombination retrieval task for evaluating disentanglement.

Significance. The paper addresses a novel and underexplored problem at the intersection of computer vision, multimodal learning, and machine olfaction. The idea of using VLM-generated descriptors—including inferred ambient smells—as supervision for e-nose representation learning is creative and well-motivated by the partial observability of visual data. The language-guided latent decomposition into object and context components is a sensible contribution, and the zero-shot recombination retrieval protocol (Table 4) provides a falsifiable test of disentanglement. The ablation studies (Tables 1, 3, 5, 9-12) are thorough and isolate the contributions of different semantic granularities, fusion strategies, and training schedules. The project page and reproducible experimental design (standard metrics, clear splits) are strengths.

major comments (3)
  1. §4.3, Table 2, and Supplementary §A (Baseline Reproduction): The claim of 'state-of-the-art' is undermined by a discrepancy in the baseline reproduction. Table 2 shows the original NYS paper [46] reported S2I R@5=29.6 (NYS*), while the authors' reproduction (NYS†) achieves only 20.0. SCENT's R@5=23.0 does not exceed the original NYS result of 29.6. However, Supplementary §A states: 'our reproduced baseline achieves performance metrics that exceed those originally reported in [46].' This directly contradicts Table 2 where NYS† (20.0) < NYS* (29.6) on R@5. This discrepancy must be resolved. If the S2I R@5 metric in Table 2 is not comparable to the one in [46] (e.g., different evaluation protocols or gallery sizes), this must be explicitly clarified. As it stands, the SOTA claim on S2I does not hold against the original reported numbers, and the supplementary text appears internally contrad
  2. §4.5 and §4.6: The central premise relies on VLM-inferred smell descriptors corresponding to actual olfactory signals. The validation in §4.5 (98.6% plausibility, 32.2% View-2 confirmation) shows the VLM infers contextually reasonable and sometimes hidden scene content, but this does not verify that the inferred smells are physically present in the e-nose signal. If the VLM hallucinates plausible-but-absent smells, the text supervision could misalign the smell encoder with semantic labels lacking physical basis. The paper would benefit from acknowledging this limitation more directly or providing a sensitivity analysis (e.g., training with only high-confidence View-2 confirmed descriptors vs. all descriptors) to demonstrate robustness to potential label noise from hallucinated smells.
  3. Supplementary §C.2, Table 7: The classification probing experiment uses VLM-generated pseudo-labels as ground truth for both training and evaluation. Since the same VLM (Qwen3VL-30B) generates both the training descriptors (O, Ctx, S) and the classification labels (Item, Environment), the probing task is partially circular: it measures whether the smell encoder aligns with VLM semantics, not whether it captures ground-truth physical categories. The paper should explicitly acknowledge this circularity and frame Table 7 as measuring VLM-semantic alignment rather than objective categorical separability. Alternatively, human-annotated labels for a subset would strengthen this claim.
minor comments (7)
  1. Table 2: The NYS* row reports S2I R@1=16.5, R@5=29.6, R@10=43.1 but leaves S2T and S2IT columns blank with em-dashes. It would help to explicitly note that the original NYS model lacks a textual head and thus cannot be evaluated on these tasks, to avoid confusion about whether the metrics were simply not available.
  2. §3.3, Eq. (2): The reconstruction loss L_rec uses an L2 norm ∥·∥₂, but the decoder d(·) predicts the raw smell signal X from concatenated latents. It is unclear whether the decoder is trained jointly with the contrastive losses or pretrained. Please clarify the training procedure for the decoder.
  3. Table 4: The 'Raw Recombination' baseline achieves R@1=3.9, higher than 'Decoded Synthesis' at R@1=2.0. This is not discussed. Please add a brief explanation for why the proposed method underperforms at R@1 but outperforms at higher k values.
  4. §4.1: The fusion weight α for S2IT retrieval is described as selected via grid search on the validation set (Supplementary §D.2), but the main text does not state the selected value. Please report the α used for the main results in Table 2.
  5. Figure 4: The y-axis label 'Sim. Score' and the x-axis labels (R@1, R@5, R@50) could be clearer. It would help to explicitly state in the caption that the x-axis corresponds to retrieval rank cutoffs and the y-axis is the mean cosine similarity between the query and the top-k retrieved items.
  6. Supplementary §A: 'our reproduced baseline achieves performance metrics that exceed those originally reported' — if this refers to metrics other than S2I R@5 (where NYS†=20.0 < NYS*=29.6), please specify which metrics exceed the original. As written, it appears to contradict Table 2.
  7. References: Several references appear to be from 2026 (e.g., [20], [25], [32], [39], [59], [67]). If these are accepted/published, please ensure citations are complete with venue/page numbers. If preprints, consider noting 'preprint' for clarity.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a thorough and constructive report. The referee correctly identifies an internal contradiction between Table 2 and Supplementary §A regarding the baseline reproduction, raises a legitimate concern about VLM-inferred smell descriptors lacking physical verification, and notes a circularity issue in the classification probing experiment. We address each point below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: §4.3, Table 2, and Supplementary §A (Baseline Reproduction): The claim of 'state-of-the-art' is undermined by a discrepancy in the baseline reproduction. Table 2 shows the original NYS paper [46] reported S2I R@5=29.6 (NYS*), while the authors' reproduction (NYS†) achieves only 20.0. SCENT's R@5=23.0 does not exceed the original NYS result of 29.6. However, Supplementary §A states: 'our reproduced baseline achieves performance metrics that exceed those originally reported in [46].' This directly contradicts Table 2 where NYS† (20.0) < NYS* (29.6) on R@5. This discrepancy must be resolved. If the S2I R@5 metric in Table 2 is not comparable to the one in [46] (e.g., different evaluation protocols or gallery sizes), this must be explicitly clarified. As it stands, the SOTA claim on S2I does not hold against the original reported numbers, and the supplementary text appears internally contrad

    Authors: The referee is correct that there is an internal inconsistency, and we appreciate this careful reading. The root cause is that the original NYS paper [46] did not release code, pretrained weights, or data splits. We constructed our own train/validation partition (5,996/936 samples), which differs from the original. Our reproduced NYS† baseline (R@5=20.0) is evaluated on our split and is not directly comparable to the original NYS* numbers (R@5=29.6), which were computed on the original (unreleased) splits with potentially different gallery sizes. The statement in Supplementary §A claiming our reproduction 'exceeds those originally reported' is incorrect as written for S2I R@5 and reflects an earlier draft comparing against a different subset of metrics. We will fix this. Specifically, we will: (1) correct Supplementary §A to accurately state that our reproduction is comparable on some metrics (e.g., R@10: 29.9 vs. 43.1 is lower, but R@20: 42.0 is comparable) but does not exceed the original on R@5; (2) add an explicit note in Table 2 clarifying that NYS* and NYS† use different data splits and are not directly comparable; (3) soften the SOTA claim in the abstract and §4.3 to state that SCENT achieves state-of-the-art among methods evaluated on our standardized splits, and that it significantly outperforms the reproduced baseline. We agree that claiming SOTA against the original NYS* numbers is not supported given the split discrepancy. revision: yes

  2. Referee: §4.5 and §4.6: The central premise relies on VLM-inferred smell descriptors corresponding to actual olfactory signals. The validation in §4.5 (98.6% plausibility, 32.2% View-2 confirmation) shows the VLM infers contextually reasonable and sometimes hidden scene content, but this does not verify that the inferred smells are physically present in the e-nose signal. If the VLM hallucinates plausible-but-absent smells, the text supervision could misalign the smell encoder with semantic labels lacking physical basis. The paper would benefit from acknowledging this limitation more directly or providing a sensitivity analysis (e.g., training with only high-confidence View-2 confirmed descriptors vs. all descriptors) to demonstrate robustness to potential label noise from hallucinated smells.

    Authors: This is a fair and important concern. We agree that the §4.5 validation demonstrates contextual plausibility, not physical presence in the e-nose signal. The 32.2% View-2 confirmation rate shows that some inferred descriptors correspond to genuinely hidden scene content, but the remaining descriptors could indeed be plausible hallucinations. We will address this in two ways. First, we will add an explicit paragraph in §5 (Limitations) acknowledging that VLM-inferred descriptors may include plausible-but-absent smells and that the current validation does not verify physical correspondence to the e-nose signal. Second, we will conduct the sensitivity analysis the referee suggests: training a variant using only the subset of descriptors that received View-2 confirmation (32.2%), versus training with all descriptors. We expect this to show robustness (since contrastive learning with noisy labels tends to be resilient, and the ablation in Table 12 already shows that adding more speculative descriptors degrades performance, suggesting the model is not simply fitting noise), but we agree the experiment is needed to demonstrate this empirically. If the View-2-confirmed subset is too small to train effectively, we will report this honestly and discuss it as a limitation. revision: yes

  3. Referee: Supplementary §C.2, Table 7: The classification probing experiment uses VLM-generated pseudo-labels as ground truth for both training and evaluation. Since the same VLM (Qwen3VL-30B) generates both the training descriptors (O, Ctx, S) and the classification labels (Item, Environment), the probing task is partially circular: it measures whether the smell encoder aligns with VLM semantics, not whether it captures ground-truth physical categories. The paper should explicitly acknowledge this circularity and frame Table 7 as measuring VLM-semantic alignment rather than objective categorical separability. Alternatively, human-annotated labels for a subset would strengthen this claim.

    Authors: The referee is correct that there is a circularity issue. The same VLM generates the training descriptors (O, Ctx, S) and the classification labels (Item, Environment), so Table 7 measures alignment with VLM-derived semantics, not objective physical categories. We will revise the framing of Table 7 and its caption to explicitly state this limitation and reframe the results as measuring VLM-semantic separability rather than ground-truth categorical separability. Regarding human-annotated labels: the NYS dataset does not provide discrete categorical labels, and generating human annotations for a meaningful subset would require domain expertise in olfactory categorization that is non-trivial to obtain on the revision timeline. We will note this as a promising direction for future work. However, we would also point out that the probing experiment is a supplementary analysis (not a core claim of the paper), and the main results in Tables 2–4 rely on retrieval metrics that do not depend on VLM pseudo-labels for evaluation. The retrieval ground truth is the paired image/sensor sample, not a VLM-generated label. revision: partial

Circularity Check

1 steps flagged · score 2.0 of 10

Minor circularity in classification probing (Table 7) where VLM-generated pseudo-labels serve as both training supervision and evaluation ground truth; main retrieval results are not circular.

  1. fitted input called prediction [Supplementary Section C.2, Table 7, and the label generation description]
    "Since the original NYS dataset [46] provides raw images and sensor data without discrete categorical labels, we utilize a pretrained VLM (Qwen3VL-30B [67]) to generate a set of closed-set pseudo-labels for the entire dataset. [...] These VLM-generated labels serve as the ground truth for our classification experiments."

    The classification probing results in Table 7 use VLM-generated pseudo-labels as both the training supervision (the smell encoder is trained with language guidance from the same VLM pipeline) and the evaluation ground truth. The linear/MLP probe is trained to predict labels that the same VLM generated, and then evaluated against those same labels. This means the probing task measures how well the olfactory representation aligns with VLM-generated categories, not with any independent ground truth. The 'classification performance' is thus partly a measure of VLM label consistency rather than genuine olfactory discriminability. However, this is a supplementary experiment, not the central claim of the paper.

full rationale

The main retrieval results (Table 2) are not circular: the smell encoder is trained with contrastive losses against VLM-generated text and CLIP image embeddings, then evaluated on held-out retrieval against the same modalities — this is standard contrastive learning with a train/test split. The S2I, S2T, and S2IT retrieval metrics measure whether the learned smell representation aligns with held-out visual and textual samples, which is an externally falsifiable benchmark. The disentanglement evaluation (Table 4) uses a zero-shot recombination protocol on unseen (Object, Context) pairs, which is a genuine generalization test. The only circular element is the classification probing in Table 7, where VLM-generated pseudo-labels serve as both training supervision and evaluation ground truth — but this is a supplementary analysis, not the paper's central SOTA claim. The paper is largely self-contained against external benchmarks, so the circularity score is low.

Assumptions & free parameters 5 free parameters · 4 assumptions · 2 invented entities

The framework introduces 5 fitted parameters (τ, λ_IS, λ_ST, λ_rec, α), of which only α is explicitly reported. The key domain assumption — that VLM-inferred smells correspond to real chemical signals — is only indirectly validated. The invented entities (latent decomposition, VLM smell descriptors) have partial independent evidence but lack chemical or human-olfactory ground truth.

free parameters (5)
  • τ (temperature) = learnable, not reported
    Learnable temperature in InfoNCE loss, standard but a fitted parameter.
  • λ_IS, λ_ST = not explicitly reported
    Weights for image-smell and smell-text contrastive losses in L_total (Eq. 1); values not stated in main text.
  • λ_rec = not explicitly reported
    Weight for reconstruction loss in disentanglement stage (Eq. 2); described as a regularizer but value not given.
  • α (fusion weight) = 0.7 (selected via grid search)
    Weight balancing visual vs. textual contributions in joint retrieval (Table 10); selected on validation set.
  • CLIP image encoder fine-tuning = 85.8M trainable params
    Decision to fine-tune rather than freeze the CLIP vision encoder; affects all comparisons.
assumptions (4)
  • domain assumption VLM-generated smell descriptors correspond to actual chemical signals present in e-nose measurements
    Section 3.1: the VLM infers 'plausible ambient smell cues' from visual context. This is the foundational assumption — if VLM hallucinates smells absent from the sensor data, the text supervision is noise.
  • domain assumption CLIP text/image embedding spaces are suitable targets for olfactory representation alignment
    Section 3.2: smell embeddings are projected into CLIP's 512-d space. This assumes CLIP's geometry can accommodate olfactory semantics.
  • ad hoc to paper Object and context odor components are linearly or additively separable in the latent space
    Section 3.3: the decomposition uses separate projection heads ϕ_obj and ϕ_ctx on the same z_S^T, assuming these factors are separable via MLP projections.
  • domain assumption The NYS dataset's 7,000 samples are sufficient to learn generalizable olfactory representations
    Entire experimental evaluation rests on this dataset; authors acknowledge scale limitations in Section 5.
invented entities (2)
  • Language-guided olfactory latent decomposition (z_S^obj, z_S^ctx) independent evidence
    purpose: Separate object-specific odor signals from contextual environmental contributions in the smell embedding
    Evaluated via zero-shot recombination retrieval (Table 4) where unseen object-context pairs are synthesized and retrieved. However, R@1 drops from 3.9 to 2.0, weakening the evidence.
  • VLM-inferred ambient smell descriptors independent evidence
    purpose: Provide semantic supervision for olfactory cues not visible in the image
    Partially validated by the held-out View-2 confirmation experiment (32.2% recall, Section 4.5), but no chemical ground truth confirms the inferred smells are physically present in sensor data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What Images Cannot Say: Language-Guided Olfactory Representation Learning." pith.science (2026). https://pith.science/paper/OG4AHFUJ

@misc{pith2026260706402,
  author       = {Pith},
  title        = {Pith review of: What Images Cannot Say: Language-Guided Olfactory Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OG4AHFUJ}},
  note         = {Machine review of arXiv:2607.06402}
}
read the original abstract

Images tell us what a scene looks like, but rarely what it would feel like to be there. While recent datasets pair visual scenes with electronic-nose measurements, aligning smell signals with images remains challenging because many olfactory cues arise from contextual environmental factors that are not directly visible in pixels. We introduce SCENT, a multimodal framework that uses language guidance as a semantic bridge between vision and olfaction. Our approach leverages Vision-Language Models (VLMs) to generate scene descriptors capturing objects, environmental context, and plausible ambient smell cues suggested by the visual scene. These descriptors provide semantic guidance for learning olfactory representations. We train a smell encoder that maps electronic-nose signals into a shared embedding space aligned with both visual and textual representations, and introduce a languageguided latent decomposition that separates object-specific odors from contextual environmental contributions. Experiments on the New York Smells dataset demonstrate that SCENT significantly improves crossmodal retrieval compared to vision-only baselines, achieving state-of-theart performance on smell-to-image and smell-to-text retrieval tasks. In addition, our framework produces interpretable olfactory representations that enable the disentanglement of complex smell mixtures. Our results reveal the importance of contextual semantic information for grounding olfactory perception in multimodal learning and pave the way for future research in this area.

Figures

Figures reproduced from arXiv: 2607.06402 by the authors.

Figure 1
Figure 1. Not everything we can smell is visible. Given only View 1 (in-sample) during training, the true olfactory context may lie outside the field of view, in View 2 (out-of-sample). A VLM can bridge this gap by inferring plausible smells from semantic context alone. We use these language-derived signals as supervision to learn richer smell representations that go beyond what is directly seen. visible directly, humans rout… view at source ↗
Figure 2
Figure 2. Textual scene descriptors via VLM inference. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Overall framework for multimodal olfactory representation learning [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Mean cosine similarity between the ground-truth and top-k retrieved items, in frozen CLIP image space (orange) and learned olfactory space (teal), for k ∈ {1, 5, 50}. Solid lines: SCENT; dashed: NYS. Visual vs. Olfactory Discriminativity [PITH_FULL_IMAGE:figures/full_…
Figure 5
Figure 5. Figure 5: Qualitative results for S2IT retrieval. For a given olfactory query, we show the ground-truth image in the Query column and the top-5 images from the retrieved (Image, Text) pairs. The Correct retrieval is highlighted. SCENT demon￾strates robust cross-modal alignment a…
Figure 6
Figure 6. Figure 6: Held-out view validation setup. A VLM infers plausible smells from View 1; a second VLM judge uses a held-out View 2 to verify the inferences, distinguishing grounded predictions from hallucinations. 5 Limitations and Conclusion Despite the performance gains of SCENT, …
Figure 7
Figure 7. Figure 7: Taxonomy distribution of the NYS benchmark. [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: Extended S2IT Qualitative Results with Semantic Textual Descrip [PITH_FULL_IMAGE:figures/full_fig_p032_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 84 canonical work pages

  1. [46]

    arXiv preprint arXiv:2511.20544 (2025)

    Ozguroglu, E., Liang, J., Liu, R., Chiquier, M., DeTienne, M., Qian, W.W., Horowitz, A., Owens, A., Vondrick, C.: New york smells: A large multimodal dataset for olfaction. arXiv preprint arXiv:2511.20544 (2025)

    work page Pith review arXiv 2025

  2. [1]

    Trends in Food Science & Technology (2020)

    Ali, M.M., Hashim, N., Abd Aziz, S., Lasekan, O.: Principles and recent advances in electronic nose for quality inspection of agricultural and food products. Trends in Food Science & Technology (2020)

    work page 2020

  3. [2]

    In: NeurIPS (2016)

    Aytar, Y., Vondrick, C., Torralba, A.: Soundnet: Learning sound representations from unlabeled video. In: NeurIPS (2016)

    work page 2016

  4. [3]

    Qwen2.5-VL Technical Report

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Li, Z., Wan, J., Wang, P., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., Lin, J.: Qwen2.5-vl technical report. arXiv preprint arXiv: 2502.13923 (2025)

    work page Pith review arXiv 2025

  5. [4]

    In: NeurIPS (2023)

    Bender, T., Sørensen, S., Kashani, A., Eldjarn Hjorleifsson, K., Hyldig, G., Hauberg, S., Belongie, S., Warburg, F.: Learning to taste: A multimodal wine dataset. In: NeurIPS (2023)

    work page 2023

  6. [5]

    $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

    Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., et al.: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164 (2024)

    work page Pith review arXiv 2024

  7. [6]

    In: BMVC (2025)

    Boudier, L., Manganelli, L., Tsonis, E., Dufour, N., Kalogeiton, V.: Training-free synthetic data generation with dual ip-adapter guidance. In: BMVC (2025)

    work page 2025

  8. [7]

    Sensors (2011)

    Brattoli, M., De Gennaro, G., De Pinto, V., Loiotile, A.D., Lovascio, S., Penza, M.: Odour detection methods: Olfactometry and chemical sensors. Sensors (2011)

    work page 2011

Show all 84 references
  1. [8]

    IEEE transactions on biomedical circuits and systems (2022)

    Chen, H., Huo, D., Zhang, J.: Gas recognition in e-nose system: A review. IEEE transactions on biomedical circuits and systems (2022)

    2022

  2. [9]

    In: NeurIPS (2023)

    Chen, S., Li, H., Wang, Q., Zhao, Z., Sun, M.T., Zhu, X., Liu, J.: VAST: A vision- audio-subtitle-text omni-modality foundation model and dataset. In: NeurIPS (2023)

    2023

  3. [10]

    In: NeurIPS (2025)

    Cicchetti, G., Grassucci, E., Comminiello, D.: A triangle enables multimodal align- ment beyond cosine similarity. In: NeurIPS (2025)

    2025

  4. [11]

    In: ICLR (2025)

    Cicchetti, G., Grassucci, E., Sigillo, L., Comminiello, D.: Gramian multimodal representation learning and alignment. In: ICLR (2025)

    2025

  5. [12]

    In: NeurIPS (1993)

    De Sa, V.: Learning classification with unlabeled data. In: NeurIPS (1993)

    1993

  6. [13]

    PloS one (2020)

    Debnath, T., Nakamoto, T.: Predicting human odor perception represented by continuous values from mass spectra of essential oils resembling chemical mixtures. PloS one (2020)

    2020

  7. [14]

    In: CVPR (2024)

    Dou, Y., Yang, F., Liu, Y., Loquercio, A., Owens, A.: Tactile-augmented radiance fields. In: CVPR (2024)

    2024

  8. [15]

    ASTM Special Technical Publi- cation (1985)

    Dravnieks, A.: Atlas of Odor Character Profiles. ASTM Special Technical Publi- cation (1985)

    1985

  9. [16]

    Dufumier, B., Castillo Navarro, J., Tuia, D., Thiran, J.P.: What to align in multi- modal contrastive learning? In: ICLR (2025)

    2025

  10. [17]

    In: ICASSP (2023)

    Elizalde, B., Deshmukh, S., Al Ismail, M., Wang, H.: CLAP: learning audio con- cepts from natural language supervision. In: ICASSP (2023)

    2023

  11. [18]

    In: International Conference on Information and Communication Technology (ICoICT) (2021)

    Erlangga, F., Wijaya, D.R., Wikusna, W.: Electronic nose dataset for classifying rice quality using neural network. In: International Conference on Information and Communication Technology (ICoICT) (2021)

    2021

  12. [19]

    NeurIPS (2024)

    Fang,X.,Mao,K.,Duan,H.,Zhao,X.,Li,Y.,Lin,D.,Chen,K.:Mmbench-video:A long-form multi-shot benchmark for holistic video understanding. NeurIPS (2024)

    2024

  13. [20]

    In: ICLR (2026) What Images Cannot Say 17

    Feng, D., Dai, W., Li, C., Pernigo, A., Wen, Y., Liang, P.P.: Smellnet: A large-scale dataset for real-world smell recognition. In: ICLR (2026) What Images Cannot Say 17

    2026

  14. [21]

    Engineering Proceedings (2023)

    Fundurulic, A., Faria, J.M., Inácio, M.L.: Advances in electronic nose sensors for plant disease and pest detection. Engineering Proceedings (2023)

    2023

  15. [22]

    International Journal of Robotics and Control Systems (2023)

    Furizal, F., Ma’arif, A., Firdaus, A.A., Rahmaniar, W.: Future potential of e- nose technology: A review. International Journal of Robotics and Control Systems (2023)

    2023

  16. [23]

    Journal of Breath Research (2023)

    Ghazaly, C., Biletska, K., Thevenot, E.A., Devillier, P., Naline, E., Grassin-Delyle, S., Scorsone, E.: Assessment of an e-nose performance for the detection of covid-19 specific biomarkers. Journal of Breath Research (2023)

    2023

  17. [24]

    In: CVPR (2023)

    Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K.V., Joulin, A., Misra, I.: Imagebind: One embedding space to bind them all. In: CVPR (2023)

    2023

  18. [25]

    Google DeepMind: Gemma model documentation.https://ai.google.dev/ gemma/docs/core(2026), accessed: 2026-06-30

    2026

  19. [26]

    In: CVPR (2024)

    Han, J., Gong, K., Zhang, Y., Wang, J., Zhang, K., Lin, D., Qiao, Y., Gao, P., Yue, X.: Onellm: One framework to align all modalities with language. In: CVPR (2024)

    2024

  20. [27]

    arXiv preprint arXiv:2405.07987 (2024)

    Huh, M., Cheung, B., Wang, T., Isola, P.: The platonic representation hypothesis. arXiv preprint arXiv:2405.07987 (2024)

    2024 arXiv

  21. [28]

    In: ICML (2021)

    Jia, C., Yang, Y., Xia, Y., Chen, Y.T., Parekh, Z., Pham, H., Le, Q., Sung, Y.H., Li, Z., Duerig, T.: Scaling up visual and vision-language representation learning with noisy text supervision. In: ICML (2021)

    2021

  22. [29]

    Science (2017)

    Keller, A., Gerkin, R.C., Guan, Y., Dhurandhar, A., Turu, G., Szalai, B., Main- land, J.D., Ihara, Y., Yu, C.W., Wolfinger, R., et al.: Predicting human olfactory perception from chemical features of odor molecules. Science (2017)

    2017

  23. [30]

    Kim, M.J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E.P., Sanketi, P.R., Vuong, Q., et al.: Openvla: An open-source vision-language-actionmodel.In:8thAnnualConferenceonRobotLearning(2025)

    2025

  24. [31]

    Frontiers in Behavioral Neuroscience (2020)

    Kontaris, I., East, B.S., Wilson, D.A.: Behavioral and neurobiological convergence of odor, mood and emotion: A review. Frontiers in Behavioral Neuroscience (2020)

    2020

  25. [32]

    In: CVPR (2026)

    Koutoupis, S., Zervou, M.A., Kontras, K., De Vos, M., Tsakalides, P., Tsagkatakis, G.: The more, the merrier: Contrastive fusion for higher-order multimodal align- ment. In: CVPR (2026)

    2026

  26. [33]

    Science (2023)

    Lee, B.K., Mayhew, E.J., Sanchez-Lengeling, B., Wei, J.N., Qian, W.W., Little, K.A., Andres, M., Nguyen, B.B., Moloy, T., Yasonik, J., et al.: A principal odor map unifies diverse tasks in olfactory perception. Science (2023)

    2023

  27. [34]

    IEEE Transactions on Machine Learning Research (2025)

    Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., Liu, Z., et al.: Llava-onevision: Easy visual task transfer. IEEE Transactions on Machine Learning Research (2025)

    2025

  28. [35]

    In: ICML (2023)

    Li, J., Li, D., Savarese, S., Hoi, S.: Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In: ICML (2023)

    2023

  29. [36]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    Liu,J.,Chen,S.,He, X.,Guo,L.,Zhu,X.,Wang,W.,Tang, J.:Valor:Vision-audio- language omni-perception pretraining model and dataset. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    2024

  30. [37]

    In: CVPR (2022)

    Liu, Y., Li, S., Wu, Y., Chen, C.W., Shan, Y., Qie, X.: Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection. In: CVPR (2022)

    2022

  31. [38]

    Neuro- computing (2022)

    Luo, H., Ji, L., Zhong, M., Chen, Y., Lei, W., Duan, N., Li, T.: Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning. Neuro- computing (2022)

    2022

  32. [39]

    arXiv preprint arXiv:2604.00002 (2026) 18 E

    Makri, E., Nakis, N., Sisson, L., Minsky, G., Tassiulas, L., Satarifard, V., Chris- takis, N.A.: Benchmark for assessing olfactory perception of large language models. arXiv preprint arXiv:2604.00002 (2026) 18 E. Tsonis et al

    2026

  33. [40]

    In: International forum of allergy & rhinology (2020)

    Moein, S.T., Hashemian, S.M., Mansourafshar, B., Khorram-Tousi, A., Tabarsi, P., Doty, R.L.: Smell dysfunction: a biomarker for covid-19. In: International forum of allergy & rhinology (2020)

    2020

  34. [41]

    In: Medical Imaging with Deep Learning (2024)

    Mordacq, J., Milecki, L., Vakalopoulou, M., Oudot, S., Kalogeiton, V.: Adapt: Multimodal learning for detecting physiological changes under missing modalities. In: Medical Imaging with Deep Learning (2024)

    2024

  35. [42]

    Expert systems with applications (2019)

    Mueller, P., Salminen, K., Nieminen, V., Kontunen, A., Karjalainen, M., Isokoski, P., Rantala, J., Savia, M., Väliaho, J., Kallio, P., et al.: Scent classification by k nearest neighbors using ion-mobility spectrometry measurements. Expert systems with applications (2019)

    2019

  36. [43]

    In: ICML (2011)

    Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y., et al.: Multimodal deep learning. In: ICML (2011)

    2011

  37. [44]

    arXiv preprint arXiv:1807.03748 (2018)

    Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748 (2018)

    2018 arXiv

  38. [45]

    In: CVPR (2016)

    Owens, A., Isola, P., McDermott, J., Torralba, A., Adelson, E.H., Freeman, W.T.: Visually indicated sounds. In: CVPR (2016)

    2016

  39. [47]

    In: ICML (2021)

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML (2021)

    2021

  40. [48]

    Nature (2020)

    Ravia, A., Snitz, K., Honigstein, D., Finkel, M., Zirler, R., Perl, O., Secundo, L., Laudamiel, C., Harel, D., Sobel, N.: A measure of smell enables the creation of olfactory metamers. Nature (2020)

    2020

  41. [49]

    cup tests

    Rodríguez,J.,Durán,C.,Reyes,A.:Electronicnoseforqualitycontrolofcolombian coffee through the detection of defects in “cup tests”. Sensors (2009)

    2009

  42. [50]

    In: AAAI (2023)

    Ruan, L., Hu, A., Song, Y., Zhang, L., Zheng, S., Jin, Q.: Accommodating audio modality in clip for multimodal processing. In: AAAI (2023)

    2023

  43. [51]

    arXiv preprint arXiv:1910.10685 (2019)

    Sanchez-Lengeling, B., Wei, J.N., Lee, B.K., Gerkin, R.C., Aspuru-Guzik, A., Wiltschko, A.B.: Machine learning for scent: Learning generalizable perceptual representations of small molecules. arXiv preprint arXiv:1910.10685 (2019)

    1910 arXiv

  44. [52]

    In: NeurIPS (2024)

    Saporta, A., Puli, A.M., Goldstein, M., Ranganath, R.: Contrasting with symile: Simple model-agnostic representation learning for unlimited modalities. In: NeurIPS (2024)

    2024

  45. [53]

    PLoS computational biology (2013)

    Snitz, K., Yablonka, A., Weiss, T., Frumin, I., Khan, R.M., Sobel, N.: Predicting odor perceptual similarity from odor structure. PLoS computational biology (2013)

    2013

  46. [54]

    Chemical senses (2010)

    Stevenson, R.J.: An initial evaluation of the functions of human olfaction. Chemical senses (2010)

    2010

  47. [55]

    Artificial Intelligence in Agriculture (2020)

    Tan, J., Xu, J.: Applications of electronic nose (e-nose) and electronic tongue (e-tongue) in food quality-related properties determination: A review. Artificial Intelligence in Agriculture (2020)

    2020

  48. [56]

    IEEE Sensors Journal (2020)

    Torres-Tello, J., Guaman, A.V., Ko, S.B.: Improving the detection of explosives in a mox chemical sensors array with lstm networks. IEEE Sensors Journal (2020)

    2020

  49. [57]

    Sensors and Actuators B: Chemical (2012)

    Vergara, A., Vembu, S., Ayhan, T., Ryan, M.A., Homer, M.L., Huerta, R.: Chemi- cal gas sensor drift compensation using classifier ensembles. Sensors and Actuators B: Chemical (2012)

    2012

  50. [58]

    In: CVPR (2015) What Images Cannot Say 19

    Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: A neural image caption generator. In: CVPR (2015) What Images Cannot Say 19

    2015

  51. [59]

    Journal of Neuroscience (2025)

    Wachowiak, M., Dewan, A., Bozza, T., O’Connell, T.F., Hong, E.J.: Recalibrating olfactory neuroscience to the range of naturally occurring odor concentrations. Journal of Neuroscience (2025)

    2025

  52. [60]

    In: ICML (2025)

    Wang, W., Nie, A., Zhou, W., Kai, Y., Hu, C.: Teaching physical awareness to llms through sounds. In: ICML (2025)

    2025

  53. [61]

    In: ECCV (2024)

    Wang, Y., Li, K., Li, X., Yu, J., He, Y., Chen, G., Pei, B., Zheng, R., Wang, Z., Shi, Y., et al.: Internvideo2: Scaling foundation models for multimodal video understanding. In: ECCV (2024)

    2024

  54. [62]

    Sensors and Actuators B: Chemical (2021)

    Wijaya, D.R., Sarno, R., Zulaika, E.: Dwtlstm for electronic nose signal processing in beef quality monitoring. Sensors and Actuators B: Chemical (2021)

    2021

  55. [63]

    Procedia Technology (2012)

    Wilson, A.D.: Review of electronic-nose technologies and algorithms to detect haz- ardous chemicals in the environment. Procedia Technology (2012)

    2012

  56. [64]

    In: CVPR (2024)

    Xiao, J., Yao, A., Li, Y., Chua, T.S.: Can i trust your answer? visually grounded video question answering. In: CVPR (2024)

    2024

  57. [65]

    In: ICML (2023)

    Xu, H., Ye, Q., Yan, M., Shi, Y., Ye, J., Xu, Y., Li, C., Bi, B., Qian, Q., Wang, W., et al.: mplug-2: A modularized multi-modal foundation model across text, image and video. In: ICML (2023)

    2023

  58. [66]

    In: CVPR (2024)

    Xue, L., Yu, N., Zhang, S., Panagopoulou, A., Li, J., Martín-Martín, R., Wu, J., Xiong, C., Xu, R., Niebles, J.C., et al.: Ulip-2: Towards scalable multimodal pre- training for 3d understanding. In: CVPR (2024)

    2024

  59. [67]

    arXiv preprint arXiv:2505.09388 (2025)

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.: Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025)

    2025 arXiv

  60. [68]

    In: NeurIPS Datasets and Benchmarks Track (2022)

    Yang, F., Ma, C., Zhang, J., Zhu, J., Yuan, W., Owens, A.: Touch and go: Learning from human-collected vision and touch. In: NeurIPS Datasets and Benchmarks Track (2022)

    2022

  61. [69]

    In: ICLR (2024)

    Yu, J., Wang, X., Tu, S., Cao, S., Zhang-Li, D., Lv, X., Peng, H., Yao, Z., Zhang, X., Li, H., et al.: Kola: Carefully benchmarking world knowledge of large language models. In: ICLR (2024)

    2024

  62. [70]

    In: CVPR (2017)

    Yuan, W., Wang, S., Dong, S., Adelson, E.: Connecting look and feel: Associating the visual and tactile properties of physical materials. In: CVPR (2017)

    2017

  63. [71]

    arXiv preprint arXiv:2501.13106 (2025)

    Zhang, B., Li, K., Cheng, Z., Hu, Z., Yuan, Y., Chen, G., Leng, S., Jiang, Y., Zhang, H., Li, X., et al.: Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106 (2025)

    2025 arXiv

  64. [72]

    In: CVPR (2022)

    Zhang, R., Guo, Z., Zhang, W., Li, K., Miao, X., Cui, B., Qiao, Y., Gao, P., Li, H.: Pointclip: Point cloud understanding by clip. In: CVPR (2022)

    2022

  65. [73]

    In: CVPR (2023)

    Zhao, Y., Misra, I., Krähenbühl, P., Girdhar, R.: Learning video representations from large language models. In: CVPR (2023)

    2023

  66. [74]

    In: CVPR (2025)

    Zhou, B., Li, L., Wang, Y., Liu, H., Yao, Y., Wang, W.: Unialign: Scaling multi- modal alignment within one unified model. In: CVPR (2025)

    2025

  67. [75]

    snout") is sampling a

    Zhu, B., Lin, B., Ning, M., Yan, Y., Cui, J., HongFa, W., Pang, Y., Jiang, W., Zhang, J., Li, Z., et al.: Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment. In: ICLR (2024) 20 E. Tsonis et al. Supplementary Material What Imag...

    2024

  68. [76]

    Choose ONE: {items_list}

    ITEM (I): The general category of the object the yellow tip is touching. Choose ONE: {items_list}

  69. [77]

    OBJECT (O): A specific, open-set name for the exact object or surface the yellow tip is touching

  70. [78]

    Choose ONE: {environments_list}

    ENVIRONMENT (E): The general environment/setting. Choose ONE: {environments_list}

  71. [80]

    invisible

    INFERRED SMELLS (S): Infer likely smells from the surrounding environment ONLY. ***CRITICAL: EXCLUDE the smell of the OBJECT (O) itself. *** Focus on "invisible" scents that are likely in the air (e.g., traffic exhaust, humidity, air conditioning, distant greenery). Respond in...

  72. [81]

    Choose exactly ONE from this list: {items_list}

    ITEM (I): The main item/object that would be shown. Choose exactly ONE from this list: {items_list}

  73. [82]

    OBJECT (O): A specific, open-set name for the exact object

  74. [83]

    Choose exactly ONE from this list: {environments_list}

    ENVIRONMENT (E): The general environment/setting. Choose exactly ONE from this list: {environments_list}

  75. [84]

    park, garage, kitchen, office, street, garden, workshop)

    CONTEXT (Ctx): A short, open-set phrase describing the background/setting (e.g. park, garage, kitchen, office, street, garden, workshop). Anything that describes where the scene is

  76. [85]

    common sense

    INFERRED SMELLS (S): Infer likely smells from the surrounding environment ONLY. ***CRITICAL: EXCLUDE the smell of the OBJECT (O) itself. *** Focus on ’invisible’ scents that are likely in the air (e.g., traffic exhaust, humidity, air conditioning, distant greenery). Respond in...

Pith tools

Reviewed July 8, 2026 · model on record in the stance chip above.