Pith. sign in

REVIEW 5 major objections 4 minor 52 references

DEFEND: A Large-scale 1M Dataset and Foundation Model for Tobacco Addiction Prevention

T0 review · 5 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a domain-specific foundation model pretrained on a new 1.13-million-image tobacco dataset, Tobacco-1M, outperforms generic vision-language models on tobacco classification, visual question answering, and zero-shot…

desk verdict The dataset is a real contribution, but the paper's external validation is compromised by overlapping numbers and unreleased artifacts. read the letter →

arxiv 2501.13950 v1 pith:2UE6WUOD submitted 2025-01-20 cs.CV

Naga VS Raviteja Chappa , Matthew Shepard , Connor McCurtain , Charlotte McCormick , Page Daniel Dobbs , Khoa Luu This is my paper · ORCID
classification cs.CV
keywords tobaccoproductclassificationfoundationmodelTobacco-1Mdatasetvision-languagepretrainingzero-shotvisualquestionansweringpublichealthsurveillancehierarchicallabels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that tobacco product surveillance has been blocked by small, narrowly scoped datasets, and that a million-image dataset with hierarchical labels plus a model trained specifically on it closes the gap. It introduces Tobacco-1M, roughly 1.13 million images across 75 product categories with four levels of labels (product type, usage context, content purpose, health impact) and free-text descriptions. On top of it, DEFEND combines a teacher-student vision backbone, a feature enhancement module, local-global coherence loss, and contrastive image-text alignment. The paper reports 83.1% top-5 accuracy on the external PHAD benchmark, 73.8% VQA accuracy on Tobacco-1M, and 45.6% zero-shot accuracy on novel categories, all beating generic vision-language baselines. If true, this is evidence that domain-specific pretraining, rather than generic foundation models, is the route to usable automated tobacco monitoring.

What carries the argument

The central mechanism is a teacher-student multimodal architecture: a teacher encoder produces a global image feature while a student encoder processes saliency-sampled local patches, with exponential moving average updates distilling knowledge from student to teacher. A Feature Enhancement Module applies intra- and cross-modal attention between BERT text features and both global and patch visual features. Three objectives train the model: a contrastive loss aligning image-text pairs, a patch-coherence loss forcing local features to agree with the global view, and a description-generation loss. The load-bearing idea is that local patch features aligned to textual product descriptions let the model notice warning labels, brand marks, and other fine-grained cues that whole-image encoders miss.

What would settle it

Independently re-annotate a random sample of, say, 1,000 Tobacco-1M images with two or more trained annotators and measure agreement; if agreement is poor, the supervised accuracy numbers are not trustworthy. Separately, run DEFEND zero-shot on newly collected social-media posts of products not in the 75 categories; if accuracy is near chance rather than the claimed 45.6%, the generalization claim fails.

Watch

Extended reading notes

Core claim

The central claim is that a domain-specific foundation model, DEFEND, pretrained on Tobacco-1M, outperforms generic vision-language models on tobacco product understanding. On the PHAD classification benchmark DEFEND reaches 78.3% top-1 and 83.1% top-5 accuracy by fine-tuning only the linear head, beating ImageNet-1K-pretrained ResNet, EfficientNet, and ViT as well as DINO, MAE, and CoCa pretrained on Tobacco-1M. On the VQA task it scores 73.8% overall accuracy across product classification, usage context, content description, and health impact questions, ahead of MDETR, MiniGPT-4, and Flamingo. In zero-shot classification of novel PHAD categories it reaches 45.6%, outperforming CLIP, CoCa, and MDETR. The paper's position is that large-scale domain data and architecture choices tuned to fine-grained product attributes are what produce these gains.

Load-bearing premise

The claim depends on the Tobacco-1M labels being accurate and representative of tobacco products as they appear online; the paper says public-health experts manually verified entries but reports no inter-annotator agreement, label-quality checks, or sampling details, so noisy or biased labels would inflate the reported numbers.

Editorial extensions

If this is right

  • Public-health agencies could use a Tobacco-1M-pretrained model to automatically flag novel products in social-media posts, reducing reliance on manual surveillance.
  • The hierarchical labels allow monitoring not just what product appears but how it is marketed and what health claims are made, supporting regulatory compliance checks.
  • The 45.6% zero-shot result implies the model can recognize products it was never trained on, which is exactly what regulators need when new nicotine products enter the market.
  • Domain-specific pretraining shifts the baseline: future tobacco-vision work should compare against a domain-pretrained model, not only ImageNet-1K or generic vision-language models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not report annotation quality or sampling source details; before deploying the dataset for surveillance, an independent label audit would be needed, and the 45.6% zero-shot figure should be re-measured on independently collected social-media images.
  • The same pretraining recipe (large hierarchical dataset plus local-global coherence plus image-text description loss) could transfer to other product-safety domains, such as detecting illicit drugs, counterfeit goods, or restricted advertising, but that transfer is untested.
  • Because Tobacco-1M is collected from official catalogs and online sources, real social-media imagery with backgrounds, hands, and occlusion may shift the distribution; the zero-shot result on PHAD videos is only partial evidence of field readiness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper presents Tobacco-1M, a claimed one-million-image dataset of tobacco products with hierarchical labels across 75 categories, and DEFEND, a vision-language foundation model combining a feature enhancement module, a local-global patch coherence loss, an image-text contrastive loss, and a description-generation loss. The authors report that DEFEND achieves 78.3% top-1 and 83.1% top-5 accuracy on the external PHAD classification benchmark, 73.8% accuracy on a Tobacco-1M VQA task, and 45.6% zero-shot accuracy on novel PHAD categories, comparing favorably against CLIP, CoCa, MDETR, and ImageNet-pretrained vision backbones. The paper is a dataset and model contribution motivated by tobacco surveillance and public health.

Significance. If the results are correct and the dataset is actually released, this would be a substantial empirical contribution: Tobacco-1M is approximately 140 times larger than previous tobacco-specific image datasets, and a domain-specific multimodal model that outperforms generic vision-language models on tobacco product understanding would be useful for public-health monitoring. The proposed architecture is a reasonable combination of existing techniques, and the paper makes concrete claims that could be reproduced. However, none of the central empirical claims can currently be checked: there is no code, no dataset release, no datasheet, no annotation-quality measurement, and no external evaluation protocol that can be verified. The paper also has no machine-checked proofs or parameter-free derivations; its contribution is entirely empirical, which makes the consistency and verifiability issues decisive for the current version.

major comments (5)
  1. [§5.3, Table 3 vs §5.2, Table 2]
  2. [§5.1, Table 1]
  3. [§5.3, Table 4]
  4. [§3.2, §3.3]
  5. [Tables 2–5]
minor comments (4)
  1. [Abstract]
  2. [§2]
  3. [§4.3.2]
  4. [References]

Circularity Check

2 steps flagged · score 6.0 of 10

The PHAD external classification result is numerically identical to the in-distribution Tobacco-1M ablation, and the external benchmark is the authors' own PHAD dataset; the central transfer claim is therefore not independently established.

  1. fitted input called prediction [Table 2 vs Table 3; Sections 5.2 and 5.3]
    "Table 2 (Tobacco-1M ablation): '✓ ✓ ✓ ✓ 78.3 83.1' for ViT-base/16. Table 3 (PHAD classification): 'DEFEND (Ours) ✓ Tobacco-1M 78.3 83.1'."

    The paper reports PHAD as an external transfer benchmark ('We evaluated DEFEND on the PHAD dataset by fine-tuning only the linear classification layer') and uses the resulting 78.3/83.1 to claim a 5.8% gain over ImageNet-pretrained models. But those exact one-decimal Acc@1/Acc@5 values are identical to the full ViT-base/16 model's ablation on Tobacco-1M, the same dataset used for pretraining. As printed, therefore, the 'PHAD prediction' is numerically the same quantity as the in-distribution Tobacco-1M evaluation; it does not provide an independent measurement of transfer. Either the external table was transcribed from the internal ablation, or the two evaluations coincide to one decimal by chance, which would itself require explanation.

  2. self citation load bearing [Section 5.3 and References [8]]
    "'Public health advocacy dataset: A dataset of tobacco usage videos from social media. TechRxiv, 2024' — reference [8] (authors Chappa, McCormick, Gongora, Dobbs, Luu). Section 5.3: 'We evaluated DEFEND on the PHAD dataset by fine-tuning only the linear classification layer.'"

    PHAD, the sole external dataset for the classification and zero-shot claims, is the authors' own prior dataset. The paper does not provide dataset access, annotation-quality metrics, or an independent replication, so the purported external validation rests on a self-citation that is not independently checkable. This is load-bearing because removing PHAD leaves only in-distribution Tobacco-1M results, which cannot support the claimed generalization.

full rationale

DEFEND is an empirical systems paper; there is no formal derivation to unwind. The measured in-distribution VQA on a held-out Tobacco-1M split is not circular by construction, and zero-shot on PHAD would be a genuine novel-category test if the PHAD numbers were independently measured. However, two load-bearing issues undercut the claimed external validation. First, the paper's PHAD classification result (78.3% Acc@1, 83.1% Acc@5) is exactly the same one-decimal pair as the full ViT-base/16 model's ablation on Tobacco-1M (Table 2). As printed, the external PHAD 'prediction' is numerically identical to the in-distribution Tobacco-1M evaluation, so it cannot serve as independent evidence of domain transfer; the claimed +5.8% over ImageNet baselines is therefore resting on an internal result unless the table is a transcription error. Second, the external benchmark PHAD is reference [8], a prior dataset by the same research group; no dataset access, annotation-quality metrics, or independent replication is provided, so the external-validation load is carried by a self-citation. The additional inconsistencies in dataset size (Table 1: 1,128,652; §5.1: 450K+200K+50K=700K and a 700K/150K/150K split) are correctness risks rather than circularity. Overall, because a central transfer prediction reduces to the in-distribution number and the external benchmark is self-authored, the paper earns a partial-circularity score of 6.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

All entries in the ledger are empirical assumptions and hyperparameters, not mathematical axioms. The dataset's representativeness and annotation accuracy are the most consequential assumptions, since every performance claim depends on them. The hyperparameters (EMA decay, sampling ratio, temperature, learning rate) are hand-chosen and standard, and their values are not justified with sensitivity analyses.

free parameters (5)
  • EMA decay rate alpha = 0.999
    Chosen by hand for teacher-student EMA updates in Section 4.3.1; standard value but affects training dynamics.
  • Adaptive sampling ratio lambda = 0.3
    Selects salient patches in Section 4.2; chosen by hand, no sensitivity analysis provided.
  • Temperature tau = not specified
    Used in contrastive loss L_cont in Section 4.6; not given a value in the paper.
  • Learning rate = 1e-3
    AdamW learning rate in Section 5.1; standard but part of the training setup.
  • Patch size = 16
    ViT-B/16 patch size in Section 5.1; standard.
assumptions (3)
  • domain assumption Manual annotations by public health experts are accurate and the Tobacco-1M dataset is representative of tobacco products in social media.
    Section 3.2 states 'Our team of public health experts manually verified and annotated each entry to ensure the accuracy of the data.' No inter-annotator agreement or quality checks are reported.
  • domain assumption The 8 PHAD categories used for zero-shot evaluation are not present in Tobacco-1M, so the zero-shot result reflects novel classes.
    Section 5.3 states 'We tested our DEFEND's ability to classify novel tobacco products using the PHAD dataset' without demonstrating category disjointness.
  • standard math Standard attention, EMA, and contrastive loss formulations are used as described and behave as intended.
    Sections 4.3-4.6 apply standard transformer and contrastive learning formulas; no new mathematical claims are made.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DEFEND: A Large-scale 1M Dataset and Foundation Model for Tobacco Addiction Prevention." pith.science (2026). https://pith.science/paper/2UE6WUOD

@misc{pith2026250113950,
  author       = {Pith},
  title        = {Pith review of: DEFEND: A Large-scale 1M Dataset and Foundation Model for Tobacco Addiction Prevention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2UE6WUOD}},
  note         = {Machine review of arXiv:2501.13950}
}
read the original abstract

While tobacco advertising innovates at unprecedented speed, traditional surveillance methods remain frozen in time, especially in the context of social media. The lack of large-scale, comprehensive datasets and sophisticated monitoring systems has created a widening gap between industry advancement and public health oversight. This paper addresses this critical challenge by introducing Tobacco-1M, a comprehensive dataset of one million tobacco product images with hierarchical labels spanning 75 product categories, and DEFEND, a novel foundation model for tobacco product understanding. Our approach integrates a Feature Enhancement Module for rich multimodal representation learning, a Local-Global Visual Coherence mechanism for detailed feature discrimination, and an Enhanced Image-Text Alignment strategy for precise product characterization. Experimental results demonstrate DEFEND's superior performance, achieving 83.1% accuracy in product classification and 73.8% in visual question-answering tasks, outperforming existing methods by significant margins. Moreover, the model exhibits robust zero-shot learning capabilities with 45.6% accuracy on novel product categories. This work provides regulatory bodies and public health researchers with powerful tools for monitoring emerging tobacco products and marketing strategies, potentially revolutionizing approaches to tobacco control and public health surveillance.

Figures

Figures reproduced from arXiv: 2501.13950 by the authors.

Figure 1
Figure 1. Examples of Our Tobacco-1M Dataset for Tobacco Product Understanding. The left section displays visual samples from [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The Distribution of Tobacco Product Categories [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparisons of Previous Methods. Prior works [14, 18, 45, 47] demonstrate significant limitations in tobacco prod￾uct analysis: naive feature fusion fails to capture nuanced prod￾uct characteristics due to oversimplified integration, basic classi￾fication architecture overlooks critical regulatory elements. DE￾FEND overcomes these limitations through Feature Enhancement Module (FEM), enabling precise feature correla… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Overview of our proposed DEFEND Framework. We employ a dual-stream design where text input is processed through a Text Encoder while image input follows two parallel paths through a Teacher-Student architecture. The Feature Enhancement Module enhances these multimodal …
Figure 5
Figure 5. Figure 5: Attention Visualization. Compared to MAE [18], our model demonstrates enhanced sensitivity to product-specific de￾tails in tobacco imagery. The model effectively highlights key product features and warning labels, maintaining robust attention even in challenging scenar…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 35 canonical work pages

  1. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736,

  2. [2]

    Emergent Visual-Semantic Hierarchies in Image-Text Representations

    Morris Alper and Hadar Averbuch-Elor. Emergent visual- semantic hierarchies in image-text representations. arXiv preprint arXiv:2407.08521, 2024. 2

    work page Pith review arXiv 2024

  3. [3]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In ICCV, 2021. 2, 8

    work page 2021

  4. [4]

    SoGAR: Self-supervised Spatiotemporal Attention-based Social Group Activity Recognition

    Naga VS Chappa, Pha Nguyen, Alexander H Nelson, Han- Seok Seo, Xin Li, Page Daniel Dobbs, and Khoa Luu. Sogar: Self-supervised spatiotemporal attention-based social group activity recognition. arXiv preprint arXiv:2305.06310, 2023. 2

    work page Pith review arXiv 2023

  5. [5]

    Spartan: Self-supervised spatiotemporal transformers ap- proach to group activity recognition

    Naga VS Chappa, Pha Nguyen, Alexander H Nelson, Han- Seok Seo, Xin Li, Page Daniel Dobbs, and Khoa Luu. Spartan: Self-supervised spatiotemporal transformers ap- proach to group activity recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5157–5167, 2023. 2

    work page 2023

  6. [6]

    Advanced deep learning techniques for tobacco usage assessment in tiktok videos

    Naga VS Raviteja Chappa, Charlotte McCormick, Su- sana Rodriguez Gongora, Page Daniel Dobbs, and Khoa Luu. Advanced deep learning techniques for tobacco usage assessment in tiktok videos. In 2024 IEEE Green Technolo- gies Conference (GreenTech), pages 162–163. IEEE, 2024. 2

    work page 2024

  7. [7]

    React: Recognize every action everywhere all at once

    Naga VS Raviteja Chappa, Pha Nguyen, Page Daniel Dobbs, and Khoa Luu. React: Recognize every action everywhere all at once. Machine Vision and Applications , 35(4):102,

    work page

  8. [8]

    Public health advocacy dataset: A dataset of tobacco usage videos from social media

    Naga Venkata Sai Raviteja Chappa, Charlotte McCormick, Susana Rodriguez Gongora, Page Daniel Dobbs, and Khoa Luu. Public health advocacy dataset: A dataset of tobacco usage videos from social media. TechRxiv, 2024. 2, 3, 6

    work page 2024

Show all 52 references
  1. [9]

    Hatt- flow: Hierarchical attention-flow mechanism for group- activity scene graph generation in videos

    Naga Venkata Sai Raviteja Chappa, Pha Nguyen, Thi Hoang Ngan Le, Page Daniel Dobbs, and Khoa Luu. Hatt- flow: Hierarchical attention-flow mechanism for group- activity scene graph generation in videos. Sensors, 24(11): 3372, 2024. 2

    2024

  2. [10]

    Feddat: An approach for foundation model finetuning in multi-modal heterogeneous federated learning

    Haokun Chen, Yao Zhang, Denis Krompass, Jindong Gu, and V olker Tresp. Feddat: An approach for foundation model finetuning in multi-modal heterogeneous federated learning. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 11285–11293, 2024. 2

    2024

  3. [11]

    Improved baselines with momentum contrastive learning

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning

  4. [12]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 5, 7

    2018 arXiv

  5. [13]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  6. [14]

    Multiscale vision transformers

    Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feicht- enhofer. Multiscale vision transformers. arXiv preprint arXiv:2104.11227, 2021. 4

    2021 arXiv

  7. [15]

    Multi-modal transfer learning between biological foun- dation models

    Juan Jose Garau-Luis, Patrick Bordes, Liam Gonzalez, Masa Roller, Bernardo P de Almeida, Lorenz Hexemer, Christo- pher Blum, Stefan Laurent, Jan Grzegorzewski, Maren Lang, et al. Multi-modal transfer learning between biological foun- dation models. arXiv preprint arXiv:2406.14...

    2024 arXiv

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 8

    2016

  9. [17]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In CVPR, 2020. 2

    2020

  10. [18]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 4, 7, 8

    2022

  11. [19]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. In International conference on machine learning, pages 4904–4916. PMLR,

  12. [20]

    Mdetr- modulated detection for end-to-end multi-modal understand- ing

    Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. Mdetr- modulated detection for end-to-end multi-modal understand- ing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 1780–1790, 2021. 8

    2021

  13. [21]

    Grounding foundation models through federated transfer learning: A general framework

    Yan Kang, Tao Fan, Hanlin Gu, Xiaojin Zhang, Lixin Fan, and Qiang Yang. Grounding foundation models through federated transfer learning: A general framework. arXiv preprint arXiv:2311.17431, 2023. 2

    2023 arXiv

  14. [22]

    Ma- chine learning models of tobacco susceptibility and current use among adolescents from 97 countries in the global youth tobacco survey, 2013-2017

    Nayoung Kim, Wei-Yin Loh, and Danielle E McCarthy. Ma- chine learning models of tobacco susceptibility and current use among adolescents from 97 countries in the global youth tobacco survey, 2013-2017. PLOS Global Public Health , 1 (12):e0000060, 2021. 2 9

    2013

  15. [23]

    Understanding e-cigarette con- tent and promotion on youtube through machine learning

    Grace Kong, Alex Sebastian Schott, Juhan Lee, Hassan Dashtian, and Dhiraj Murthy. Understanding e-cigarette con- tent and promotion on youtube through machine learning. Tobacco control, 32(6):739–746, 2023. 2

    2023

  16. [24]

    A multimodal deep learning architecture for smoking detec- tion with a small data approach

    R ´obert Lakatos, P´eter Pollner, Andr´as Hajdu, and Tam´as Jo´o. A multimodal deep learning architecture for smoking detec- tion with a small data approach. Frontiers in Artificial Intel- ligence, 7:1326050, 2024. 2

    2024

  17. [25]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 2

    2024

  18. [26]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 7

    2017 arXiv

  19. [27]

    Sgdr: Stochas- tic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochas- tic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016. 7

    2016 arXiv

  20. [28]

    Visual relationship detection with language priors

    Cewu Lu, Ranjay Krishna, Michael Bernstein, and Li Fei- Fei. Visual relationship detection with language priors. In Proceedings of the European Conference on Computer Vi- sion (ECCV), pages 852–869, 2016. 2

    2016

  21. [29]

    Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, 32, 2019. 8

    2019

  22. [30]

    Molfm: A multimodal molecular foundation model

    Yizhen Luo, Kai Yang, Massimo Hong, Xing Yi Liu, and Zaiqing Nie. Molfm: A multimodal molecular foundation model. arXiv preprint arXiv:2307.09484, 2023. 2

    2023 arXiv

  23. [31]

    A scalable hierarchical distributed language model

    Andriy Mnih and Geoffrey E Hinton. A scalable hierarchical distributed language model. Advances in neural information processing systems, 21, 2008. 2

    2008

  24. [32]

    Influence of user profile attributes on e-cigarette– related searches on youtube: Machine learning clustering and classification

    Dhiraj Murthy, Juhan Lee, Hassan Dashtian, Grace Kong, et al. Influence of user profile attributes on e-cigarette– related searches on youtube: Machine learning clustering and classification. JMIR infodemiology, 3(1):e42218, 2023. 2

    2023

  25. [33]

    Using Computer Vision to Detect E-cigarette Con- tent in TikTok Videos

    Dhiraj Murthy, Rachel R Ouellette, Tanvi Anand, Srijith Radhakrishnan, Nikhil C Mohan, Juhan Lee, and Grace Kong. Using Computer Vision to Detect E-cigarette Con- tent in TikTok Videos. Nicotine & Tobacco Research , 26 (Supplement 1):S36–S42, 2024. 2, 3

    2024

  26. [34]

    Insect- foundation: A foundation model and large-scale 1m dataset for visual insect understanding

    Hoang-Quan Nguyen, Thanh-Dat Truong, Xuan Bac Nguyen, Ashley Dowling, Xin Li, and Khoa Luu. Insect- foundation: A foundation model and large-scale 1m dataset for visual insect understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, ...

    2024

  27. [35]

    Type-to-track: Retrieve any object via prompt-based track- ing

    Pha Nguyen, Kha Gia Quach, Kris Kitani, and Khoa Luu. Type-to-track: Retrieve any object via prompt-based track- ing. Advances in Neural Information Processing Systems , 36, 2024. 2

    2024

  28. [36]

    Classification of lapses in smokers attempting to stop: A su- pervised machine learning approach using data from a pop- ular smoking cessation smartphone app

    Olga Perski, Kezhi Li, Nikolas Pontikos, David Simons, Stephanie P Goldstein, Felix Naughton, and Jamie Brown. Classification of lapses in smokers attempting to stop: A su- pervised machine learning approach using data from a pop- ular smoking cessation smartphone app. Nicotin...

    2023

  29. [37]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  30. [38]

    Efficientnet: Rethinking model scaling for convolutional neural networks

    Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105–6114. PMLR,

  31. [39]

    Learning to compose dynamic tree structures for visual contexts

    Kaihua Tang, Hanwang Zhang, Baoyuan Wu, Wenhan Luo, and Wei Liu. Learning to compose dynamic tree structures for visual contexts. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 6619–6628, 2019. 2

    2019

  32. [40]

    Scalable Surveil- lance of E-Cigarette Products on Instagram and TikTok Us- ing Computer Vision

    Julia Vassey, Chris J Kennedy, Ho-Chun Herbert Chang, Ashley S Smith, and Jennifer B Unger. Scalable Surveil- lance of E-Cigarette Products on Instagram and TikTok Us- ing Computer Vision. Nicotine & Tobacco Research, 26(5): 552–560, 2023. 2, 3

    2023

  33. [41]

    Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors

    Chien-Yao Wang, Alexey Bochkovskiy, and Hong- Yuan Mark Liao. Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7464–7475, 2023. 2

    2023

  34. [42]

    Actionclip: A new paradigm for video action recognition

    Mengmeng Wang, Jiazheng Xing, and Yong Liu. Actionclip: A new paradigm for video action recognition. arXiv preprint arXiv:2109.08472, 2021. 2

    2021 arXiv

  35. [43]

    Ip102: A large-scale benchmark dataset for insect pest recognition

    Xiaoping Wu, Chi Zhan, Yu-Kun Lai, Ming-Ming Cheng, and Jufeng Yang. Ip102: A large-scale benchmark dataset for insect pest recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8787–8796, 2019. 2, 3

    2019

  36. [44]

    mplug-2: A modularized multi-modal foundation model across text, image and video

    Haiyang Xu, Qinghao Ye, Ming Yan, Yaya Shi, Jiabo Ye, Yuanhong Xu, Chenliang Li, Bin Bi, Qi Qian, Wei Wang, et al. mplug-2: A modularized multi-modal foundation model across text, image and video. In International Con- ference on Machine Learning, pages 38728–38748. PMLR,

  37. [45]

    Seed the views: Hi- erarchical semantic alignment for contrastive representation learning

    Haohang Xu, Xiaopeng Zhang, Hao Li, Lingxi Xie, Wen- rui Dai, Hongkai Xiong, and Qi Tian. Seed the views: Hi- erarchical semantic alignment for contrastive representation learning. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 45(3):3753–3767, 2023. 4

    2023

  38. [46]

    Linguistic structures as weak supervision for visual scene graph generation

    Keren Ye and Adriana Kovashka. Linguistic structures as weak supervision for visual scene graph generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8289–8299, 2021. 2

    2021

  39. [47]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mo- jtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022. 4, 6, 8

    2022 arXiv

  40. [48]

    Bridging knowledge graphs to generate scene graphs

    Alireza Zareian, Svebor Karaman, and Shih-Fu Chang. Bridging knowledge graphs to generate scene graphs. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIII 16, pages 606–623. Springer, 2020. 2

    2020

  41. [49]

    Learning visual commonsense for robust scene graph 10 generation

    Alireza Zareian, Zhecan Wang, Haoxuan You, and Shih-Fu Chang. Learning visual commonsense for robust scene graph 10 generation. In Computer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Pro- ceedings, Part XXIII 16, pages 642–657. Springer, 2020

    2020

  42. [50]

    Learning human action recognition representations without real humans

    Howard Zhong, Samarth Mishra, Donghyun Kim, SouY- oung Jin, Rameswar Panda, Hilde Kuehne, Leonid Karlin- sky, Venkatesh Saligrama, Aude Oliva, and Rogerio Feris. Learning human action recognition representations without real humans. Advances in Neural Information Processing Sy...

    2024

  43. [51]

    Learning to generate scene graph from natural language supervision

    Yiwu Zhong, Jing Shi, Jianwei Yang, Chenliang Xu, and Yin Li. Learning to generate scene graph from natural language supervision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1823–1834, 2021. 2

    2021

  44. [52]

    MiniGPT-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. MiniGPT-4: Enhancing vision-language understanding with advanced large language models. In The Twelfth International Conference on Learning Representa- tions, 2024. 8 11

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.