Pith. sign in

REVIEW 5 major objections 5 minor 32 references

Probabilistic cross-modal retrieval can run as a single matrix-vector multiply on compute-in-memory hardware.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 10:01 UTC pith:OPRLUUKV

load-bearing objection A clever, well-motivated reformulation for CiM-compatible probabilistic retrieval, but the accuracy gains over deterministic baselines likely come from end-to-end training rather than the polynomial surrogate, a mechanism the paper never isolates. the 5 major comments →

arxiv 2607.20358 v1 pith:OPRLUUKV submitted 2026-07-22 cs.ET cs.AR

PolySim: Deterministic Polynomial Surrogates for Cross-Modal Retrieval on CiM

Xinzhao Li , Charles Power , Pengyu Ren , Jongun Won , Likai Pei , Yuting Hu , Jinjun Xiong , Alptekin Vardar
show 5 more authors
Ningyuan Cao Xiaobo Sharon Hu Thomas K\"ampfe Kai Ni Ruiyang Qin
This is my paper
classification cs.ET cs.AR
keywords cross-modal retrievalprobabilistic embeddingscompute-in-memorycrossbar arraysChebyshev polynomial approximationorder-bilinear similaritydeterministic inferenceretrieval acceleration
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that probabilistic cross-modal retrieval—where each sample is a Gaussian rather than a point—can be made deterministic without sacrificing its accuracy advantage, by approximating each Gaussian dimension with low-order Chebyshev coefficients and scoring with a learned order-bilinear kernel. If true, this removes the sampling step that blocks such models from compute-in-memory (CiM) crossbars, reducing inference to a single matrix-vector product. Across six video, image, and audio benchmarks, the method matches or beats the sampling-based probabilistic baseline while improving recall over deterministic embeddings by up to 10.3 percentage points, and it degrades gracefully under realistic device noise and 8-bit quantization. The paper's central bet is that distributional similarity can be linearized: the learned bilinear interactions substitute for Monte Carlo averaging, and the reported numbers support that bet.

Core claim

On its own terms, PolySim establishes that the variance signal in a Gaussian embedding—what its probabilistic predecessors extract by Monte Carlo sampling—can be repackaged as deterministic polynomial coefficients and read out through a learned order-bilinear similarity. The resulting score is algebraically a dot product in a transformed space, so the database can be transformed offline and each query costs one crossbar-compatible matrix-vector multiply. The paper reports that this surrogate matches or exceeds the accuracy of sampling-based probabilistic retrieval across six benchmarks and beats plain deterministic embeddings by up to 10.3% R@1, while maintaining robustness under simulated d

What carries the argument

The load-bearing piece is the polynomial surrogate embedding: for each dimension of a Gaussian output, the marginal density is expanded in a truncated Chebyshev basis, and the coefficients {c_{d,k}} from all dimensions are concatenated into one deterministic vector. Similarity is then computed by a learnable order-bilinear kernel A that mixes polynomial orders per dimension, plus an optional mean-residual term; because the bilinear form can be rewritten as an inner product after applying A^{1/2}, the whole inference path collapses to a single matrix-vector multiplication. The ablations show the coefficients alone are useless under a plain dot product (0.1 R@1 on one benchmark), so the learne

Load-bearing premise

The claim rests on the assumption that a low-order per-dimension Chebyshev expansion of each Gaussian, scored by a learned order-bilinear kernel, preserves enough of the distribution's similarity structure to outperform deterministic embeddings—an empirical assumption with no stated error bound, and one that the paper's own ablation shows fails if the learned kernel is replaced by a plain dot product.

What would settle it

A direct check: on a held-out set, compute the true expected similarity under the Gaussian densities via a large number of Monte Carlo samples and compare its ranking to PolySim's surrogate ranking; if the surrogate's ranking diverges on pairs where variance differs, or if setting all variances to zero does not degrade PolySim below the deterministic baseline, the claim that uncertainty is being captured would be falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If PolySim is correct, probabilistic retrieval can run on conventional crossbar arrays at the same single-MVM cost as deterministic retrieval, eliminating the 5–23x latency overhead of sampling and the off-chip random-number and nonlinear-accumulation hardware it required.
  • Because the method treats the Gaussian (mean, variance) as an interface, any existing probabilistic projector can be retrofitted with the polynomial surrogate and learned kernel without retraining the backbone.
  • At 8-bit precision, retrieval accuracy holds near full precision, indicating standard multi-level memory cells suffice; the sharp drop at 4-bit and 2-bit defines a precision floor for current crossbar hardware.
  • Across video, image, and audio benchmarks, the surrogate beats deterministic baselines, suggesting the variance signal, once linearized, is a genuine accuracy source rather than a modeling artifact.
  • The equivalence to a dot product in a transformed space means database-side transformation is a one-time offline cost, so query-side hardware requirements stay identical to classical CiM retrieval.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves open whether the learned order-bilinear kernel approximates a universal function of the Gaussian densities; if so, the same kernel might transfer across datasets or modalities without retraining, a property the paper does not test.
  • The near-total collapse of the naive coefficient dot product suggests the polynomial representation is not intrinsically informative; the real discovery may be that a learned bilinear map can synthesize distributional similarity from low-order marginals—an insight the ablations support but do not explain mechanistically.
  • A natural stress test the authors do not report: zero out the predicted variance at inference. If PolySim still outperforms the deterministic baseline, its gain cannot be attributed to uncertainty modeling, and the paper's framing would need revision.
  • The 2-bit degradation points to a co-design opportunity: if coefficient vectors have exploitable structure (e.g., few dominant orders), custom analog cells or hybrid arrays tuned to that sparsity could push below the observed precision floor.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. PolySim proposes a deterministic reformulation of probabilistic cross-modal retrieval for compute-in-memory (CiM) hardware. It converts per-dimension Gaussian embeddings (μ, σ) into low-order Chebyshev polynomial coefficients, concatenates them into a deterministic surrogate embedding, and computes similarity via a learned order-bilinear kernel (Eq. 5). The method is trained end-to-end with an asymmetric multi-positive InfoNCE loss (Eqs. 6–7). Experiments on six benchmarks (video, image, audio) report R@1 improvements over deterministic DET baselines by up to 10.3% and parity with PCME, while reducing the similarity-scoring stage to a single crossbar-compatible MVM. CrossSim simulations under device non-idealities and post-training quantization experiments are presented as evidence of hardware robustness.

Significance. The paper addresses a real and timely problem: probabilistic retrieval methods like PCME are sampling-based and therefore structurally incompatible with deterministic single-step crossbar MVMs. The proposed reformulation—replacing Monte Carlo sampling with polynomial coefficient embeddings and a learnable bilinear scorer—is a plausible route to bringing uncertainty-aware embeddings to CiM, and the CrossSim evaluation is a useful step. If validated with properly controlled experiments, the approach could be a significant contribution to edge-side cross-modal retrieval and CiM co-design. However, the significance currently hinges on whether the polynomial surrogate itself, rather than the learned scoring and end-to-end training, is responsible for the reported gains; the paper does not yet provide this evidence.

major comments (5)
  1. [§4.1, Table 1] The comparison against DET is not controlled. DET is described as the backbone's default point-vector with dot-product similarity, while PolySim trains a Gaussian projection head, the polynomial coefficients, and the order-bilinear matrix A end-to-end with a new multi-positive InfoNCE objective. Thus the reported improvement over DET could stem entirely from the additional trainable components, not from the polynomial surrogate. The paper must include a deterministic control that receives the same distributional inputs (μ, log σ) and the same training objective but omits the polynomial expansion (e.g., a linear or bilinear scorer on (μ, log σ)). Without such a baseline, the central claim that the polynomial representation preserves distributional information is unsupported.
  2. [§3.3, Eqs. (3)–(4)] The Chebyshev expansion is not specified precisely. Chebyshev polynomials are orthogonal on a finite interval, but the Gaussian density is defined on R. The paper states that 'coefficients are obtained by evaluating the Gaussian density under this basis' but does not define the mapping from x to the Chebyshev argument, the truncation interval, the normalization, or the formula for c_{d,k}. This is load-bearing because the entire surrogate embedding and all subsequent similarity computations are defined in terms of these coefficients. As written, the method cannot be reproduced or formally analyzed.
  3. [§3.4, Eq. (5)] The claim that similarity can be rewritten as an inner product via ̃c = A^{1/2} c requires A to be positive semidefinite. The paper only says that A is symmetrized during training; it does not impose PSD. If A is not PSD, A^{1/2} may not exist and the stated transformation is invalid. A single-MVM form can still be achieved by pre-transforming database vectors with A itself (storing A c_v), so this is correctable, but the current text is technically inaccurate. The shared A across all dimensions is also a strong assumption that should be justified or ablated.
  4. [§4.3, Table 4] The ablation shows that a dot product over the polynomial coefficient embedding yields near-random performance (0.1 R@1 on MSRVTT-ImageBind), while a diagonal matrix raises it to 39.7. This indicates that the learned scoring function, not the polynomial representation alone, is responsible for most of the measured performance. The paper's statement that the polynomial representation 'preserves distributional information' is therefore not supported. A controlled experiment feeding (μ, log σ) to the same learnable bilinear scoring function is needed to isolate the contribution of the polynomial expansion.
  5. [Abstract and Table 3] The claim that 'inference is a single crossbar-compatible matrix-vector multiplication' applies only to the similarity-scoring step after the polynomial coefficients are computed. The transformation from (μ, σ) to Chebyshev coefficients is a nonlinear operation that cannot be a single MVM and is not accounted for in Table 3. Please specify how the query-side coefficient computation is executed on the crossbar (e.g., via a separate MLP, lookup table, or off-chip preprocessing) and state clearly which stages of the pipeline are included in the '1 MVM' claim.
minor comments (5)
  1. [References] Reference [25] is cited for OpenCLIP, but the listed paper (Wu et al., 'Large-scale contrastive language-audio pretraining...') is CLAP. Please cite the actual OpenCLIP source (e.g., Ilharco et al., 2021).
  2. [Table 2] The row for RRAM1 (Device-1) is labeled as 1-level but lists four deviation values (L0–L3). Clarify whether this device has four levels or the table columns are misaligned.
  3. [§4.1 and Table 6] The reference to 'Table 6' for per-epoch training time appears in §4.1, but Table 6 is in §4.4. Reorder or renumber for consistency.
  4. [§4.3] The text says 'the similarity function defined in Eq. (4)' but Eq. (4) is the polynomial embedding, not the similarity function (Eq. 5). Correct the cross-reference.
  5. [§5 or Conclusion] The claim of being 'the first method to enable probabilistic cross-modal retrieval on CiM' should be qualified with a brief discussion of prior attempts to make probabilistic or uncertainty-aware retrieval deterministic for hardware (e.g., moment-based approximations, deterministic KL divergence measures). Otherwise the novelty claim is hard to assess.

Circularity Check

0 steps flagged

No circular derivation: the Chebyshev surrogate is a mathematical transformation, the bilinear kernel is learned and evaluated on held-out data, and the paper's ablations directly test rather than assume the representation's sufficiency.

full rationale

The claimed derivation chain is not circular. Eq. (2)-(4) construct a deterministic embedding by expanding each per-dimension Gaussian density in a Chebyshev basis defined by the standard recurrence (3); this is a mathematical transformation of the input embedding, not a definition in terms of the retrieval target. Eq. (5) defines a similarity with a learnable matrix A, which is trained by the InfoNCE objective (6)-(7) and evaluated on held-out splits of six public benchmarks; no fitted parameter is renamed as a prediction. The central comparisons to DET and PCME are empirical, not derived from the surrogate by construction. The paper's own ablation (Table 4) shows the raw coefficient dot product collapses to 0.1 R@1, making explicit that the polynomial representation alone is not sufficient and that the learned order-bilinear kernel carries most of the measured performance; this is an honest decomposition rather than a self-fulfilling definition. Self-citations (e.g., [6], [8], [11], [15]) appear only as background about prior CiM retrieval acceleration and crossbar constraints, and none is load-bearing for the claim that polynomial surrogates preserve distributional information. The only substantive caveat is experimental-design related — DET appears to be an untrained backbone feature baseline, so the R@1 gain over DET may not isolate the polynomial surrogate from the benefit of end-to-end training — but that is a threat to internal validity, not circularity. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The method introduces no new physical entity: no new particle, force, or dimension. It introduces a new mathematical representation (polynomial surrogate embedding) and a new learned kernel, both of which are parameters, not invented entities. The free parameters listed are the ones the reader must take on faith. The core assumption is that polynomial expansion in the coefficient space preserves semantic uncertainty well enough after learning the kernel.

free parameters (4)
  • Polynomial degree K = 3 (per Figure 6, where K=3 saturates performance)
    K is a design choice fitted to the validation performance on the benchmarks; the paper states 'low-order truncation (K << D) is sufficient' and Figure 6 shows K=2 insufficient, K=3 saturates.
  • Order-bilinear matrix A = Learned during training; no final values reported
    The K x K matrix in Eq. (5) is a learned parameter; it is the main contributor to the accuracy gain over the naive dot product (Table 4).
  • Residual weight gamma = Learned during training; no values reported
    Learnable weight for the residual mean-embedding term in Eq. (5).
  • InfoNCE temperature tau and loss weights alpha, beta = Not reported
    Standard training hyperparameters; not reported in the paper, making reproduction difficult.
axioms (3)
  • domain assumption Chebyshev expansion of a 1D Gaussian density truncated at low order (K=3) preserves enough distributional information for retrieval.
    This is the core modeling assumption; Figure 6 supports it empirically on two datasets, but no theoretical guarantee is given.
  • domain assumption The learned order-bilinear similarity in coefficient space can serve as a valid proxy for distributional similarity between Gaussian embeddings.
    Eq. (5) defines the surrogate; no error bound or equivalence theorem is provided. Table 4 shows the surrogate outperforms a naive dot product by a large margin, indicating the learned kernel is doing the work.
  • domain assumption Crossbar arrays perform only deterministic single-step MVM without on-chip RNG or nonlinear accumulation.
    Section 2.3; standard and reasonable for conventional CiM, but it is the premise that motivates the whole method.

pith-pipeline@v1.3.0-alltime-deepseek · 11651 in / 7659 out tokens · 49254 ms · 2026-08-01T10:01:12.135204+00:00 · methodology

0 comments
read the original abstract

Cross-modal retrieval on edge devices benefits from probabilistic embeddings that capture semantic uncertainty, but deploying them on compute-in-memory (CiM) hardware remains an open problem. The core difficulty is a sampling gap: probabilistic methods such as PCME rely on Monte Carlo sampling and nonlinear distance evaluation at inference, which are fundamentally incompatible with CiM crossbar arrays that support only deterministic, single-step matrix-vector multiplication. Few existing probabilistic retrieval methods can be executed on a conventional crossbar. To bridge this gap, we propose PolySim, a framework that reformulates probabilistic retrieval into a fully deterministic pipeline. PolySim approximates each Gaussian embedding dimension using low-order polynomial bases and computes similarity via a learnable order-bilinear kernel, eliminating stochastic sampling while preserving distributional information. In experiments on six benchmarks spanning video, image, and audio retrieval, PolySim improves R@1 over deterministic baselines by up to 10.3\% and matches or exceeds PCME, while reducing inference to a single crossbar-compatible matrix-vector multiplication. CrossSim evaluation under realistic device non-idealities confirms robust deployment on conventional crossbar arrays. To the best of our knowledge, PolySim is the first method to enable probabilistic cross-modal retrieval on CiM hardware.

Figures

Figures reproduced from arXiv: 2607.20358 by Alptekin Vardar, Charles Power, Jinjun Xiong, Jongun Won, Kai Ni, Likai Pei, Ningyuan Cao, Pengyu Ren, Ruiyang Qin, Thomas K\"ampfe, Xiaobo Sharon Hu, Xinzhao Li, Yuting Hu.

Figure 1
Figure 1. Figure 1: Probabilistic retrieval (PCME) improves ac [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) Crossbar-compatible single-step MVM. (b) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed framework. PolySim converts probabilistic Gaussian embeddings into a [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of similarity computation. PCME [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: CrossSim evaluation under device non-idealities across different datasets and backbone models. Blue and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Effect of polynomial degree on retrieval per [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: R@1 retrieval under post-training quantiza [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 2 linked inside Pith

  1. [1]

    Enabling on-device large language model personalization with self-supervised data selection and synthe- sis

    Ruiyang Qin, Jun Xia, Zhenge Jia, Meng Jiang, Ahmed Abbasi, Peipei Zhou, Jingtong Hu, and Yiyu Shi. Enabling on-device large language model personalization with self-supervised data selection and synthe- sis. InProceedings of the 61st ACM/IEEE design automation conference, pages 1–6, 2024

    2024

  2. [2]

    Retrieval- augmented generation for knowledge-intensive nlp tasks, 2021

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval- augmented generation for knowledge-intensive nlp tasks, 2021

    2021

  3. [3]

    Empirical guidelines for deploying llms onto resource- constrained edge devices.ACM Transactions on Design Automation of Electronic Systems, 30(5):1–58, 2025

    Ruiyang Qin, Dancheng Liu, Chenhui Xu, Zheyu Yan, Zhaoxuan Tan, Zhenge Jia, Amir Nassereldine, Jiajie Li, Meng Jiang, Ahmed Ab- basi, et al. Empirical guidelines for deploying llms onto resource- constrained edge devices.ACM Transactions on Design Automation of Electronic Systems, 30(5):1–58, 2025

    2025

  4. [4]

    Joshua Yang

    Qiangfei Xia and J. Joshua Yang. Memristive crossbar arrays for brain- inspired computing.Nature Materials, 18(4):309–323, Apr 2019

    2019

  5. [5]

    A crossbar array of magnetoresistive memory devices for in-memory computing.Nature, 601(7892):211–216, Jan 2022

    Jung et al. A crossbar array of magnetoresistive memory devices for in-memory computing.Nature, 601(7892):211–216, Jan 2022

    2022

  6. [6]

    Robust implementation of retrieval-augmented generation on edge- based computing-in-memory architectures

    Ruiyang Qin, Zheyu Yan, Dewen Zeng, Zhenge Jia, Dancheng Liu, Jianbo Liu, Ahmed Abbasi, Zhi Zheng, Ningyuan Cao, Kai Ni, et al. Robust implementation of retrieval-augmented generation on edge- based computing-in-memory architectures. InProceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design, pages 1–9, 2024

    2024

  7. [7]

    Nvcim-pt: An nvcim-assisted prompt tuning framework for edge llms, 2024

    Ruiyang Qin, Pengyu Ren, Zheyu Yan, Liu Liu, Dancheng Liu, Amir Nassereldine, Jinjun Xiong, Kai Ni, Sharon Hu, and Yiyu Shi. Nvcim-pt: An nvcim-assisted prompt tuning framework for edge llms, 2024

    2024

  8. [8]

    Cq-cim: Hardware-aware embedding shaping for robust cim-based retrieval

    Xinzhao Li, Alptekin Vardar, Franz Müller, Navya Goli, Umamaheswara Tida, Kai Ni, X Sharon Hu, Thomas Kämpfe, and Ruiyang Qin. Cq-cim: Hardware-aware embedding shaping for robust cim-based retrieval. arXiv preprint arXiv:2602.20083, 2026

    arXiv 2026

  9. [9]

    Probabilistic embeddings for cross-modal retrieval

    Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio de Rezende, Yannis Kalantidis, and Diane Larlus. Probabilistic embeddings for cross-modal retrieval. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8415–8424, June 2021

    2021

  10. [10]

    Improved probabilistic image-text representations

    Sanghyuk Chun. Improved probabilistic image-text representations. arXiv preprint arXiv:2305.18171, 2023

    Pith/arXiv arXiv 2023

  11. [11]

    Sharon Hu, Ningyuan Cao, and Kai Ni

    Pengyu Ren, Xingtian Wang, Boyang Cheng, Jiahui Duan, Giuk Kim, Xuezhong Niu, Halid Mulaosmanovic, Stefan Duenkel, Sven Beyer, X. Sharon Hu, Ningyuan Cao, and Kai Ni. Probabilistic tree inference enabled by fdsoi ferroelectric fets, 2026

    2026

  12. [12]

    Cross-modal retrieval: a systematic review of methods and future directions.Proceedings of the IEEE, 112(11):1716–1754, 2025

    Tianshi Wang, Fengling Li, Lei Zhu, Jingjing Li, Zheng Zhang, and Heng Tao Shen. Cross-modal retrieval: a systematic review of methods and future directions.Proceedings of the IEEE, 112(11):1716–1754, 2025

    2025

  13. [13]

    Tiny-align: Bridging automatic speech recognition and large language model on edge

    Ruiyang Qin, Dancheng Liu, Gelei Xu, Amir Nassereldine, Zheyu Yan, Chenhui Xu, Yuting Hu, X Sharon Hu, Jinjun Xiong, and Yiyu Shi. Tiny-align: Bridging automatic speech recognition and large language model on edge. In2025 IEEE/ACM International Conference On Computer Aided Design (ICCAD), pages 1–9. IEEE, 2025

    2025

  14. [14]

    Open-ended multi-modal relational reasoning for video question an- swering

    Haozheng Luo, Ruiyang Qin, Chenwei Xu, Guo Ye, and Zening Luo. Open-ended multi-modal relational reasoning for video question an- swering. In2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 363–369. IEEE, 2023

    2023

  15. [15]

    Prob- abilistic memory for trustworthy edge intelligence.arXiv preprint arXiv:2607.02465, 2026

    Likai Pei, Jiahao Zheng, Xueji Zhao, Emilie Ye, Jianbo Liu, Hanqing Tao, Ming-Yen Lee, Ruiyang Qin, Yiyu Shi, Shimeng Yu, et al. Prob- abilistic memory for trustworthy edge intelligence.arXiv preprint arXiv:2607.02465, 2026

    Pith/arXiv arXiv 2026

  16. [16]

    Representation learning with contrastive predictive coding, 2019

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, 2019

    2019

  17. [17]

    Clotho: An audio captioning dataset, 2019

    Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen. Clotho: An audio captioning dataset, 2019

    2019

  18. [18]

    Msr-vtt: A large video description dataset for bridging video and language

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  19. [19]

    Vatex: A large-scale, high-quality multilingual dataset for video-and-language research, 2020

    Xin Wang, Jiawei Wu, Junkun Chen, Lei Li, Yuan-Fang Wang, and William Yang Wang. Vatex: A large-scale, high-quality multilingual dataset for video-and-language research, 2020

    2020

  20. [20]

    Lawrence Zitnick, and Piotr Dollár

    Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. Microsoft coco: Common objects in context, 2015

    2015

  21. [21]

    Plummer, Liwei Wang, Chris M

    Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collect- ing region-to-phrase correspondences for richer image-to-sentence models, 2016

    2016

  22. [22]

    AudioCaps: Generating captions for audios in the wild

    Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. AudioCaps: Generating captions for audios in the wild. In Jill Burstein, Christy Doran, and Thamar Solorio, editors,Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)...

    2019

  23. [23]

    Image- bind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Image- bind: One embedding space to bind them all. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15180–15190, 2023

    2023

  24. [24]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning (ICML), pages 8748–8763, 2021

    2021

  25. [25]

    Large-scale contrastive language- audio pretraining with feature fusion and keyword-to-caption aug- mentation

    Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg- Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language- audio pretraining with feature fusion and keyword-to-caption aug- mentation. InIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023

    2023

  26. [26]

    Plimpton, Sapan Agarwal, Richard Schiek, Isaac Richter, Christopher Bennett, Benjamin Feinberg, Tianyao Xiao, and Curtis Brinker

    StevenJ. Plimpton, Sapan Agarwal, Richard Schiek, Isaac Richter, Christopher Bennett, Benjamin Feinberg, Tianyao Xiao, and Curtis Brinker. Crosssim, 09 2016

    2016

  27. [27]

    Fully hardware-implemented memristor convolutional neural network.Nature, 577(7792):641–646, 2020

    Peng Yao, Huaqiang Wu, Bin Gao, Jianshi Tang, Qingtian Zhang, Wen- qiang Zhang, J Joshua Yang, and He Qian. Fully hardware-implemented memristor convolutional neural network.Nature, 577(7792):641–646, 2020

    2020

  28. [28]

    Architecture-circuit-technology co-optimization for resistive random access memory-based computation-in-memory chips

    Jianbo Liu et al. Architecture-circuit-technology co-optimization for resistive random access memory-based computation-in-memory chips. Science China Information Sciences, 66(10):200408, 2023

    2023

  29. [29]

    Switching pathway-dependent strain-effects on the ferroelectric properties and structural deformations in orthorhombic HfO2.Journal of Applied Physics, 131(15), 2022

    Wenbo Wei et al. Switching pathway-dependent strain-effects on the ferroelectric properties and structural deformations in orthorhombic HfO2.Journal of Applied Physics, 131(15), 2022

    2022

  30. [30]

    Dnn+neurosim v2.0: An end-to-end benchmarking framework for compute-in-memory accelerators for on-chip training.Trans

    Xiaochen Peng, Shanshi Huang, Hongwu Jiang, Anni Lu, and Shimeng Yu. Dnn+neurosim v2.0: An end-to-end benchmarking framework for compute-in-memory accelerators for on-chip training.Trans. Comp.- Aided Des. Integ. Cir. Sys., 40(11):2306–2319, November 2021

    2021

  31. [31]

    Post- training 4-bit quantization of convolution networks for rapid- deployment, 2019

    Ron Banner, Yury Nahshan, Elad Hoffer, and Daniel Soudry. Post- training 4-bit quantization of convolution networks for rapid- deployment, 2019

    2019

  32. [2019]

    Association for Computational Linguistics