Pith. sign in

REVIEW 5 major objections 6 minor 28 references

Approximate Fiber Product: A Preliminary Algebraic-Geometric Perspective on Multimodal Embedding Alignment

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that multimodal alignment can be modeled as the approximate fiber product of polynomial-ring embeddings over a shared space with tolerance ε, and that the embedding space splits orthogonally into shared and…

desk verdict The tolerance-based fiber product is a seed of a good idea, but the paper's central robustness claim is false and contradicted by its own Theorem 3; the promised rigor is not there as written. read the letter →

arxiv 2412.00373 v1 pith:BYBDUENU submitted 2024-11-30 cs.LG cs.AImath.AG

Dongfang Zhao This is my paper · ORCID
classification cs.LGcs.AImath.AG
keywords multimodalalignmentapproximatefiberproductalgebraicgeometryembeddingspacedecompositionorthogonalsubspacespolynomialringrepresentationdimensionalityallocationrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a mathematical foundation for multimodal alignment built from algebraic geometry. It represents image and text data as polynomials over finite rings, $\mathbb{Z}_{256}[x]$ and $\mathbb{Z}_{|V|}[x]$, and defines aligned image-text pairs as exactly those whose embeddings $f(P)$ and $g(Q)$ lie within a tolerance $\epsilon$ in a shared space $Z$; this set is the approximate fiber product. It further hypothesizes that $Z$ decomposes orthogonally as $Z = Z_s \oplus Z_I \oplus Z_T$, with $Z_s$ carrying shared semantics and $Z_I$, $Z_T$ carrying modality-specific features. If the framework is right, alignment precision, noise robustness, and the allocation of embedding dimensions all become expressible in one formal language, giving contrastive alignment a principled alternative to purely empirical tuning.

What carries the argument

The central construction is the approximate fiber product $\mathbb{Z}_{256}[x] \times_{Z,\epsilon} \mathbb{Z}_{|V|}[x] = \{(P,Q) : \|f(P)-g(Q)\| \le \epsilon\}$, which relaxes the classical equality condition $f(P)=g(Q)$ to an $\epsilon$-ball condition and thereby converts alignment into a parameterized geometric object whose size, stability, and asymptotics can be analyzed. The second load-bearing mechanism is the orthogonal decomposition $Z = Z_s \oplus Z_I \oplus Z_T$ together with the projection operators $\Pi_s, \Pi_I, \Pi_T$; it is what turns the abstract fiber product into statements about shared semantics, modality-specific features, and how to split embedding dimensions.

What would settle it

Take a trained image-text model and, for a large sample of embeddings, compute the covariance between $\Pi_s(z)$, $\Pi_I(z)$, and $\Pi_T(z)$; nonzero off-diagonal covariance blocks would refute the orthogonal decomposition. Alternatively, measure the number of matched pairs within distance $\epsilon$ and check whether it follows the predicted $\epsilon^d \exp(-\| \mu_f - \mu_g\|^2/(2(\sigma_f^2+\sigma_g^2)))$ growth; a clear mismatch would refute the asymptotic claim.

Watch

Extended reading notes

Core claim

The central claim is that the set of aligned image-text pairs is the approximate fiber product $\mathbb{Z}_{256}[x] \times_{Z,\epsilon} \mathbb{Z}_{|V|}[x] = \{(P,Q) : \|f(P)-g(Q)\| \le \epsilon\}$, a tolerance-relaxed pullback of the two embedding maps $f,g$ into the shared real polynomial space $Z \subset \mathbb{R}[x]$. The paper derives that the size of this set is monotone in $\epsilon$, converges to $|X|\cdot|Y|$ as $\epsilon\to\infty$, scales asymptotically as $\epsilon^d \exp(-\| \mu_f - \mu_g\|^2/(2(\sigma_f^2+\sigma_g^2)))$ for Gaussian embeddings, and changes only by a tolerance shift of $2\eta$ under bounded noise of size $\eta$. It then states a decomposition hypothesis: $Z = Z_s \oplus Z_I \oplus Z_T$ with orthogonal projections satisfying $\Pi_s + \Pi_I + \Pi_T = \mathrm{Id}$, so that alignment error, alignment volume, and dimensionality allocation can all be written in terms of the three subspaces.

Load-bearing premise

All of the decomposition results rest on the assumption that real image and text embeddings split into pairwise orthogonal subspaces $Z_s$, $Z_I$, $Z_T$ whose projections sum to the identity; if learned embeddings are entangled, the alignment error, alignment volume, and dimensionality-allocation formulas do not follow.

Editorial extensions

If this is right

  • If the approximate fiber product is the right model, $\epsilon$ becomes a design parameter governing the precision-versus-noise tradeoff: $\epsilon\to 0$ recovers exact alignment and $\epsilon\to\infty$ admits every pair.
  • The asymptotic formula $|\cdot| \propto \epsilon^d \exp(-\| \mu_f - \mu_g\|^2/(2(\sigma_f^2+\sigma_g^2)))$ predicts that in high-dimensional spaces alignment becomes sharply sensitive to $\epsilon$ and to the mismatch between the two modalities' embedding distributions.
  • If the orthogonal decomposition holds, each embedding norm splits as $\|z\|^2 = \|\Pi_s(z)\|^2 + \|\Pi_I(z)\|^2 + \|\Pi_T(z)\|^2$, allowing shared and modality-specific information to be measured and regularized separately.
  • The dimensionality allocation rule $d_s \propto (\sigma_f^2+\sigma_g^2)/(\sigma_f^2 \sigma_g^2)$, $d_I \propto \sigma_f^2/\sigma_g^2$, $d_T \propto \sigma_g^2/\sigma_f^2$ gives an explicit recipe for choosing the sizes of the three subspaces in a multimodal model.
  • Bounded noise of size $\eta$ shifts the alignment region's tolerance by at most $2\eta$, so the framework yields a quantitative robustness guarantee for aligned pairs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The quantitative results depend only on the metric and on distributional assumptions, not on the polynomial-ring encoding; a purely metric-space formulation would likely preserve the main theorems if the ring representation were dropped.
  • The orthogonal-decomposition hypothesis can be tested on existing contrastive models by checking whether the covariance between the three projected components vanishes; non-zero off-diagonal blocks would mean the decomposition is only an approximation.
  • The size of the approximate fiber product at a given $\epsilon$ suggests a natural definition of alignment recall: the fraction of true matched pairs whose embeddings fall within tolerance, linking $\epsilon$ directly to retrieval evaluation.
  • One could estimate $\epsilon$ from data by fitting the predicted growth curve to observed matched-pair counts, rather than setting it heuristically as the paper leaves it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes an algebraic-geometric framework for multimodal embedding alignment. Images and text are represented as polynomials over Z_256[x] and Z_|V|[x], and alignment is modeled by the approximate fiber product I ×_{Z,ε} T = {(i,t) : ||f(i) - g(t)|| ≤ ε}. The paper also hypothesizes an orthogonal decomposition Z = Zs ⊕ ZI ⊕ ZT into shared and modality-specific subspaces, and uses this to derive properties such as monotonicity of the fiber product size, noise tolerance, dimensionality allocation, and geometric interpretations via manifolds, fiber bundles, and sheaves. The central claim is that this provides a rigorous mathematical foundation for analyzing multimodal alignment.

Significance. If correct, the approximate fiber product would offer a clean formal language for alignment tolerance and robustness, and the subspace decomposition would provide a principled way to separate shared from modality-specific information. The paper does introduce a clear definition and proves some elementary results correctly, notably Theorem 3 (the ε+2η inclusion) and the monotonicity statement in Theorem 2. However, the paper's load-bearing robustness assertion in Section 2.3 is false, and several propositions in Section 3 are either false, asserted without proof, or conditional on an unverified hypothesis. As it stands, the manuscript does not establish the claimed rigorous foundation, and the errors are not confined to presentation issues.

major comments (5)
  1. [Section 2.3, 'Robustness Under Perturbations'] The claimed inclusion fδ(X) ×_{Z,ε} gδ(Y) ⊆ f(X) ×_{Z,ε} g(Y) if and only if η ≤ ε/2 is false. From ||fδ(x) - gδ(y)|| ≤ ε, the triangle inequality gives only ||f(x) - g(y)|| ≤ ε + 2η, not ≤ ε; the 'if' direction would require η ≤ 0. This is contradicted by the paper's own Theorem 3 in Section 2.4, which correctly proves the weaker inclusion with ε+2η. A concrete counterexample (ε=1, η=0.5, f(x)=1.1, g(y)=0, δf(x)=-0.5, δg(y)=0.5) yields fδ(x)=0.6 and gδ(y)=0.5, so the pair lies in the perturbed ε-product but not in the original ε-product. Because this claim anchors the paper's noise-tolerance analysis, the central robustness property is unsupported.
  2. [Section 3.2, Proposition 5] Proposition 5 asserts that if dim(Zs) is much smaller than dim(Z), then for any ε > 0 the quantity sup_{(i,t)∈I×T} ||fs(i) - gs(t)||² is at least ε. This is false: if fs(i) = gs(t) = 0 for all i,t, the supremum is 0, which is less than every ε > 0 regardless of dim(Zs). The proof's informal statement that a small subspace 'cannot accommodate sufficient shared features' does not establish the claimed lower bound. The proposition would need additional assumptions, such as a minimum variance or non-degeneracy of the shared projections.
  3. [Section 3.4, Proposition 8] The optimal dimensionality allocation in Proposition 8 is not derived from a well-posed optimization problem: the objective F(ds,dI,dT) is never defined, and the proof only restates variance-ratio heuristics without deriving the proportionalities from first principles. The argument jumps from 'alignment capacity inversely proportional to total variance' to the specific formula ds ∝ (σf²+σg²)/(σf²σg²), and no normalization or constraint is used to ensure ds+dI+dT = d. Consequently, the claimed optimality has no formal basis.
  4. [Section 3.5, Proposition 10] Proposition 10 defines F(zs) = {(zI,zT) ∈ F | fI(i) + gT(t) = zs} and claims non-emptiness if and only if ||fI(i)||² + ||gT(t)||² = ||zs||². This is dimensionally inconsistent: fI(i) + gT(t) lies in ZI⊕ZT, which is orthogonal to Zs, so the equality fI(i) + gT(t) = zs forces both sides to be zero (assuming a genuine direct sum). Even if that inconsistency is set aside, equality of squared norms is not sufficient for vector equality, so the 'if and only if' statement fails. The fiber bundle consistency claim is therefore not established.
  5. [Sections 3.1 and 5] The orthogonal decomposition Z = Zs ⊕ ZI ⊕ ZT is introduced as a hypothesis in Section 3.1, yet the entire Section 3—including the embedding decomposition, the alignment error E, the optimization objective L, and Proposition 8—treats it as an established fact, and Section 5 presents it as a proposed structured decomposition. No evidence (empirical or theoretical) is provided that real image/text embedding spaces admit such pairwise orthogonal subspaces with Πs+ΠI+ΠT = Id. Since the decomposition is load-bearing for all of Section 3, the paper should either prove it under stated assumptions on f and g, or explicitly present the results as conditional on a testable hypothesis with a concrete validation strategy.
minor comments (6)
  1. [Section 2.3] The integral formula for |X ×_{Z,ε} Y| is stated without the measure-theoretic assumptions needed for the densities µf and µg; please specify the setting or provide a derivation.
  2. [Section 2.3] The asymptotic scaling |X ×_{Z,ε} Y| ∝ ε^d · exp(-||µf - µg||² / (2(σf²+σg²))) is asserted without derivation or specification of the proportionality constant; please clarify the steps.
  3. [Section 3.1] The phrase 'pairwise disjoint' for linear subspaces is imprecise; linear subspaces always share the zero vector, so the intended condition is 'intersect only at {0}'.
  4. [Section 2.2] The commutative diagram appears malformed in the text; please ensure the arrows and labels render correctly.
  5. [Section 3.3] In the definition of Lorth, the notation zs·zI is undefined; please clarify whether these are sample vectors, projections, and how the sum over the batch is taken.
  6. [References] Several references are incomplete, including 'E James Robinson et al.' and 'Collin Miles et al.'; please provide full author lists and publication details.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: the optimal dimensionality allocation of Proposition 8 is an assumed ansatz renamed as a theorem; the core fiber-product definition is otherwise self-contained, and the false robustness claim is a correctness error rather than a circular step.

  1. self definitional [Section 3.4, Proposition 8 (Optimal Dimensionality Allocation) and its proof]
    "To determine an optimal dimensionality allocation, we consider the following optimization problem: max_{ds,dI,dT} F(ds,dI,dT), where F is a task-specific performance metric... Proposition 8 ... the optimal allocation satisfies: ds ∝ (σ²f+σ²g)/(σ²f·σ²g), dI ∝ σ²f/σ²g, dT ∝ σ²g/σ²f. Proof ... Combining these considerations, the dimensionality of Zs should grow with the alignment capacity: ds ∝ (σ²f+σ²g)/(σ²f·σ²g)."

    F is never specified, so no derivation links the variances to an optimum. The proof asserts 'Alignment Capacity ∝ 1/(σ²f+σ²g)' and 'therefore, to maximize alignment, the dimensionality ds allocated to Zs must reflect the combined variability,' then 'Combining these considerations' outputs the very proportionality the proposition states. No optimization is performed; the only constraint used, ds+dI+dT=d, is insufficient to determine any of the three formulas. The claimed optimal allocation is thus the authors' variance-ratio ansatz written into the proof and renamed a theorem.

full rationale

There are no load-bearing self-citations: all references are to external prior work, and the approximate fiber product itself is defined self-containedly. Theorems 1–3 are direct consequences of the definitions or the triangle inequality, and the Z=Zs⊕ZI⊕ZT decomposition is explicitly labeled a hypothesis, so those parts are not circular in the prohibited sense. The one genuine circular step is Proposition 8: with the objective F undefined, the 'optimal' allocation is the same proportionality inserted as the proof's 'considerations.' Separately, Section 2.3's inclusion claim 'if and only if η≤ε/2' is mathematically false and contradicts Theorem 3, but a false step is a correctness defect rather than an input-equivalence circularity, so it does not further raise the circularity score. The score 6 reflects that one central-sounding predictive claim reduces by construction, while the core fiber-product construction retains independent mathematical content.

Assumptions & free parameters 3 free parameters · 6 assumptions · 4 invented entities

The framework rests on algebraic encoding of data as polynomials, an assumed orthogonal decomposition of embeddings, Gaussian distribution assumptions for asymptotics, and an unconstructed sheaf and moduli structure. None of these are derived or validated, so the ledger is heavy relative to the results.

free parameters (3)
  • epsilon = unspecified
    Tolerance in the approximate fiber product, chosen by hand to balance precision and noise; all alignment results depend on it.
  • lambda = unspecified
    Weight for the orthogonality loss in the proposed optimization objective in Section 3.3.
  • gamma = unspecified
    Weight for the specificity loss in the proposed optimization objective in Section 3.3.
assumptions (6)
  • domain assumption Image and text data are faithfully represented by polynomials over Z256[x] and Z|V|[x] with pixel and token coefficients.
    Section 2.1 introduces this encoding, but it discards spatial and syntactic structure beyond coefficient order; no empirical evidence is given that semantic information is preserved.
  • domain assumption Real embedding spaces admit the orthogonal direct-sum decomposition Z=Zs⊕ZI⊕ZT with projection operators Πs, ΠI, ΠT.
    Section 3.1 labels this a hypothesis; all decomposition propositions and the optimization objective depend on it.
  • domain assumption Embedding distributions are Gaussian when deriving the asymptotic size of the approximate fiber product.
    Section 2.3 assumes Gaussian µf and µg; no justification is given for real multimodal embeddings.
  • standard math Embedding functions f and g are continuous and the embedding space Z is compact for Theorem 1.
    Theorem 1 states these as hypotheses; compactness follows from standard topology if they hold.
  • ad hoc to paper A presheaf F on the embedding space exists and satisfies the sheaf condition.
    Section 3.6 introduces the sheaf without constructing its sections or restriction maps; it is illustrative.
  • domain assumption The shared subspace Zs is a Riemannian manifold with curvature κ and there is a moduli space M of semantic categories.
    Section 3.5 uses these structures for Propositions 9 and 10 and the fiber variety interpretation; they are not constructed from data.
invented entities (4)
  • Approximate fiber product X ×_{Z,ε} Y
    purpose: Models pairs of image and text embeddings aligned within tolerance ε.
    Introduced in Section 2.2; no falsifiable prediction or experimental validation is attached to it.
  • Orthogonal subspace decomposition Z = Zs ⊕ ZI ⊕ ZT
    purpose: Separates shared semantics from modality-specific image and text features.
    Hypothesized in Section 3.1; no evidence that real trained embeddings admit such a decomposition.
  • Presheaf F on the embedding space
    purpose: Formalizes local and global consistency of embeddings across open sets.
    Introduced in Section 3.6 without a concrete construction.
  • Moduli space M of semantic categories
    purpose: Parameterizes semantic categories whose fibers form the shared subspace.
    Introduced in Section 3.5; no construction or connection to data is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Approximate Fiber Product: A Preliminary Algebraic-Geometric Perspective on Multimodal Embedding Alignment." pith.science (2026). https://pith.science/paper/BYBDUENU

@misc{pith2026241200373,
  author       = {Pith},
  title        = {Pith review of: Approximate Fiber Product: A Preliminary Algebraic-Geometric Perspective on Multimodal Embedding Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BYBDUENU}},
  note         = {Machine review of arXiv:2412.00373}
}
abstract

Multimodal tasks, such as image-text retrieval and generation, require embedding data from diverse modalities into a shared representation space. Aligning embeddings from heterogeneous sources while preserving shared and modality-specific information is a fundamental challenge. This paper provides an initial attempt to integrate algebraic geometry into multimodal representation learning, offering a foundational perspective for further exploration. We model image and text data as polynomials over discrete rings, \( \mathbb{Z}_{256}[x] \) and \( \mathbb{Z}_{|V|}[x] \), respectively, enabling the use of algebraic tools like fiber products to analyze alignment properties. To accommodate real-world variability, we extend the classical fiber product to an approximate fiber product with a tolerance parameter \( \epsilon \), balancing precision and noise tolerance. We study its dependence on \( \epsilon \), revealing asymptotic behavior, robustness to perturbations, and sensitivity to embedding dimensionality. Additionally, we propose a decomposition of the shared embedding space into orthogonal subspaces, \( Z = Z_s \oplus Z_I \oplus Z_T \), where \( Z_s \) captures shared semantics, and \( Z_I \), \( Z_T \) encode modality-specific features. This decomposition is geometrically interpreted via manifolds and fiber bundles, offering insights into embedding structure and optimization. This framework establishes a principled foundation for analyzing multimodal alignment, uncovering connections between robustness, dimensionality allocation, and algebraic structure. It lays the groundwork for further research on embedding spaces in multimodal learning using algebraic geometry.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 21 canonical work pages

  1. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Stefan Leutenegger, Katie Millican, Malcolm Reynolds, Aäron van den Oord, et al. Flamingo: a visual language model for few-shot learning. arXiv preprint arXiv:2204.14198, 2022

    arXiv 2022

  2. [2]

    Bruno buchberger's phd thesis 1965: An algorithm for finding a basis for the residue class ring of a zero-dimensional polynomial ideal

    Bruno Buchberger. Bruno buchberger's phd thesis 1965: An algorithm for finding a basis for the residue class ring of a zero-dimensional polynomial ideal. Journal of Symbolic Computation, 41 0 (3-4): 0 475--511, 2006

    work page 1965

  3. [3]

    Topology and data

    Gunnar Carlsson. Topology and data. Bulletin of the American Mathematical Society, 46 0 (2): 0 255--308, 2009

    2009

  4. [4]

    Uniter: Universal image-text representation learning

    Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. arXiv preprint arXiv:1909.11740, 2020

    arXiv 1909

  5. [5]

    BpForms and BcForms: Tools for concretely describing non-canonical polymers and complexes to facilitate comprehensive biochemical networks

    Justin M Curry. Sheaves, cosheaves and applications. arXiv preprint arXiv:1903.10042, 2019

    work page Pith review arXiv 1903

  6. [6]

    Canonical correlation analysis: An overview with application to learning methods

    David R Hardoon, Sandor Szedmak, and John Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods. Neural Computation, 16 0 (12): 0 2639--2664, 2004

    work page 2004

  7. [7]

    Moduli of Curves

    Joe Harris and Ian Morrison. Moduli of Curves. Springer, 1995

    work page 1995

  8. [8]

    Algebraic Geometry

    Robin Hartshorne. Algebraic Geometry. Springer, 1977

    work page 1977

Show all 28 references
  1. [9]

    Disentangling factors of variation in deep representations using adversarial training

    Zhengli Hu, Yang Yang, Ruslan Salakhutdinov, and Phillip MS Lim. Disentangling factors of variation in deep representations using adversarial training. In Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  2. [10]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. Proceedings of the International Conference on Machine Learning, 2021

    2021

  3. [11]

    Principal Component Analysis

    Ian Jolliffe. Principal Component Analysis. Springer, 2002

    2002

  4. [12]

    Supervised multimodal bitransformers for classifying images and text

    Douwe Kiela, Y-Lan Boureau, Maximilian Nickel, Bartlomiej Jokiel, and Davide Testuggine. Supervised multimodal bitransformers for classifying images and text. arXiv preprint arXiv:1909.02950, 2019

    1909 arXiv

  5. [13]

    Tensors: Geometry and Applications

    Joseph M Landsberg. Tensors: Geometry and Applications. American Mathematical Society, 2012

    2012

  6. [14]

    Learning the parts of objects by non-negative matrix factorization

    Daniel D Lee and H Sebastian Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401 0 (6755): 0 788--791, 1999

    1999

  7. [15]

    Visualbert: A simple and performant baseline for vision and language

    Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019

    1908 arXiv

  8. [16]

    Oscar: Object-semantics aligned pre-training for vision-language tasks

    Xiujun Li, Xi Yin, Chunyuan Li, Xiaowei Hu, Pengchuan Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. Proceedings of the European Conference on Computer Vision, 2020

    2020

  9. [17]

    Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in Neural Information Processing Systems, 2019

    2019

  10. [18]

    Modeling modality-specific and shared information for multimodal data representation learning

    Chao Ma, Wei Xu, and Thomas Huang. Modeling modality-specific and shared information for multimodal data representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

    2018

  11. [19]

    Topology and generalization in neural networks

    Collin Miles et al. Topology and generalization in neural networks. Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  12. [20]

    Polynomial Optimization and Applications

    Jiawang Nie. Polynomial Optimization and Applications. Society for Industrial and Applied Mathematics (SIAM), 2012

    2012

  13. [21]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Cliff Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Mach...

    2021

  14. [22]

    Deep learning theory via algebraic geometry and statistical mechanics

    E James Robinson et al. Deep learning theory via algebraic geometry and statistical mechanics. arXiv preprint arXiv:1703.09263, 2017

    2017 arXiv

  15. [23]

    Variational bi-lstm for multimodal conditional text generation

    Weizhi Shi, Furu Wei, Ming Zhou, and Wenjie Li. Variational bi-lstm for multimodal conditional text generation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2019

    2019

  16. [24]

    Lxmert: Learning cross-modality encoder representations from transformers

    Hao Tan and Mohit Bansal. Lxmert: Learning cross-modality encoder representations from transformers. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 2019

    2019

  17. [25]

    Learning and Generalization: With Applications to Neural Networks

    Mathukumalli Vidyasagar. Learning and Generalization: With Applications to Neural Networks. Springer, 2002

    2002

  18. [26]

    On deep multimodal representation learning

    William Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. On deep multimodal representation learning. In International Conference on Machine Learning (ICML), 2016

    2016

  19. [27]

    Cross-modal contrastive learning for text-to-image generation

    Bowen Zhang, Ting Li, Ting Zhang, Yulun Zhang, Wanli Ouyang, and Bolei Zhang. Cross-modal contrastive learning for text-to-image generation. arXiv preprint arXiv:2101.04702, 2021

    2021 arXiv

  20. [28]

    Split neural networks for multimodal fusion

    Yang Zhang, Benjamin Recht, Max Simchowitz, Moritz Hardt, and Benjamin Recht. Split neural networks for multimodal fusion. In Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.