pith. sign in

arxiv: 2604.08324 · v2 · submitted 2026-04-09 · 💻 cs.NE · cs.AI

Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimization

Benjamin L\'eger , Kazem Meidani , Christian Gagn\'e This is my paper

Pith reviewed 2026-05-10 17:24 UTC · model grok-4.3

classification 💻 cs.NE cs.AI
keywords symbolic regressionlatent space optimizationmulti-modal learninggenetic programmingcross-modal alignmentSNIPcontrastive pre-training
0
0 comments X

The pith

SNIP's cross-modal alignment does not improve during optimization even as fitness rises, and stays too coarse for effective symbolic search.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether SNIP, a contrastive model that aligns encoders for symbolic expressions and their numeric evaluations in one latent space, can turn continuous optimization into better genetic programming searches for mathematical expressions. Experiments track alignment quality while fitness improves and find no corresponding gain in how well the two modalities match. The learned alignment turns out too coarse to map small improvements in the numeric space back to precise, principled changes in the symbolic structures. A sympathetic reader cares because this shows the multi-modal route to latent space optimization for symbolic regression has not yet delivered the expected bridging of genotype and phenotype.

Core claim

The paper claims that cross-modal alignment does not improve during optimization, even as fitness increases, and that the alignment learned by SNIP is too coarse to efficiently conduct principled search in the symbolic space. While multi-modal latent space optimization holds potential for symbolic regression, effective alignment-guided optimization remains unrealized in practice, and fine-grained alignment is identified as a critical direction for future work.

What carries the argument

SNIP, a contrastive pre-training model that aligns symbolic and numeric encoders in a shared latent space to learn the phenotype-genotype mapping for latent space optimization.

If this is right

  • Multi-modal latent space optimization for symbolic regression requires advances in fine-grained alignment to achieve effective bi-modal search.
  • Coarse alignment limits the ability to use numeric-space improvements to guide precise symbolic changes.
  • The potential of combining multi-modal learning with genetic programming for symbolic tasks depends on realizing better modality alignment.
  • Future methods must address the gap between current contrastive pre-training and the precision needed for principled symbolic optimization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar coarse-alignment problems could appear in other contrastive models applied to structured combinatorial domains such as programs or circuits.
  • Alternative training objectives or auxiliary losses that enforce finer semantic matching might overcome the current limitations.
  • Hybrid approaches that retain some direct symbolic operations alongside latent optimization could mitigate reliance on perfect alignment.

Load-bearing premise

The chosen metrics for cross-modal alignment and the optimization process accurately capture whether the alignment enables effective bi-modal search without confounding factors from the experimental setup or SNIP hyperparameters.

What would settle it

A measurement showing that cross-modal alignment scores rise in step with fitness improvements during optimization, or that latent-space distances permit finer symbolic manipulations than currently observed, would challenge the central claim.

Figures

Figures reproduced from arXiv: 2604.08324 by Benjamin L\'eger, Christian Gagn\'e, Kazem Meidani.

Figure 1
Figure 1. Figure 1: Overview of Latent Space Optimization and multi-modal alignment. a) Traditional LSO-GP framework : a continuous [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Evolution of (a) 𝑅 2 fitness (top) and (b) cross-modal alignment (bottom) during LSO across all Feynman and Stro￾gatz equations, averaged over all equations. While fitness improves consistently, alignment remains flat or decreases, indicating that optimization does not exploit cross-modal alignment to produce better symbolic solutions. Iteration 0 corresponds to the model’s one-shot prediction, iteration 1… view at source ↗
Figure 3
Figure 3. Figure 3: Distribution of ranks for the correct expression [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Sensitivity analysis excluding constant change perturbations on Feynman. (a) Accuracy comparison showing that even [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
read the original abstract

Symbolic regression (SR) aims to discover mathematical expressions from data, a task traditionally tackled using Genetic Programming (GP) through combinatorial search over symbolic structures. Latent Space Optimization (LSO) methods use neural encoders to map symbolic expressions into continuous spaces, transforming the combinatorial search into continuous optimization. SNIP (Meidani et al., 2024), a contrastive pre-training model inspired by CLIP, advances LSO by introducing a multi-modal approach: aligning symbolic and numeric encoders in a shared latent space to learn the phenotype-genotype mapping, enabling optimization in the numeric space to implicitly guide symbolic search. However, this relies on fine-grained cross-modal alignment, whereas literature on similar models like CLIP reveals that such an alignment is typically coarse-grained. In this paper, we investigate whether SNIP delivers on its promise of effective bi-modal optimization for SR. Our experiments show that: (1) cross-modal alignment does not improve during optimization, even as fitness increases, and (2) the alignment learned by SNIP is too coarse to efficiently conduct principled search in the symbolic space. These findings reveal that while multi-modal LSO holds significant potential for SR, effective alignment-guided optimization remains unrealized in practice, highlighting fine-grained alignment as a critical direction for future work.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 3 minor

Summary. The manuscript investigates whether the multi-modal latent space optimization approach SNIP, which aligns symbolic and numeric encoders via contrastive learning, enables effective bi-modal search for symbolic regression. The central claims, supported by experiments, are that cross-modal alignment fails to improve during optimization despite rising fitness values, and that the learned alignment remains too coarse-grained to facilitate efficient, principled exploration in the symbolic expression space.

Significance. Should the empirical findings prove robust, this paper offers a significant contribution to the intersection of genetic programming and multi-modal machine learning by delivering a critical analysis that tempers enthusiasm for direct application of CLIP-like models to LSO in symbolic regression. It provides concrete evidence of limitations in current alignment quality and identifies fine-grained alignment as an open challenge, which could steer future work toward more sophisticated contrastive objectives or hybrid search strategies. The analysis is strengthened by its focus on falsifiable predictions about alignment dynamics during optimization.

major comments (1)
  1. [Section 4] Section 4 (Experiments): While the full manuscript provides details on the alignment metric, optimization loop, SNIP hyperparameters, and benchmark suite (addressing the abstract's brevity), the lack of reported variance across multiple random seeds or statistical tests for the no-improvement claim in alignment vs. fitness makes it difficult to rule out sampling artifacts as a confounder for finding (1).
minor comments (3)
  1. [Abstract] Abstract: The two main experimental findings could be enumerated more explicitly to improve scannability for readers.
  2. [Section 2] Section 2: A brief expansion on how the chosen latent-space similarity metric relates to symbolic equivalence measures (e.g., tree edit distance) would clarify why it is expected to capture 'coarseness' relevant to search efficiency.
  3. [Figures] Figure captions: Including the exact SNIP training hyperparameters and dataset sizes used for each plot would aid reproducibility without requiring cross-reference to the text.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive review and for recommending minor revision. We appreciate the recognition of our paper's contribution in highlighting limitations of current multi-modal alignment approaches for latent space optimization in symbolic regression. We address the major comment below.

read point-by-point responses

  1. Referee: [Section 4] Section 4 (Experiments): While the full manuscript provides details on the alignment metric, optimization loop, SNIP hyperparameters, and benchmark suite (addressing the abstract's brevity), the lack of reported variance across multiple random seeds or statistical tests for the no-improvement claim in alignment vs. fitness makes it difficult to rule out sampling artifacts as a confounder for finding (1).

    Authors: We agree that reporting variance across multiple random seeds and including statistical tests would strengthen the evidence for our claim that cross-modal alignment does not improve with increasing fitness. In the revised manuscript, we will rerun the key experiments with at least five independent random seeds and report means with standard deviations for the alignment metrics plotted against fitness. We will also add Pearson or Spearman correlation coefficients (with p-values) between alignment quality and fitness to quantify the lack of positive relationship and rule out sampling artifacts as a potential confounder. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical claims are self-contained

full rationale

The paper reports new experimental observations on cross-modal alignment in SNIP for symbolic regression, showing that alignment does not improve with fitness and remains too coarse. These findings rest on direct measurements from the optimization loop, latent-space metrics, and benchmark runs rather than any derivation, fitted parameter renamed as prediction, or self-referential equation. The citation to prior SNIP work (Meidani et al., 2024) merely describes the baseline method under test; the central claims are independent empirical results that can be inspected against the stated metrics, hyperparameters, and data. No load-bearing step reduces to a self-definition or self-citation chain.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper's conclusions depend on the experimental measurements of alignment being valid indicators of optimization effectiveness, which relies on domain assumptions from contrastive learning and GP.

axioms (1)
  • domain assumption The contrastive pre-training in SNIP learns a meaningful phenotype-genotype mapping.
    This is assumed from the prior SNIP work and tested in this paper.

pith-pipeline@v0.9.0 · 5532 in / 1207 out tokens · 49685 ms · 2026-05-10T17:24:26.074534+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages · 1 internal anchor

  1. [1]

    Philipp Anthes, Dominik Sobania, and Franz Rothlauf. 2025. Transformer seman- tic genetic programming for symbolic regression. InProceedings of the Genetic and Evolutionary Computation Conference. 952–960

    work page 2025

  2. [2]

    Tommaso Bendinelli, Luca Biggio, and Pierre-Alexandre Kamienny. 2023. Control- lable neural symbolic regression. InInternational Conference on Machine Learning. PMLR, 2063–2077

    work page 2023

  3. [3]

    Amanda Bertschinger, James Bagrow, and Joshua Bongard. 2024. Evolving Form and Function: Dual-Objective Optimization in Neural Symbolic Regression Net- works. InProceedings of the Genetic and Evolutionary Computation Conference. 277–285

    work page 2024

  4. [4]

    Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambat- tista Parascandolo. 2021. Neural symbolic regression that scales. InInternational Conference on Machine Learning. PMLR, 936–945

    work page 2021

  5. [5]

    Victor Caetano, Matheus Cândido Teixeira, and Gisele Lobo Pappa. 2023. Sym- bolic Regression Trees as Embedded Representations. InProceedings of the Genetic and Evolutionary Computation Conference

    work page 2023

  6. [6]

    Ziliang Chen, Tianang Xiao, Jusheng Zhang, Yongsen Zheng, and Xipeng Chen

    work page

  7. [7]

    Understanding Hardness of Vision-Language Compositionality from A Token-level Causal Lens.arXiv preprint(2025)

    work page 2025

  8. [8]

    Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. 2018. Syntax- Directed Variational Autoencoder for Structured Data. InInternational Conference on Learning Representations

    work page 2018

  9. [9]

    2013.Practical methods of optimization

    Roger Fletcher. 2013.Practical methods of optimization. John Wiley & Sons

    work page 2013

  10. [10]

    Xiaoxu Han, Chengzhen Ning, Jinghui Zhong, Fubiao Yang, Yu Wang, and Xin Mu. 2025. Discovering Mathematical Equations with Diffusion Language Model. arXiv preprint arXiv:2509.13136(2025)

    work page arXiv 2025

  11. [11]

    Xiaoxu Han, Jinghui Zhong, Zhitong Ma, Xin Mu, and Nikola Gligorovski. 2025. Transformer-Assisted Genetic Programming for Symbolic Regression.IEEE Computational Intelligence Magazine(2025)

    work page 2025

  12. [12]

    Pierre-Alexandre Kamienny, Stéphane d’Ascoli, Guillaume Lample, and François Charton. 2022. End-to-end symbolic regression with transformers. InAdvances in Neural Information Processing Systems, Vol. 35. 10269–10281. Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimization Preprint, 2026,

    work page 2022

  13. [13]

    Matt J Kusner, Brooks Paige, and José Miguel Hernández-Lobato. 2017. Grammar variational autoencoder. InInternational conference on machine learning. PMLR, 1945–1954

    work page 2017

  14. [14]

    William La Cava, Bogdan Burlacu, Marco Virgolin, Michael Kommenda, Patryk Orzechowski, Fabrício Olivetti de França, Ying Jin, and Jason H Moore. 2021. Contemporary symbolic regression methods and their relative performance. Advances in neural information processing systems2021, DB1 (2021), 1

    work page 2021

  15. [15]

    Martha Lewis, Nihal V Nayak, Peilin Yu, Qinan Yu, Jack Merullo, Stephen H Bach, and Ellie Pavlick. 2022. Does CLIP Bind Concepts? Probing Compositionality in Large Image Models.arXiv preprint(2022)

    work page 2022

  16. [16]

    Wenqiang Li, Weijun Li, Linjun Sun, Min Wu, Lina Yu, Jingyi Liu, Yanjie Li, and Songsong Tian. 2022. Transformer-based model for symbolic regression via joint supervised learning. InThe Eleventh International Conference on Learning Representations

    work page 2022

  17. [17]

    Yanjie Li, Jingyi Liu, Min Wu, Lina Yu, Weijun Li, Xin Ning, Wenqiang Li, Meilan Hao, Yusong Deng, and Shu Wei. 2025. MMSR: symbolic regression is a multi- modal information fusion task.Information Fusion114 (2025), 102681

    work page 2025

  18. [18]

    Paweł Liskowski, Iwo Błądek, and Krzysztof Krawiec. 2018. Neuro-guided ge- netic programming: prioritizing evolutionary search with neural networks. In Proceedings of the Genetic and Evolutionary Computation Conference. 1143–1150

    work page 2018

  19. [19]

    Paweł Liskowski, Krzysztof Krawiec, Nihat Engin Toklu, and Jerry Swan. 2020. Program Synthesis as Latent Continuous Optimization: Evolutionary Search in Neural Embeddings. InProceedings of the Genetic and Evolutionary Computation Conference. 400–408

    work page 2020

  20. [20]

    Kazem Meidani, Parshin Shojaee, Chandan K Reddy, and Amir Barati Farimani

    work page

  21. [21]

    InInternational Conference on Learning Representations

    SNIP: Bridging mathematical symbolic and numeric realms with unified pre-training. InInternational Conference on Learning Representations

    work page

  22. [22]

    Sebastian Mežnar, Sašo Džeroski, and Ljupčo Todorovski. 2023. Efficient generator of mathematical expressions for symbolic regression.Machine Learning112, 11 (2023), 4563–4596

    work page 2023

  23. [23]

    Seyedali Mirjalili, Seyed Mohammad Mirjalili, and Andrew Lewis. 2014. Grey wolf optimizer.Advances in engineering software69 (2014), 46–61

    work page 2014

  24. [24]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748(2018)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  25. [25]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational conference on machine learning. PmLR, 8748–8763

    work page 2021

  26. [26]

    Shun Sato and Issei Sato. 2025. Can Test-time Computation Mitigate Memo- rization Bias in Neural Symbolic Regression?arXiv preprint arXiv:2505.22081 (2025)

    work page arXiv 2025

  27. [27]

    Shengbang Tong, Erik Jones, and Jacob Steinhardt. 2023. Mass-Producing Failures of Multimodal Systems with Language Models.Advances in Neural Information Processing Systems(2023)

    work page 2023

  28. [28]

    Ryan T Tymkow, Benjamin D Schnapp, Mojtaba Valipour, and Ali Ghodshi

    work page

  29. [29]

    Symbolic-Diffusion: Deep Learning Based Symbolic Regression with D3PM Discrete Token Diffusion.arXiv preprint arXiv:2510.07570(2025)

    work page arXiv 2025

  30. [30]

    Leonardo Vanneschi, Mauro Castelli, and Sara Silva. 2014. A survey of semantic methods in genetic programming.Genetic Programming and Evolvable Machines 15, 2 (2014), 195–214

    work page 2014

  31. [31]

    Martin Vastl, Jonáš Kulhánek, Jiří Kubalík, Erik Derner, and Robert Babuška

    work page

  32. [32]

    Symformer: End-to-end symbolic regression using transformer-based architecture.IEEE Access12 (2024), 37840–37849

    work page 2024

  33. [33]

    Henrik Voigt, Paul Kahlmeyer, Kai Lawonn, Michael Habeck, and Joachim Giesen

    work page

  34. [34]

    Analyzing Generalization in Pre-Trained Symbolic Regression.arXiv preprint arXiv:2509.19849(2025)

    work page arXiv 2025

  35. [35]

    Tan Wang, Kevin Lin, Linjie Li, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. 2023. Equivariant similarity for vision- language foundation models. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision. 11998–12008

    work page 2023

  36. [36]

    Stephan M Winkler, Michael Affenzeller, Bogdan Burlacu, Gabriel Kronberger, Michael Kommenda, and Philipp Fleck. 2018. Similarity-based analysis of pop- ulation dynamics in genetic programming performing symbolic regression. In Genetic Programming Theory and Practice XIV. Springer, 1–17

    work page 2018

  37. [37]

    Piotr Wyrwiński and Krzysztof Krawiec. 2025. Learning Semantics-aware Search Operators for Genetic Programming. InProceedings of the Genetic and Evolution- ary Computation Conference Companion. 659–662

    work page 2025

  38. [38]

    Zihan Yu, Jingtao Ding, Yong Li, and Depeng Jin. 2025. Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimiz- ing description length. InThe Thirteenth International Conference on Learning Representations

    work page 2025

  39. [39]

    Zihan Yu, Guanren Wang, Jingtao Ding, Huandong Wang, and Yong Li. 2025. Be- yond Formula Complexity: Effective Information Criterion Improves Performance and Interpretability for Symbolic Regression.arXiv preprint arXiv:2509.21780 (2025)

    work page arXiv 2025