Pith. sign in

REVIEW 4 major objections 6 minor 52 references

Consistency of Compositional Generalization across Multiple Levels

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Vision-and-language models are inconsistent across levels of novel composition, and a meta-learning curriculum improves that consistency without sacrificing accuracy.

desk verdict Introduces a genuinely new evaluation axis—cross-level compositional consistency—with a new benchmark and a model-agnostic training framework; the empirical gains are consistent across baselines, but the complexity proxy and dataset numbers need scrutiny. read the letter →

arxiv 2412.13636 v1 pith:6BJE7YBD submitted 2024-12-18 cs.CV cs.AI

Chuanhao Li , Zhen Li , Chenchen Jing , Xiaomeng Fan , Wenbo Ye , Yuwei Wu , Yunde Jia This is my paper · ORCID
classification cs.CVcs.AI
keywords compositionalgeneralizationconsistencyvisualquestionansweringmeta-learningmultileveloptimizationsampleweightingtemporalvideogroundingGQA-CCG
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing vision-and-language models are good at some novel compositions and bad at others, but the paper argues that the deeper problem is consistency: when a model handles a hard phrase-phrase composition, it should also handle the simpler phrase-word and word-word compositions derived from it. Most current models, including large pretrained ones, answer barely 40% of such derived triplets consistently, even when they score well on individual levels. The paper introduces GQA-CCG, a VQA dataset of 8,702 triplet questions, and a consistency metric that counts triplets answered entirely correctly. It then proposes a meta-learning framework, MLO, that trains a model by progressively fitting validation sets of increasing compositional complexity; across five VQA baselines and one temporal-video-grounding baseline, MLO improves both per-level accuracy and consistency, with the strongest result (CFR+MLO) raising consistency from 46.46% to 49.27% while keeping overall accuracy above 74% and GQA test-dev accuracy stable.

What carries the argument

The machinery is a multilevel optimization loop over K validation sets $\{\mathcal{D}_v^i\}_{i=1}^K$, partitioned from the training set by an approximate complexity score: the length of the longest phrase in the question, with Eq. (1) balancing similar complexity against similar set size. Each validation set has its own meta-weight-net, a small fully connected network with sigmoid output, that takes the question embedding and outputs a sample weight $w_d$. During parameter optimization the model minimizes the weighted training loss $\mathcal{L}_t$; during meta optimization the weight networks are updated sequentially, from the simplest to the most complex validation set, so that later weight networks fit sets whose harder compositions presuppose the earlier ones. Reversing the order (complex to simple) yields almost no gain, which the paper reads as evidence that the ordering, not the reweighting alone, carries the effect.

What would settle it

A direct test would be to rerun the framework with the validation sets reordered by a different complexity proxy, such as human difficulty ratings, parse-tree depth, or the number of novel concept pairs. If the simple-to-complex ordering no longer outperforms complex-to-simple or simultaneous weighting, the claimed curriculum mechanism is not what drives the consistency gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that cross-level consistency is a real, separately measurable property of compositional generalization, distinct from per-level accuracy, and that it can be improved by explicit simple-to-complex curriculum. On its new GQA-CCG benchmark, models with up to 37B parameters achieve only about 40% consistency, showing that even models that master many novel compositions do not reliably master the simpler compositions that are their building blocks. The proposed MLO framework divides the training set into K validation sets of increasing complexity, assigns each a meta-weight-net that produces per-sample weights, and optimizes the model and the weight networks in a nested loop so that the validation sets are fit in order from simple to complex. The results show consistent gains in accuracy and consistency across attention-based, graph-based, module-network, and pretrained VQA baselines, and across temporal video grounding, with no loss on the IID GQA test set.

Load-bearing premise

The load-bearing premise is that the length of the longest phrase in a question is a valid proxy for how hard it is to compose the concepts in that question; if that ordering is wrong, the framework is reweighting samples rather than actually teaching compositions from simple to complex.

Editorial extensions

If this is right

  • If consistency is a distinct axis of compositional generalization, evaluations should report triplet-level consistency alongside per-level accuracy, since models can score high on individual levels while failing the consistency test.
  • The simple-to-complex ordering is load-bearing: swapping it for complex-to-simple removes the gains, so training curricula for compositional tasks should respect the derivation order of compositions.
  • The framework transfers across model families and tasks, so consistency improvement is not tied to a single architecture or to visual question answering alone.
  • The IID results on GQA indicate that on these baselines the consistency gains do not trade off against standard in-distribution accuracy, which is the failure mode of many OOD-oriented methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The consistency metric could serve as a probe for systematicity in large language and vision-language models: a classifier over incorrect triplet patterns might reveal whether failures concentrate at the phrase-word bridge or the word-word base, giving a diagnostic that accuracy alone cannot provide.
  • Because the complexity proxy is longest-phrase length, an immediate test is whether other proxies, such as parse-tree depth or number of novel concept pairs, reproduce the ordering effects; if gains persist under different proxies, the curriculum mechanism is robust, and if not, the gains may come from reweighting rather than from simple-to-complex teaching.
  • A natural extension is to apply the same nested-loop reweighting to generative tasks, measuring consistency of instructed outputs rather than correctness of multiple-choice answers.
  • The near-40% ceiling for large pretrained models suggests that scale and broad pretraining do not by themselves buy cross-level consistency; the paper's results point to training-time curricula as a cheaper complement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces a new evaluation criterion for compositional generalization in vision-and-language models, namely consistency across phrase-phrase, phrase-word, and word-word levels of novel compositions. It proposes a meta-learning framework (MLO) that partitions the training set into validation sets ordered by an approximate compositional-complexity measure (longest phrase length), uses separate meta-weight-nets to reweight samples in each validation set, and optimizes the model and the meta-weight-nets in a multilevel, simple-to-complex fashion. The authors construct a new VQA benchmark, GQA-CCG, with 8,702 triplets of questions at the three composition levels, define a triplet-level consistency metric, and report experiments on five VQA baselines and one temporal-video-grounding baseline, along with ablations and IID generalization checks on GQA.

Significance. If the results are trustworthy, the paper identifies a genuinely new and measurable failure mode of current vision-and-language models: even large pretrained models achieve only around 40% consistency across composition levels, and the proposed MLO framework consistently improves both consistency and per-level accuracy on multiple baselines while retaining IID accuracy. The released GQA-CCG dataset and the consistency metric are concrete contributions that could support future work. However, the significance is currently conditional: the dataset statistics contain an unresolved internal inconsistency, the central complexity proxy is unvalidated against the actual composition-novelty structure, and the reported gains lack error bars or significance testing.

major comments (4)
  1. [GQA-CCG Dataset (paragraph after Manual Review)] The reported dataset totals are internally inconsistent. The text states that DCCG consists of 8,702 triplets and 18,983 samples, but 8,702 × 3 = 26,106, and the three per-level counts (5,125 + 8,102 + 5,756) sum exactly to 18,983. If a triplet always contains one phrase-phrase, one phrase-word, and one word-word sample, the triplet count and the per-level counts cannot differ as reported. Since Eq. (9) computes consistency at the triplet level and Table 1 reports per-level accuracy, the reader cannot determine whether the consistency metric and the per-level accuracies are computed over the same data. Please provide an exact breakdown of unique samples and triplet memberships, or correct the arithmetic, so the benchmark statistics are auditable.
  2. [Validation Set Construction (Eq. (1) and surrounding text)] The paper asserts that 'compositional complexity is proportional to the length of the longest phrase in a question' and uses this to order validation sets from simple to complex. No evidence is given that L(Q) tracks the levels of novel compositions defined in Eq. (8). A question can contain a short phrase-phrase novel composition such as 'golden dog' + 'white cat' with a small L(Q), while a long relative-clause question can have a large L(Q) with no novel composition. The multilevel optimization in Eq. (3) is only meaningful as a simple-to-complex curriculum if the validation sets actually increase in compositional difficulty. Please validate the proxy (e.g., by reporting the distribution of L(Q) across the three GQA-CCG levels, or by showing that the S→C advantage in Table 4 persists when validation sets are built from explicit composition-novelty annotations rather than phrase length).
  3. [Experimental results (Tables 1-4)] All experimental results are single-run; no error bars, multiple seeds, or significance tests are reported. The headline consistency improvements are modest in absolute terms (e.g., MAC from 30.82% to 34.10%; CFR from 46.46% to 49.27%), and for models trained with different random seeds such differences could plausibly arise from run-to-run variance. Please report mean ± standard deviation over at least three seeds, or provide bootstrap confidence intervals for the main tables, so the claimed improvements can be distinguished from noise.
  4. [Multilevel Optimization (Eq. (3))] The nested formulation in Eq. (3) is notationally confusing: the loops are written as if each LOOP is a constrained argmin over one meta-weight-net, but the actual training alternates parameter optimization and meta optimization, and the meta-weight-nets are updated sequentially. The paper should clarify the precise algorithm (e.g., pseudocode or a clear step-by-step description) and state how the approximation via the implicit function theorem interacts with the sequential updates. This is important for reproducibility, especially because the supplementary details are referenced but not included in this version.
minor comments (6)
  1. [GQA-CCG Dataset] In the second paragraph, 'we use the the train balanced split' contains a duplicated 'the'.
  2. [Sample Generation Pipeline] In Automatic Generation, 'we fill associated infos' should be 'we fill the associated information'.
  3. [Table 1 and Experimental Settings] The model names 'LLaV A-1.5-Xtuner' and 'LLaV A-1.6' should be written without the space, as 'LLaVA-1.5-Xtuner' and 'LLaVA-1.6'.
  4. [Conclusion] The conclusion says the dataset enables 'qualitative evaluation', which appears to be a typo for 'quantitative evaluation'.
  5. [Eq. (3)] The notation 'LOOPK : ...' followed by 'LOOP2 : s.t. ...' is hard to read as a nested optimization; consider rewriting as a sequence of explicit optimization problems or an algorithmic loop.
  6. [Compositional Generalization Performance] The phrase 'varies in parameters' should be 'varying in parameter count' or 'with different numbers of parameters'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: MLO gains are measured on a held-out GQA-CCG test set, and the longest-phrase complexity proxy is an unvalidated assumption rather than a fitted input.

full rationale

The paper's central claim is that multilevel optimization (MLO) improves consistency of compositional generalization across levels. The supporting experiments (Table 1) compare each baseline with and without MLO on GQA-CCG, which is constructed from the GQA val_all split, while the training set Dt and validation sets Dv_i are derived from the GQA train_balanced split. The meta-weight-nets are optimized on validation losses over Dv_i (Eqs. 3-5), which are subsets of the training set; the GQA-CCG test labels and triplet structure never enter the optimization. Therefore the reported consistency and per-level accuracy improvements are not forced by construction. The consistency metric in Eq. (9) is a new definition, but it is applied to held-out triplets and no model parameter is fitted to it. The paper's main vulnerability is the assertion in the Validation Set Construction section that 'compositional complexity is proportional to the length of the longest phrase in a question,' used in Eq. (1) to order validation sets. This is an unvalidated proxy: GQA-CCG's difficulty is defined by novel compositions via the inclusion relation in Eq. (8), not by surface phrase length. However, this is a validity or correctness concern about the curriculum mechanism, not circularity: the training signal is not derived from the test metric. Self-citations (e.g., Jing et al. 2022 on reasoning consistency, Li et al. 2023b and 2024 on primitives and substitutivity) appear in related work and are not load-bearing for the central empirical claim. Thus the derivation chain is self-contained against the held-out benchmark, and no circular step can be exhibited.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the dataset and the curriculum mechanism. No continuous free parameters are fitted to the reported test labels. The main hand-chosen design choices are K and M, plus the complexity proxy and the phrase parser. The dataset and metric are introduced by the authors, so the benchmark is internal to the paper; this increases the need for independent replication but is not circular in the derivational sense.

free parameters (2)
  • K (number of validation sets) = not reported
    Hyperparameter controlling the granularity of the simple-to-complex curriculum; chosen by hand and not specified numerically in the main text.
  • M (number of in-context exemplars per question prefix) = not reported
    Used in GQA-CCG construction to select diverse in-context examples; affects the distribution of generated questions and is not reported.
assumptions (5)
  • ad hoc to paper The length of the longest phrase in a question is a valid proxy for compositional complexity.
    Adopted in 'Validation Set Construction' (Eq. 1) to split the training set into ordered difficulty groups; if this proxy fails, the curriculum mechanism is not actually simple-to-complex.
  • domain assumption benepar phrase extraction accurately identifies phrases in questions.
    Used both for constructing validation sets and for detecting seen and novel compositions in GQA-CCG; parsing errors would mislabel compositional complexity and novelty.
  • domain assumption GPT-3.5 with in-context examples plus manual review produces correct and comparable QA pairs for the generated word-word and phrase-word questions.
    The GQA-CCG benchmark and the consistency metric rely on the correctness of these generated questions and answers; the paper reports manual review but no inter-annotator agreement.
  • domain assumption Multiple meta-weight-nets with three-layer MLP can learn to produce sample weights that induce useful curriculum ordering.
    The framework's performance depends on the capacity and trainability of the meta-weight-nets, which is assumed rather than proven.
  • standard math The implicit function theorem approximation (Lorraine et al. 2020) gives accurate enough meta-gradients for the sequential multilevel optimization.
    Used in 'Meta Optimization' to approximate the best-response Jacobian; this is standard math from the cited literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Consistency of Compositional Generalization across Multiple Levels." pith.science (2026). https://pith.science/paper/6BJE7YBD

@misc{pith2026241213636,
  author       = {Pith},
  title        = {Pith review of: Consistency of Compositional Generalization across Multiple Levels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6BJE7YBD}},
  note         = {Machine review of arXiv:2412.13636}
}
read the original abstract

Compositional generalization is the capability of a model to understand novel compositions composed of seen concepts. There are multiple levels of novel compositions including phrase-phrase level, phrase-word level, and word-word level. Existing methods achieve promising compositional generalization, but the consistency of compositional generalization across multiple levels of novel compositions remains unexplored. The consistency refers to that a model should generalize to a phrase-phrase level novel composition, and phrase-word/word-word level novel compositions that can be derived from it simultaneously. In this paper, we propose a meta-learning based framework, for achieving consistent compositional generalization across multiple levels. The basic idea is to progressively learn compositions from simple to complex for consistency. Specifically, we divide the original training set into multiple validation sets based on compositional complexity, and introduce multiple meta-weight-nets to generate sample weights for samples in different validation sets. To fit the validation sets in order of increasing compositional complexity, we optimize the parameters of each meta-weight-net independently and sequentially in a multilevel optimization manner. We build a GQA-CCG dataset to quantitatively evaluate the consistency. Experimental results on visual question answering and temporal video grounding, demonstrate the effectiveness of the proposed framework. We release GQA-CCG at https://github.com/NeverMoreLCH/CCG.

Figures

Figures reproduced from arXiv: 2412.13636 by the authors.

Figure 1
Figure 1. Illustration of multiple levels of novel compo [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed framework. a training set Dt, we first divide Dt into multiple valida￾tion sets {Dvi } K i=1 based on the compositional complexity of samples. A larger i indicates more complex samples in Dvi . Then we introduce K meta-weight-nets into the model, and use i-th meta-weight-net to generate sample weights for the samples in Dvi . Finally, we train the model and the meta￾weight-nets via a multile… view at source ↗
Figure 3
Figure 3. Validation set construction in the context of VQA, [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Overview of the pipeline for constructing GQA [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparisons between MMN+MLO (Ours) and MMN (Chen et al. 2021). for the questions with associated simple phrase-word/word￾word compositions. (2) MMN+MLO (Ours) makes predic￾tions accurately on all questions. These observations show that our framework is effe…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 41 canonical work pages

  1. [1]

    W.; Ilharco, G.; Wortsman, M.; and Schmidt, L

    Awadalla, A.; Gao, I.; Gardner, J.; Hessel, J.; Hanafy, Y.; Zhu, W.; Marathe, K.; Bitton, Y.; Gadre, S.; Sagawa, S.; Jitsev, J.; Kornblith, S.; Koh, P. W.; Ilharco, G.; Wortsman, M.; and Schmidt, L. 2023. OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models. arXiv preprint arXiv:2308.01390

    arXiv 2023

  2. [2]

    Bienenstock, E. 1996. Composition. In Brain theory, 269--300. Elsevier

    work page 1996

  3. [3]

    Chen, W.; Gan, Z.; Li, L.; Cheng, Y.; Wang, W.; and Liu, J. 2021. Meta module network for compositional visual reasoning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 655--664

    work page 2021

  4. [4]

    Cho, J.; Lei, J.; Tan, H.; and Bansal, M. 2021. Unifying vision-and-language tasks via text generation. In Proceedings of the International Conference on Machine Learning, 1931--1942. PMLR

    work page 2021

  5. [5]

    W.; Kim, D.-J.; Ryu, H.; and Kweon, I

    Cho, J. W.; Kim, D.-J.; Ryu, H.; and Kweon, I. S. 2023. Generative bias for robust visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11681--11690

    work page 2023

  6. [6]

    K.; Neiswanger, W.; Xie, P.; and Xing, E

    Choe, S. K.; Neiswanger, W.; Xie, P.; and Xing, E. 2023. Betty: An Automatic Differentiation Library for Multilevel Optimization. In Proceedings of the International Conference on Learning Representations

    work page 2023

  7. [7]

    Contributors, O. 2023 a . OpenCompass: A Universal Evaluation Platform for Foundation Models. https://github.com/open-compass/opencompass

    work page 2023

  8. [8]

    Contributors, X. 2023 b . XTuner: A Toolkit for Efficiently Fine-tuning LLM. https://github.com/InternLM/xtuner

    work page 2023

Show all 52 references
  1. [9]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language T...

    2019

  2. [10]

    Dong, X.; Zhang, P.; Zang, Y.; Cao, Y.; Wang, B.; Ouyang, L.; Wei, X.; Zhang, S.; Duan, H.; Cao, M.; Zhang, W.; Li, Y.; Yan, H.; Gao, Y.; Zhang, X.; Li, W.; Li, J.; Chen, K.; He, C.; Zhang, X.; Qiao, Y.; Lin, D.; and Wang, J. 2024. InternLM-XComposer2: Mastering Free-form Text...

    2024 arXiv

  3. [11]

    A.; and Pylyshyn, Z

    Fodor, J. A.; and Pylyshyn, Z. W. 1988. Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1-2): 3--71

    1988

  4. [12]

    Hu, R.; Rohrbach, A.; Darrell, T.; and Saenko, K. 2019. Language-conditioned graph networks for relational reasoning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10294--10303

    2019

  5. [13]

    A.; and Manning, C

    Hudson, D. A.; and Manning, C. D. 2018. Compositional Attention Networks for Machine Reasoning. In Proceedings of the International Conference on Learning Representations

    2018

  6. [14]

    A.; and Manning, C

    Hudson, D. A.; and Manning, C. D. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6700--6709

    2019

  7. [15]

    Jing, C.; Jia, Y.; Wu, Y.; Liu, X.; and Wu, Q. 2022. Maintaining reasoning consistency in compositional visual question answering. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5099--5108

    2022

  8. [16]

    Kitaev, N.; Cao, S.; and Klein, D. 2019. Multilingual Constituency Parsing with Self-Attention and Pre-Training. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3499--3505. Florence, Italy: Association for Computational Linguistics

    2019

  9. [17]

    Li, B.; Zhang, Y.; Chen, L.; Wang, J.; Yang, J.; and Liu, Z. 2023 a . Otter: A Multi-Modal Model with In-Context Instruction Tuning. arXiv preprint arXiv:2305.03726

    2023 arXiv

  10. [18]

    Li, C.; Li, Z.; Jing, C.; Jia, Y.; and Wu, Y. 2023 b . Exploring the Effect of Primitives for Compositional Generalization in Vision-and-Language. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19092--19101

    2023

  11. [19]

    Li, C.; Li, Z.; Jing, C.; Wu, Y.; Zhai, M.; and Jia, Y. 2024. Compositional Substitutivity of Visual Reasoning for Visual Question Answering. In Proceedings of the European Conference on Computer Vision, 143--160. Springer

    2024

  12. [20]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023 c . Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597

    2023 arXiv

  13. [21]

    Li, J.; Xie, J.; Qian, L.; Zhu, L.; Tang, S.; Wu, F.; Yang, Y.; Zhuang, Y.; and Wang, X. E. 2022. Compositional temporal grounding with structured variational cross-graph correspondence learning. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition,...

    2022

  14. [22]

    Liu, H.; Li, C.; Li, Y.; Li, B.; Zhang, Y.; Shen, S.; and Lee, Y. J. 2024. LLaVA-NeXT: Improved reasoning, OCR, and world knowledge

    2024

  15. [23]

    Liu, Q.; An, S.; Lou, J.-G.; Chen, B.; Lin, Z.; Gao, Y.; Zhou, B.; Zheng, N.; and Zhang, D. 2020. Compositional generalization by learning analytical expressions. Advances in Neural Information Processing Systems, 33: 11416--11427

    2020

  16. [24]

    Lorraine, J.; Vicol, P.; and Duvenaud, D. 2020. Optimizing millions of hyperparameters by implicit differentiation. In International conference on artificial intelligence and statistics, 1540--1552. PMLR

    2020

  17. [25]

    M.; and V \"a rbrand, P

    Migdalas, A.; Pardalos, P. M.; and V \"a rbrand, P. 2013. Multilevel optimization: algorithms and applications, volume 20. Springer Science & Business Media

    2013

  18. [26]

    X.; Do, T.; Tran, H.; Tjiputra, E.; Tran, Q

    Nguyen, B. X.; Do, T.; Tran, H.; Tjiputra, E.; Tran, Q. D.; and Nguyen, A. 2022. Coarse-to-Fine Reasoning for Visual Question Answering. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 4557--4565

    2022

  19. [27]

    E.; Sigaud, O.; Perrin, N.; Laterre, A.; Kas, D.; Beguir, K.; and de Freitas, N

    Pierrot, T.; Ligner, G.; Reed, S. E.; Sigaud, O.; Perrin, N.; Laterre, A.; Kas, D.; Beguir, K.; and de Freitas, N. 2019. Learning compositional neural programs with recursive tree search and planning. Advances in Neural Information Processing Systems, 32

    2019

  20. [28]

    L.; Moreno, R.; and Br \"u nken, R

    Plass, J. L.; Moreno, R.; and Br \"u nken, R. 2010. Cognitive load theory

    2010

  21. [29]

    M.; and Benton, M

    Radziwill, N. M.; and Benton, M. C. 2017. Evaluating quality of chatbots and intelligent conversational agents. arXiv preprint arXiv:1704.04579

    2017 arXiv

  22. [30]

    Ray, A.; Sikka, K.; Divakaran, A.; Lee, S.; and Burachas, G. 2019. Sunny and Dark Outside?! Improving Answer Consistency in VQA through Entailed Question Generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Join...

    2019

  23. [31]

    T.; Guestrin, C.; and Singh, S

    Ribeiro, M. T.; Guestrin, C.; and Singh, S. 2019. Are red roses red? evaluating consistency of question-answering models. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 6174--6184

    2019

  24. [32]

    Saqur, R.; and Narasimhan, K. 2020. Multimodal graph networks for compositional generalization in visual question answering. In Advances in Neural Information Processing Systems, 3070--3081

    2020

  25. [33]

    R.; Tendulkar, P.; Parikh, D.; Horvitz, E.; Ribeiro, M

    Selvaraju, R. R.; Tendulkar, P.; Parikh, D.; Horvitz, E.; Ribeiro, M. T.; Nushi, B.; and Kamar, E. 2020. Squinting at vqa models: Introspecting vqa models with sub-questions. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10003--10011

    2020

  26. [34]

    Shah, M.; Chen, X.; Rohrbach, M.; and Parikh, D. 2019. Cycle-consistency for robust visual question answering. In 2019 IEEE. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6642--6651

    2019

  27. [35]

    Shu, J.; Xie, Q.; Yi, L.; Zhao, Q.; Zhou, S.; Xu, Z.; and Meng, D. 2019. Meta-weight-net: Learning an explicit mapping for sample weighting. Advances in neural information processing systems, 32

    2019

  28. [36]

    Sun, Q.; Cui, Y.; Zhang, X.; Zhang, F.; Yu, Q.; Wang, Y.; Rao, Y.; Liu, J.; Huang, T.; and Wang, X. 2024. Generative multimodal models are in-context learners. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14398--14409

    2024

  29. [37]

    Tan, H.; and Bansal, M. 2019. LXMERT: Learning Cross-Modality Encoder Representations from Transformers. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing, 5100--5111

    2019

  30. [38]

    Tascon-Morales, S.; M \'a rquez-Neila, P.; and Sznitman, R. 2023. Logical Implications for Visual Question Answering Consistency. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6725--6735

    2023

  31. [39]

    Wang, K.; Wu, Y.; Cen, J.; Pan, Z.; Li, X.; Wang, Z.; Cao, Z.; and Lin, G. 2024. Self-Supervised Class-Agnostic Motion Prediction with Spatial and Temporal Consistency Regularizations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14638--14647

    2024

  32. [40]

    Wang, Q.; Liu, L.; Jing, C.; Chen, H.; Liang, G.; Wang, P.; and Shen, C. 2023 a . Learning Conditional Attributes for Compositional Zero-Shot Learning. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11197--11206

    2023

  33. [41]

    Wang, W.; Lv, Q.; Yu, W.; Hong, W.; Qi, J.; Wang, Y.; Ji, J.; Yang, Z.; Zhao, L.; Song, X.; et al. 2023 b . Cogvlm: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079

    2023 arXiv

  34. [42]

    H.; Shang, X.; Yuan, Z.; Sun, Y.; and Liu, J

    Xu, L.; Huang, M. H.; Shang, X.; Yuan, Z.; Sun, Y.; and Liu, J. 2023. Meta compositional referring expression segmentation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19478--19487

    2023

  35. [43]

    Xu, X.; Chen, X.; Liu, C.; Rohrbach, A.; Darrell, T.; and Song, D. 2018. Fooling vision and language models despite localization and attention mechanism. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4951--4961

    2018

  36. [44]

    Yang, L.; Kong, Q.; Yang, H.-K.; Kehl, W.; Sato, Y.; and Kobori, N. 2023. Deco: Decomposition and reconstruction for compositional temporal grounding via coarse-to-fine contrastive ranking. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 23130--23140

    2023

  37. [45]

    Yang, X.; Zuo, Y.; Ramasinghe, S.; Bazzani, L.; Avraham, G.; and van den Hengel, A. 2024. ViewFusion: Towards Multi-View Consistency via Interpolated Denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9870--9880

    2024

  38. [46]

    Ye, Q.; Xu, H.; Ye, J.; Yan, M.; Hu, A.; Liu, H.; Qian, Q.; Zhang, J.; and Huang, F. 2024. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13040--13051

    2024

  39. [47]

    Yuan, Y.; Wang, S.; Jiang, M.; and Chen, T. Y. 2021. Perception matters: Detecting perception failures of vqa models using metamorphic testing. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16908--16917

    2021

  40. [48]

    Zhang, S.; Peng, H.; Fu, J.; Lu, Y.; and Luo, J. 2021. Multi-scale 2d temporal adjacency networks for moment localization with natural language. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12): 9073--9087

    2021

  41. [49]

    Zhang, Y.; Luo, H.; and Lei, Y. 2024. Towards CLIP-driven Language-free 3D Visual Grounding via 2D-3D Relational Enhancement and Consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13063--13072

    2024

  42. [50]

    Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

    2023 arXiv

  43. [51]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.