Pith. sign in

REVIEW 17 cited by

CoCoT: Contrastive Chain-of-Thought Prompting for Large Multimodal Models with Multiple Image Inputs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.02582 v1 pith:D24PM5SG submitted 2024-01-05 cs.CV

Daoan Zhang , Junming Yang , Hanjia Lyu , Zijian Jin , Yuan Yao , Mingkai Chen , Jiebo Luo This is my paper · ORCID
classification cs.CV
keywords modelslmmsmultipleimageinputslargemultimodalcocot
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

When exploring the development of Artificial General Intelligence (AGI), a critical task for these models involves interpreting and processing information from multiple image inputs. However, Large Multimodal Models (LMMs) encounter two issues in such scenarios: (1) a lack of fine-grained perception, and (2) a tendency to blend information across multiple images. We first extensively investigate the capability of LMMs to perceive fine-grained visual details when dealing with multiple input images. The research focuses on two aspects: first, image-to-image matching (to evaluate whether LMMs can effectively reason and pair relevant images), and second, multi-image-to-text matching (to assess whether LMMs can accurately capture and summarize detailed image information). We conduct evaluations on a range of both open-source and closed-source large models, including GPT-4V, Gemini, OpenFlamingo, and MMICL. To enhance model performance, we further develop a Contrastive Chain-of-Thought (CoCoT) prompting approach based on multi-input multimodal models. This method requires LMMs to compare the similarities and differences among multiple image inputs, and then guide the models to answer detailed questions about multi-image inputs based on the identified similarities and differences. Our experimental results showcase CoCoT's proficiency in enhancing the multi-image comprehension capabilities of large multimodal models.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TaskGalaxy: Scaling Multi-modal Instruction Fine-tuning with Tens of Thousands Vision Task Types

    cs.CV 2025-02 conditional novelty 7.0 of 10

    The authors release TaskGalaxy, a 413k-sample multimodal instruction dataset with 19,227 hierarchically organized task types, and report consistent benchmark gains when it is added to LLaVA-v1.5 and InternVL-Chat-v1.0.

  2. PDB-Eval: An Evaluation of Large Multimodal Models for Description and Explanation of Personalized Driving Behavior

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Introduces PDB-Eval, a dual-view benchmark for fine-grained driver behavior description and explanation, and shows fine-tuning on it boosts performance on driving QA and downstream intention and recognition tasks.

  3. Can Multimodal Large Language Models Understand Spatial Relations?

    cs.CV 2025-05 conditional novelty 6.0 of 10

    SpatialMQA, a new spatial-relation benchmark, shows the top MLLM reaches 48.14% accuracy versus 98.40% for humans.

  4. A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMs

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A three-paradigm evaluation framework shows that reasoning over text descriptions of images (Componential Analysis) outperforms direct visual reasoning on Bongard and Winoground benchmarks, and that many open-source V...

  5. CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models

    cs.CV 2024-12 conditional novelty 6.0 of 10

    CoMT is the first benchmark to ask LVLMs to produce interleaved image and text rationales, and current models perform near random on it.

  6. MageBench: Bridging Large Multimodal Models to Agents

    cs.CV 2024-12 conditional novelty 6.0 of 10

    MageBench introduces a 483-scenario benchmark showing current large multimodal models are far weaker than humans at agent tasks requiring continuous visual feedback and planning.

  7. MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large Models

    cs.AI 2025-12 conditional novelty 5.0 of 10

    MIND improves multimodal reasoning by training on diverse correct and deliberately wrong rationales with two-stage correction and contrastive alignment, reporting SOTA on ScienceQA, A-OKVQA, and M3CoT.

  8. VisRAG2.0: Mitigating Visual Hallucinations via Evidence-Guided Multi-Image Reasoning in Visual Retrieval-Augmented Generation

    cs.CL 2025-10 conditional novelty 5.0 of 10

    EVisRAG, an evidence-guided observe-record-reason-answer pipeline trained with reward-scoped GRPO, improves multi-image visual QA accuracy by about 19% over its backbone VLM across five benchmarks.

  9. From Answers to Rationales: Self-Aligning Multimodal Reasoning with Answer-Oriented Chain-of-Thought

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Answer-oriented chain-of-thought prompts that generate both positive and negative reasoning data, combined with iterative DPO, improve multimodal LLM reasoning on several benchmarks.

  10. Extracting Multimodal Learngene in CLIP: Unveiling the Multimodal Generalizable Knowledge

    cs.CV 2025-06 conditional novelty 5.0 of 10

    MM-LG extracts a compact multimodal and unimodal block set from CLIP via distillation and uses it to initialize smaller vision-language and vision models, outperforming previous Learngene methods and sometimes pre-tra...

  11. WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning

    cs.CV 2025-06 conditional novelty 5.0 of 10

    WeThink, a 120K-image QA dataset with AI-generated reasoning paths, combined with hybrid-reward reinforcement learning, improves a 7B vision-language model across 14 benchmarks.

  12. Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination

    cs.CV 2024-11 conditional novelty 5.0 of 10

    VIC improves multimodal LLM accuracy on hallucination and general VQA benchmarks by reasoning from text before seeing the image, though gains are inconsistent on open-source models.

  13. PathCoT: Chain-of-Thought Prompting for Zero-shot Pathology Visual Reasoning

    cs.LG 2025-06 conditional novelty 4.0 of 10

    A chain-of-thought prompting method with four pathology experts and a self-evaluation step improves zero-shot accuracy on the PathMMU pathology visual-reasoning benchmark.

  14. A New Method to Capturing Compositional Knowledge in Linguistic Space

    cs.CV 2024-12 reject novelty 4.0 of 10

    YUKINO reports state-of-the-art scores on SugarCREPE and Winoground by injecting image-derived pseudo-tokens into captions and comparing yes versus no prompts, a protocol that likely inflates the results.

  15. How Vision-Language Tasks Benefit from Large Pre-trained Models: A Survey

    cs.CV 2024-12 conditional novelty 4.0 of 10

    A survey that categorizes pre-trained-model-based vision-language methods into four challenge-driven paradigms, with performance tables and a discussion of risks.

  16. Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation

    cs.AI 2024-12 conditional novelty 4.0 of 10

    Chain-of-thought prompting makes LLMs imitate exemplar formatting, concentrate their final answer probabilities, and activate a wider set of final-layer neurons.

  17. Multi-granular Training Strategies for Robust Multi-hop Reasoning Over Noisy and Heterogeneous Knowledge Sources

    cs.CL 2025-02 reject novelty 2.0 of 10

    AMKOR is described as a state-of-the-art multi-hop QA system, but the paper provides no reproducible evidence and the reported numbers appear unverifiable.

Pith tools