Pith. sign in

REVIEW 3 cited by

Benchmarking Large Multimodal Models against Common Corruptions

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.11943 v1 pith:STLCNAFA submitted 2024-01-22 cs.LG cs.CLcs.CRcs.CVcs.MM

Benchmarking Large Multimodal Models against Common Corruptions

Jiawei Zhang , Tianyu Pang , Chao Du , Yi Ren , Bo Li , Min Lin This is my paper
classification cs.LG cs.CLcs.CRcs.CVcs.MM
keywords commoncorruptionslmmsbenchmarkinglargemmcbenchmodelsmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This technical report aims to fill a deficiency in the assessment of large multimodal models (LMMs) by specifically examining the self-consistency of their outputs when subjected to common corruptions. We investigate the cross-modal interactions between text, image, and speech, encompassing four essential generation tasks: text-to-image, image-to-text, text-to-speech, and speech-to-text. We create a comprehensive benchmark, named MMCBench, that covers more than 100 popular LMMs (totally over 150 model checkpoints). A thorough evaluation under common corruptions is critical for practical deployment and facilitates a better understanding of the reliability of cutting-edge LMMs. The benchmarking code is available at https://github.com/sail-sg/MMCBench

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Can Multimodal Large Language Models Understand OCT?

    cs.CV 2026-07 conditional novelty 6.0

    OCT-Bench, a 20-task benchmark across 10,076 questions, shows current MLLMs score up to 62% overall but only 43% on clinical reasoning over OCT images.

  2. DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification

    cs.CL 2026-05 unverdicted novelty 6.0

    DeltaRubric decomposes multimodal preference evaluation into self-generated planning and verification steps within a single model, producing large accuracy improvements on VL-RewardBench via multi-role reinforcement learning.

  3. Confidence-Aware Tool Orchestration for Robust Video Understanding

    cs.CV 2026-06 unverdicted novelty 5.0

    Robust-TO integrates per-frame reliability scores into tool orchestration and a confidence-cost GRPO reward to improve video reasoning robustness under corruption.