Pith. sign in

REVIEW 3 major objections 5 minor 12 references

SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that conflicting scene text systematically degrades multimodal geolocation and redirects predictions toward injected geographic targets, raising median localization error 4.8-fold across 20 models.

desk verdict A strong, well-executed benchmark for MLLM conflict resolution, but the directed-shift headline needs a Random-condition null before it fully lands. read the letter →

arxiv 2608.04244 v1 pith:RER5YUPY submitted 2026-08-04 cs.CV cs.CL

Sirun Li , Minghao Liu , Ling Dai , Yong Li , Haoxin Lyu , Junting Zhou , Fan Zhang This is my paper · ORCID
classification cs.CVcs.CL
keywords multimodallargelanguagemodelstext-visionconflictresolutionvisualgeolocationcounterfactualbenchmarkscenetextperturbationrobustnessbiasscoretrap-fitrate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SIGNPOST-Bench tests whether multimodal large language models can arbitrate between visual scene structure and readable scene text when the two disagree. Each source image becomes a counterfactual quintuplet—Original, Blank, Similar, Random, Adversarial—so that only the meaning of selected text spans changes while surrounding non-textual content is meant to stay fixed. Across 20 models and four datasets, the benchmark reports that Adversarial text raises median localization error from 282 km to 1,347 km (4.8x), that 6.5–20.1% of adversarial predictions land within 50 km of an injected geographic target, and that every model moves closer to the injected target on average relative to the Blank baseline. The authors argue this establishes visual geolocation as a continuous diagnostic for text–vision conflict and shows that clean-input capability does not predict conflict robustness. The stakes are practical: any image-based localization system that reads signs is exposed to misleading or tampered text.

What carries the argument

The load-bearing object is the counterfactual quintuplet: five matched images of the same scene—Original, Blank, Similar, Random, Adversarial—differing only in localized scene-text edits generated by an MLLM and rendered by an image-editing model, with the Adversarial variant naming a real place from a different continent. Three diagnostic metrics carry the argument: Weighted Localization Accuracy (WLA) scores geodesic error with exponential decay; Text Bias Score (TBS) measures the paired change in ground-truth error from the Blank text-removed baseline to an edited variant; and Trap-Fit Rate (TFR) plus paired Trap Distance Reduction (TDR) measure whether predictions move toward the geocoded injected target. A three-tier scene-text coupling taxonomy (Portable, Cultural, Geo-Specific) stratifies how much native text helps and how much conflicting replacement hurts.

What would settle it

Take a random sample of Blank and Adversarial pairs beyond the 120 audited images and have independent raters or automated detectors identify non-text differences (buildings, sky, road geometry) that correlate with the injected target; if such differences appear systematically, the directed-shift conclusion would be confounded by editing artifacts rather than text semantics.

Watch

Extended reading notes

Core claim

The paper's central claim is that conflicting scene text is a systematic, measurable failure mode in current multimodal large language models, not a rare edge case. Using a controlled five-condition counterfactual design, it shows that replacing native scene text with a geographically conflicting place name degrades localization for every model on every dataset: macro-averaged WLA falls from 47.11 to 29.89, median error grows 4.8-fold, and 6.5–20.1% of adversarial predictions land within 50 km of the injected target. The paired Trap Distance Reduction is positive on average for all 20 models (model means 343–1,926 km), and compatible, unrelated, and conflicting text replacements produce distinct, ordered effects relative to the text-removed Blank baseline. The authors conclude that text–vision conflict should be evaluated separately from clean-input capability, because the two are not interchangeable.

Load-bearing premise

The counterfactual edits are assumed to change only the meaning of the selected scene-text spans while preserving surrounding non-textual content; only 120 edited images were human-audited, so subtle edit artifacts could contribute to the measured shifts.

Editorial extensions

If this is right

  • Adversarial scene text increases median localization error from 282 km to 1,347 km (4.8x) and lowers mean WLA from 47.11 to 29.89 across 20 models.
  • Conflicting text causes directed, target-aligned shifts: 6.5–20.1% of adversarial predictions fall within 50 km of the injected target, and every model shows a positive mean paired Trap Distance Reduction.
  • Text semantics matter, not just readable text: Similar replacements reduce error by 379 km relative to Blank, while Random and Adversarial replacements increase it by 959 km and 1,577 km.
  • Conflict robustness is separable from localization capability: models with modest clean-input performance can rank high in robustness, so capability scores do not predict behavior under conflict.
  • Prompting models to detect conflict does not reliably fix the failure: on two tested models, defense prompting improved conflict detection for one but lowered it for the other, and neither improved both detection and localization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the central claim holds, benchmarks that oversample Portable and Cultural text may understate text-conflict risk: Geo-Specific text (T3) yields the largest adversarial WLA drop (25.14 points, 42.2% relative).
  • The positive mean Trap Distance Reduction is driven by a minority of large shifts (attraction rates 44.5–60.6%), so average targetward movement can coexist with many samples staying put; deployment risk may concentrate in a small fraction of images.
  • A natural extension is to apply the same quintuplet logic to other continuous-output tasks, such as depth, time, or heading estimation, to test whether directed text-driven shifts are a general arbitration failure rather than a geolocation-specific one.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces SIGNPOST-Bench, a counterfactual benchmark for studying how multimodal large language models arbitrate between scene text and visual evidence in the continuous output space of geographic coordinates. Each of 5,111 source images becomes a quintuplet of matched variants (Original, Blank, Similar, Random, Adversarial) by locally removing or replacing selected scene-text spans, and 20 MLLMs from seven providers are evaluated on all variants, producing 511,100 model-image evaluations. The headline claims are: (i) conflicting scene text degrades geolocation, raising median error from 282 km to 1,347 km (4.8x) and lowering mean WLA from 47.11 to 29.89; (ii) compatible, unrelated, and conflicting text replacements produce distinct and ordered effects relative to the Blank baseline via TBS; (iii) adversarial text induces directed shifts toward the injected target, with 6.5-20.1% of geocodable adversarial predictions within 50 km of the target and positive mean paired Trap Distance Reduction for all 20 models; and (iv) capability and conflict robustness separate (C vs. R scores). The paper also reports a two-model probing/defense analysis and a scene-text coupling taxonomy (T1/T2/T3) with disclosed inter-annotator agreement.

Significance. If the directed-shift claim survives the missing control discussed below, this is a strong and useful benchmark. Concrete strengths: the scale (511,100 model-image pairs); the breadth (20 models, seven providers, four datasets); the public release of metadata and code (CC-BY-4.0/MIT) with deterministic reconstruction instructions; the disclosed audit protocols; and the sensitivity analyses, with MCRS rankings stable under weight/exponent variations (Kendall tau of at least 0.905) and component ablations (tau of at least 0.947), plus alpha and trap-radius sweeps. The five-condition paired design with a text-ablated Blank reference is methodologically sound for the degradation claim, and the continuous-coordinate output space is a genuine operational improvement over discrete VQA/classification tasks because it measures both magnitude and direction of text-induced shifts. The headline numbers are direct measurements rather than fitted quantities, so circularity is not a concern, and the released data allow independent re-testing of the paper's falsifiable predictions (for example, that every evaluated model degrades under adversarial text).

major comments (3)
  1. [Section 6; Eqs. (3)-(4); Tables 7 and 17] The stress-test concern is valid and lands on this exact section: the directed-shift claim is supported only by TFR and TDR computed for Adversarial predictions against the same group's geocoded trap, and no null control is reported. The Random condition shares the same visual substrate and editing pipeline, its predictions already exist for every model, and the paper itself describes it (Section 3; Figure 1) as the generic text sensitivity control, yet the distance from Random-variant predictions to the same 1,732 trap coordinates is never computed. This is not a cosmetic omission: the traps are by construction (Appendix A prompt; Section 4 geocoding) real place names that Nominatim resolves, often to well-known cities or landmarks, so population priors can place predictions near the trap without the text naming it; moreover, the paper's own TBS results (Section 6) show that Random text has a substantial generic effect (959 km mean error increase). TBS and WLA are computed relative to ground truth and cannot test directionality, so the missing Random-condition TFR and TDR against the same traps is the correct control. I request, on the same 1,732 geocodable groups: Random TFR (fraction of Random predictions within 50 km of the adversarial trap), the paired difference TDR_Random = D(blank, trap) - D(random, trap), and a permutation null that reassigns each group's trap to another group. If Adversarial TFR/TDR are not substantially and consistently larger than these baselines, the abstract's claim of directed shifts toward geographic targets introduced by conflicting text should be replaced by a weaker, baseline-quantified claim.
  2. [Section 6; Table 17] The claim that every evaluated model exhibits a positive mean paired Trap Distance Reduction is reported without uncertainty quantification, although Table 17 shows attraction rates as low as 44.5% (Claude-Sonnet-4.6) and median TDR at or near zero for several models (Gemini-2.5-Pro median -0.01 km; Claude-Sonnet-4.6 median 0.00 km), with the text itself attributing the positive means to concentrated large shifts. For a headline claim of this form, please report per-model bootstrap 95% confidence intervals or a paired Wilcoxon signed-rank test on TDR, and, once the Random baseline of the previous comment is available, a paired test of the Adversarial-versus-Random TDR contrast. The four qualitative examples in Table 19 illustrate large effects but cannot substitute for a distributional statement over the 1,732 geocodable groups.
  3. [Section 4 (Quality Assurance); Appendix B] The counterfactual premise that the five variants differ only in text semantics rests on a human audit of 120 edited images, about 0.6% of the 20,444 edited variants, sampled only from Similar, Random, and Adversarial (not Blank), with artifact severity, context damage, and naturalness aggregated across conditions (1.32 +/- 0.78, 1.14 +/- 0.52, 4.00 +/- 1.26) and readability reported globally (12.5% partially readable, none unreadable). As reported, one cannot verify that Adversarial edits, the condition carrying the headline claims, are not systematically more artifact-heavy or less legible than Similar or Random edits, which would confound TBS, TFR, and TDR independently of text meaning. Please report these statistics per condition (including Blank), give the readability breakdown per condition, and, if feasible, enlarge the audit with a focus on Adversarial variants.
minor comments (5)
  1. [Abstract; Table 7] The abstract's range '6.5-20.1% of adversarial predictions lie less than 50 km from the injected target' is the equal-dataset macro-average over four datasets whose per-dataset TFR values differ by more than an order of magnitude (for example, GoogleSV 1.66% for Gemini-3.1-Pro versus YFCC4K 35.68% for Qwen3-VL-235B); please state at the headline that the range is a macro-average and show the dataset spread, and add a Table 1 footnote noting that TFR and TDR are computed only on the 33.9% geocodable cohort of Table 6.
  2. [Section 6 (Semantic intervention effects)] The sentence stating that Similar replacements reduce error by 379 km on average, whereas Random and Adversarial replacements increase error by 959 km and 1,577 km, respectively, should state the aggregation convention (mean TBS across models and datasets) and the sign convention of Eq. (2) inline, since negative TBS means the edited prediction is closer to ground truth than the Blank prediction.
  3. [Section 4 (Adversarial target geocoding); Table 6] Because the injected target is defined as the top-ranked Nominatim result (Section 4), multi-reference place names can yield trap coordinates different from the intended injected place; please report how many of the 1,732 geocodable targets are ambiguous and confirm, as the qualitative examples in Table 19 suggest, that the intended and resolved targets coincide in the overwhelming majority of cases.
  4. [Appendix H (Table 18)] The trap-radius sensitivity sweep (Table 18) is reported only for the T3 subset of two representative models; given that TFR is a headline metric, either extend the sweep to additional models or justify why the T3 two-model analysis is representative of the all-tier macro-average reported in Table 7.
  5. [Table 2] In Table 2, the 'Removed' column counts all losses after OCR selection, including pre-generation screening and post-generation cleanup; a one-sentence clarification in the caption would help readers avoid misreading it as a peculiarity of the synthesis stage.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SIGNPOST-Bench results are direct paired measurements, not reductions to fitted parameters or self-citations.

full rationale

SIGNPOST-Bench's central claims are empirical measurements of model outputs under constructed counterfactual variants. The headline numbers (median error 282 to 1,347 km, TFR 6.5–20.1%, positive mean paired TDR per model) are computed directly from model predictions and geocoded targets via the paper's own definitions in Eqs. 1–4; no parameter is fitted to these outcomes and then renamed as a prediction. The MCRS weights, WLA decay constant, and 50 km trap radius are design choices, not fitted inputs, and the paper reports sensitivity analyses (Appendix D, Table 18) showing rankings are stable. The paper's use of the Blank variant as a paired baseline is definitional to TBS and TDR, not circular: the quantities being measured are the differences between model outputs under two image conditions. The skeptic's concern about the absence of a Random-variant TFR/TDR null is a validity gap regarding whether target-following is specifically semantic, but it does not make the reported Adversarial-vs-Blank measurements equivalent to their inputs by construction. Self-citations (e.g., REVERSE, Unified Urban Tuning, spatiotemporal contrastive learning) appear only in related work and are not load-bearing premises of the benchmark formulation or evaluation. No uniqueness theorem, ansatz-smuggling citation, or renaming of a known result was found. Accordingly, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are postulated. The counterfactual quintuplet, injected target, and diagnostic metrics operationalize existing scene-text, geolocation, and robustness concepts; they are measurement constructs rather than entities requiring independent evidence.

free parameters (3)
  • WLA decay constant alpha = 0.005
    Sets how quickly WLA decays with geodesic error; chosen so WLA is 0.5 at 138 km and below 0.01 at 1,000 km. Sensitivity analysis shows relative model ordering is stable under alpha values 0.002, 0.005, and 0.01.
  • Trap radius tau = 50 km
    Threshold for Trap-Fit Rate; the authors state it balances target specificity and sensitivity. Tier-controlled sensitivity from 10 to 500 km shows gradual TFR changes, but the absolute TFR numbers depend on this choice.
  • MCRS weights and exponents = C = 0.50*orig + 0.50*blank; R = 0.22*rho_rnd + 0.44*rho_adv + 0.17*qTBS + 0.17*qTFR; MCRS = 100 * C^0.40 * R^0.60
    Hand-chosen weights and exponent favor conflict handling. Kendall tau is at least 0.905 under tested variations, so reported rankings are robust, but absolute composite scores depend on these design choices.
assumptions (4)
  • domain assumption The synthetic edits are localized to selected scene-text spans and preserve surrounding non-textual content; Blank is a clean text-removal baseline.
    All paired diagnostics (TBS, TDR) and artifact controls rest on this. Only 120 edited images were audited; mean artifact severity is 1.32/5 and context damage is 1.14/5, with substantial variance in text naturalness (4.00 +/- 1.26).
  • domain assumption The top-ranked Nominatim geocoding result for each adversarial place name is the correct intended target coordinate.
    TFR and TDR use these coordinates as y_trap. Table 6 reports that only 33.9% of adversarial targets geocode successfully; geocoding errors would bias directed-shift estimates.
  • domain assumption Source images carry accurate ground-truth coordinates from IM2GPS3K, YFCC4K, GoogleSV, and BaiduSV.
    All WLA and TBS errors are measured against y_gt. Inaccurate labels would affect all conditions similarly but could still distort absolute error magnitudes.
  • domain assumption One accepted temperature-0 response per image is a reliable estimate of each model's behavior, and coordinate parsing succeeds for accepted responses.
    No repeated sampling or confidence intervals are reported; retries replace only failed, empty, or unparseable responses, as described in Appendix B.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/RER5YUPY

@misc{pith2026260804244,
  author       = {Pith},
  title        = {Pith review of: SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RER5YUPY}},
  note         = {Machine review of arXiv:2608.04244}
}
read the original abstract

Multimodal large language models (MLLMs) make grounded predictions in real-world scenes by combining visual and textual cues, yet existing benchmarks rarely reveal how they arbitrate between these evidence sources when they conflict. We introduce SIGNPOST-Bench, a controlled counterfactual benchmark for evaluating text-vision conflict resolution. Each source image is transformed into a counterfactual quintuplet of Original, Blank, Similar, Random, and Adversarial variants. Synthetic, localized scene-text interventions are designed to preserve non-textual content, enabling paired measurements of changes in localization performance and directed shifts toward geographic targets introduced by conflicting text. SIGNPOST-Bench contains 5,111 counterfactual groups and 25,555 image variants from four datasets. We evaluate 20 MLLMs from seven providers. Compared with Original images, Adversarial variants raise median localization error from 282 km to 1,347 km, a 4.8-fold increase. Among geocodable adversarial samples, 6.5-20.1% of predictions lie less than 50 km from the injected target across models, and every evaluated model exhibits a positive mean paired reduction in target distance from Blank to Adversarial. Compatible, unrelated, and conflicting text replacements produce distinct effects on model predictions, while clean-input localization performance does not fully predict robustness to conflicting text. These results establish visual geolocation as a continuous diagnostic of scene-text arbitration and provide a controlled framework for evaluating how MLLMs resolve conflicting multimodal evidence.

Figures

Figures reproduced from arXiv: 2608.04244 by the authors.

Figure 1
Figure 1. SIGNPOST-Bench overview. Left: a scene from Holland, Michigan, illustrates the central problem: native scene text may evoke the Netherlands, whereas the surrounding visual cues support a North American location. Top: synthetic, localized scene-text edits transform each source image into a counterfactual quintuplet of Original, Blank, Similar, Random, and Adversarial variants, providing a native-text reference, a tex… view at source ↗
Figure 2
Figure 2. SIGNPOST-Bench construction pipeline. Images from four sources are screened for editable scene-text spans. An MLLM generates three replacement types, and localized operations designed to preserve non-textual content produce the four edited images. Together with Original, they form the quintuplet within each counterfactual group. Sampled human audits assess taxonomy labels and edit quality. Models and Evaluation Setu… view at source ↗
Figure 3
Figure 3. Model-wise conflict profile. (a) Original-to-Adversarial WLA degradation; marker shape distinguishes the two [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Two-model probing and defense results: WLA (left) [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Prompt card for generating Similar, Random, and Adversarial scene-text replacements. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Prompt card for standard coordinate prediction. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Prompt card for structured probing [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Prompt card for conflict-aware defense prompting. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Cross-task generalization prompt for the scene-text consistency diagnostic. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 9
Figure 9. Figure 9: Cross-task generalization prompt (continued): country identification under text–vision conflict. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: MCRS component breakdown: Capability C vs. Conflict Robustness R for all 20 models, colored by provider. Probing and defense results [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Stratified analysis by scene-text coupling tier. Marker positions show WLA under Original, Blank, and Adversarial [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Representative samples from IM2GPS3K, YFCC4K, GoogleSV, and BaiduSV (top left to bottom right). [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Examples of T1 Portable, T2 Cultural, and T3 Geo-Specific scene text (left to right). [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: Per-dataset adversarial WLA across all 20 models, ordered by their four-dataset mean. Vulnerability magnitude varies [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Adversarial Text Bias Score across all 20 models, grouped by provider. [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: Original and adversarial GoogleSV images used in the probing–defense comparison. The edit replaces “BEKINS” [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: Prompt-conditioned evidence arbitration for the adversarial image in Figure 16. Structured probing trusts the injected [PITH_FULL_IMAGE:figures/full_fig_p025_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 10 canonical work pages

  1. [1]

    Priority: street/road signs > city/district names > store names with location cues > other text

    Identify the top 1--3 geo-informative texts in the image. Priority: street/road signs > city/district names > store names with location cues > other text. Return 1--3 entries. Skip illegible, tiny, or irrelevant text such as license plates and watermarks. If no legible text is found, return {"texts": []}

    work page

  2. [2]

    For each selected text, describe where it appears in the image

    work page

  3. [3]

    texts": [{

    For each selected text, generate three replacements: - similar: plausible in the same region/language and should not change the perceived location; - random: unrelated text in a different language/script, using diverse choices; - adversarial: a real place name from a different continent, using diverse targets. Rules: - Keep replacements short, approximate...

    work page

  4. [4]

    scene_region

    Give your final answer for the country or region. Output ONLY valid JSON: { "scene_region": "country/region inferred from visual cues only", "text_region": "country/region suggested by text, or 'none' if no text", "trusted_source": "Visual or Textual or Both", "final_region": "your final country/region answer", "reason": "brief explanation" } Figure 9: Cr...

    work page 2016

  5. [6]

    Describe the main visual cues in the scene (architecture, vegetation, terrain, road layout, traffic conventions)

    work page

  6. [7]

    Describe what the visible text suggests, if any (signs, storefronts, addresses, license plates)

    work page

  7. [8]

    visual_cues

    Decide whether the text is consistent with the visual scene. Output ONLY valid JSON: { "visual_cues": "brief description of visual evidence", "text_cues": "brief description of text and what location it suggests, or 'none' if no readable text", "consistency": "Consistent or Conflict", "reason": "brief explanation of why consistent or conflicting" } Figure...

    work page

  8. [9]

    Infer the most likely country or broad region based on the visual scene only (architecture, vegetation, terrain, road layout, traffic conventions, vehicle types)

    work page

Show all 12 references
  1. [10]

    Note any visible text (signs, storefronts, license plates) and what location it suggests

  2. [11]

    If the scene and text disagree, explain which source is more reliable and why

  3. [196]

    ComfyUIContributors.2023

    Springer. ComfyUIContributors.2023. ComfyUI. https://github.com/ Comfy-Org/ComfyUI. Software repository; accessed 2026- 07-29. Cui, X.; Aparcedo, A.; Jang, Y. K.; and Lim, S.-N. 2024. On the Robustness of Large Multimodal Models Against ImageAdversarialAttacks. InProceedingsof...

    2023 arXiv

  4. [2022]

    In Advances in Neural Information Processing Systems, vol- ume 35, 17612–17625

    Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning. In Advances in Neural Information Processing Systems, vol- ume 35, 17612–17625. Liu, Y.; Duan, H.; Zhang, Y.; et al. 2024. MMBench: Is Your Multi-Modal Model an All-Around Player? ...

    2024 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.