GETReason: Enhancing Image Context Extraction through Hierarchical Multi-Agent Reasoning

Abhinav Rawat; Chitta Baral; Shikhhar Siingh; Vivek Gupta

REVIEW 4 major objections 5 minor 38 references

GETReason: Enhancing Image Context Extraction through Hierarchical Multi-Agent Reasoning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GETReason tries to show that a hierarchical multi-agent pipeline—scene graph, tailored prompts, and event/temporal/geospatial agents with cross-validation—extracts more accurate event context from public event images than monolithic…

desk verdict The framework and metric are real, but Tables 2 and 3 contradict each other on the headline event score, so the main result needs a fix before it is citable. read the letter →

arxiv 2505.21863 v3 pith:D36FICA2 submitted 2025-05-28 cs.CV cs.CL

Shikhhar Siingh , Abhinav Rawat , Chitta Baral , Vivek Gupta This is my paper

classification cs.CVcs.CL

keywords multi-agentreasoningvision-languagemodelsevent-centricimageunderstandinggeospatialtemporalcontextextractionGREATmetricpubliceventimages

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Public event images carry context that ordinary captions miss: where an event happened, when, and why it matters. This paper tries to establish that a structured multi-agent pipeline extracts that context more reliably than asking one vision-language model to do everything in a single prompt. GETReason builds a scene graph, writes role-specific prompts, and runs event, temporal, and geospatial agents whose outputs are cross-fed for a second pass. The authors introduce GREAT, a metric that rewards semantically similar event descriptions, geographically close locations, and temporally close dates, and report that on TARA their framework reaches a total of 60.4 against 53.3 for the strongest baseline, with the largest gain in event understanding. If the claim holds, newsrooms, archivists, and educators could turn raw event photos into structured, checkable event records.

What carries the argument

The load-bearing object is GETReason's three-layer pipeline: a scene-graph generation layer that extracts entities, attributes, and relationships and then infers the scene's broader abstract idea; a prompt-generation layer that writes role-specific prompts from that graph; and a multi-agentic extraction layer with three specialized agents—Event, Temporal, and Geospatial—each producing structured JSON with values and reasoning, followed by a cross-extraction pass that feeds every agent the other agents' outputs. The evaluation mechanism is GREAT, which scores event output by cosine similarity between concatenated event-plus-background embeddings, geospatial output by Haversine great-circle distance from the ground-truth coordinates, and temporal output by a granularity-weighted closeness score with tolerance thresholds for decade, year, month, and day.

What would settle it

Re-score a random sample of TARA images using event and background labels written by independent human annotators (or by a different model family), then recompute GREAT; if GETReason's event-score lead over zero-shot chain-of-thought shrinks to near zero, the reported gains reflect agreement with the Gemini-generated pseudo-labels rather than genuine event understanding.

Watch

Extended reading notes

Core claim

The paper's central claim is that event-centric image understanding reduces to producing a structured triplet—location, time, and event narrative—and that the best way to produce it is to decompose the reasoning across specialized agents rather than asking a single VLM for one answer. GETReason chains a scene-graph agent and an abstract agent, a prompt agent that generates targeted instructions, and three extraction agents (Event, Temporal, Geospatial) that output JSON with values plus reasoning; a cross-extraction pass then feeds each agent the other two agents' outputs to reduce hallucinations and refine guesses. Under the GREAT metric on TARA with Gemini 1.5 Pro-002, the framework posts a total of 60.4 percent versus 53.3 for the strongest baseline, led by an event score of 70.3, and the paper reads the results as showing consistent gains across reasoning types, with the caveat that Good Guesser remains competitive on geospatial reasoning in several comparisons.

Load-bearing premise

The load-bearing premise is that the augmented TARA* event and background labels, written by Gemini 1.5 Pro from the image plus its NYT article, are trustworthy ground truth; because the same model family is the main evaluation subject, the event score may reward agreement with a machine-generated label rather than true event understanding.

Editorial extensions

If this is right

If correct, GETReason gives a template for turning a single event photo into a machine-readable JSON record of place, date, and event narrative, with per-claim reasoning attached.
The GREAT metric, with its cosine event score, Haversine geospatial score, and weighted temporal tolerance, gives future systems a target that rewards approximate correctness instead of exact string matching.
The ablation that removes the image from later agents drops geospatial performance sharply on TARA, indicating that visual grounding rather than world knowledge alone is doing the work.
Removing the abstract generator, prompt generator, or scene-graph agent one at a time changes the total by less than a point, suggesting the pipeline can tolerate component failures without collapsing.
On WikiTiLo, where no event labels exist, the same framework still leads on combined geospatial-temporal scores, extending the approach to images without article metadata.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the TARA* event labels were generated by Gemini 1.5 Pro from image-plus-article input and the same model family is the strongest evaluated system, the reported event margin may partly measure agreement with the labeler's writing style rather than independent event identification; re-annotating a sample with human experts would test this.
The same event-time-place triplet is exactly the meta-context needed to flag out-of-context images in misinformation settings, where the claim accompanying a photo can be checked against what the photo actually shows.
The per-agent JSON structure makes the system auditable in a way monolithic captions are not: a downstream user could read the reasoning strings to see why the model placed an event in Jakarta on October 20, 2019, and could correct a single agent without rerunning the whole pipeline.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick read of GETReason (arXiv:2505.21863). The framework is real engineering, the metric is thoughtfully designed, and the appendix is unusually thorough — but the paper's own tables contradict each other on the headline event score, and until that is resolved the central claim does not stand.

What's new: a three-layer multi-agent pipeline (scene graph → prompt generation → specialized extraction agents with a cross-extraction pass), the GREAT metric (cosine similarity for events, Haversine distance with a Dmax cap for location, tolerance-weighted granularity for time), and an augmented TARA* with event/background labels. The Appendix B working example runs the whole pipeline end-to-end and is genuinely useful. Evaluation spans three model families and nine baselines, and the authors are honest about limitations. The citation pattern is fine — they engage the relevant prior work.

The problem: Table 2 reports GETReason on TARA with Gemini 1.5 Pro-002 at Event 70.3 / Total 60.4. Table 3, labeled the same configuration, reports Event 37.7 / Total 47.3. Geo and Temp are identical in both tables, so it is not an arithmetic issue — the published GREAT formula reproduces both totals. If 37.7 is the real event score, GETReason falls below COTzeroshot, Direct Extraction, and Partial Cross Extraction, and the claimed 7.1-point gain inverts. The stress-test note is correct, and this is load-bearing.

Secondary concerns, in proportion. The TARA* event labels are generated by Gemini 1.5 Pro from image plus article, and the main evaluation runs Gemini 1.5 Pro; the event score is partly agreement with a same-family pseudo-label. The paper's own Limitations admit many TARA images are unrelated to their articles, which makes this worse. Human validation of a sample, or a different label model, would fix most of it. The deduction analysis is more layered still: VLM-generated reasoning evaluated against VLM-generated reasoning. Also, all results are single runs without error bars, and the metric weights are hand-chosen with no sensitivity analysis. That is minor relative to the table conflict, but worth a page.

Where I disagree with the reader: this is not weakly supported across the board. The design is coherent, the ablations are sensible, and the authors openly acknowledge Good Guesser beating them on geospatial — despite one loose sentence claiming consistent superiority across all reasoning types.

Who this is for: people working on multimodal reasoning, event-centric image understanding, or VLM evaluation metrics. It deserves a serious referee, but with required major revision: reconcile the tables, validate the labels, and report variance. I would not cite the headline numbers as they stand.

Referee Report

4 major / 5 minor

Summary. The paper proposes GETReason, a hierarchical multi-agent framework that decomposes event-centric image understanding into scene-graph generation, abstract-concept inference, prompt generation, and specialized event, temporal, and geospatial agents with a direct-then-cross extraction strategy. It also introduces the GREAT metric, which combines semantic event similarity, Haversine-based geospatial similarity, and weighted temporal granularity scoring, and it augments the TARA and WikiTiLo datasets with structured labels. The central empirical claim is that GETReason outperforms monolithic and baseline prompting approaches on event, temporal, and geospatial extraction across Gemini 1.5 Pro-002, GPT-4o mini, and Qwen2.5-VL-7B, with a headline total GREAT score of 60.4 on TARA for Gemini 1.5 Pro-002.

Significance. If the reported results are reproducible, GETReason would be a useful demonstration that structured, role-specialized multi-agent reasoning can improve contextual image understanding over monolithic VLM prompting, and GREAT would provide a more graded alternative to exact-match evaluation for this task. The paper's strengths include a clean task decomposition, evaluation on two datasets and three model families, release of code and data, and an unusually candid Limitations section. However, the empirical support is currently undermined by internal numerical inconsistencies and by an evaluation design in which the main ground-truth labels are generated by the same model family used for evaluation; these issues must be resolved before the central claim can be accepted.

major comments (4)

[§6, Tables 2, 3, and 5] Tables 2 and 3 report the same configuration—GETReason on TARA with Gemini 1.5 Pro-002—with contradictory event scores: Table 2 gives Event 70.3 and Total 60.4, while Table 3 gives Event 37.7 and Total 47.3. The GREAT formula in Section 5 (Total = 0.4*Event + 0.3*Geo + 0.3*Temp) reproduces both totals exactly, so the discrepancy is in the event score rather than in the arithmetic. Table 5 compounds the problem: removing the abstract generator, the prompt generator, or the scene-graph agent yields Event scores of 71.3, 71.0, and 71.2, each higher than the full system's Event 37.7 in Table 3. If the Table 3 event value is the correct one, GETReason's total of 47.3 falls below COTzeroshot (53.3), Direct Extraction (57.6), and Partial Cross Extraction (59.3), reversing the main claim. The authors must clarify which numbers are correct and provide raw outputs or logs; the headline result is not currently reproducible from the paper's own tables.
[§3.1, §6, Limitations] The TARA* ground-truth event and background labels are generated by Gemini 1.5 Pro from the image together with its NYT article (Section 3.1), and the main evaluation is run with Gemini 1.5 Pro-002 (Section 6). The paper's own Limitations section states that many TARA images are unrelated to their corresponding articles. As a result, the GREAT event score may measure agreement between the model and pseudo-labels produced by the same model family from information not available at inference time, rather than image-grounded understanding. I request a human-verified subset of TARA* or independently produced labels, and separate reporting of scores on images verified to match their articles, before the event-level comparison can be trusted.
[§6, Tables 2–7] All scores in Tables 2–7 come from single runs with no confidence intervals, error bars, or significance tests. This matters because the margin over the strongest baseline in Table 2 is 2.6 points (60.4 vs. Good Guesser 57.8), not a large margin relative to typical VLM run-to-run variation. The text's assertion that GETReason 'consistently achieves the best performance for all reasoning types' is also internally contradicted by Table 2, where Good Guesser's geospatial score (76.1) exceeds GETReason's (69.4), a point the authors acknowledge in Section 6.1. Please report multiple runs or seeds with error bars and qualify the 'best on all reasoning types' claim accordingly.
[§5] The GREAT metric contains several hand-set constants—D_max = 1000 km; T_decade = 50, T_year = 5, T_month = 6, T_day = 15; unit weights w_year = 1.25, w_month = 1.5, w_day = 1.5; and overall weights 0.4/0.3/0.3—but no sensitivity analysis is reported. Because several of the reported differences between methods are only a few points, the ranking may depend on these choices; at minimum, a robustness check over reasonable alternative values is needed to establish that the comparative results are not artifacts of the chosen constants.

minor comments (5)

[Table 6] For QwenVL2.5-7B on WikiTiLo, the Cross Extraction row reports Geo 33.4, Temp 27.9, and Total 22.3; with the stated 0.5/0.5 weighting the total should be 30.65. Please correct the arithmetic.
[Table 3 caption] The caption says 'Gemini 1.5 Pro-002 on TARA' but the table also contains WikiTiLo rows; rename the caption or split the table.
[Appendix A and Table 2] Some prompt excerpts contain corrupted token formatting, such as 'St at e_o r_ Pr ov in ce' and '\textunderscore idea', and the Good Guesser row in Table 2 has fused numbers ('Good Guesser76.131...'); these should be cleaned before publication.
[Table 7] The table header writes 'GPT-40 mini' instead of 'GPT-4o mini'.
[§6, Deduction Analysis] The deduction scores 81.4%, 76.9%, and 70.2% are introduced without defining the subset of data or the exact comparison protocol; please clarify how these were computed.

Circularity Check

2 steps flagged · score 4.0 of 10

Event ground truth is Gemini-generated while Gemini is the scored model, making the Event component of GREAT a self-consistency check; Geo/Temp and the framework itself are not circular, and the Table 2/3 conflict is a separate soundness issue.

other [Section 3.1 'TARA*: TARA Enhancement' and Section 5 'Event Evaluation' / Section 6.1 'Results and Analysis']
"The image, along with its associated news article, was provided to the VLM to simultaneously generate the primary event, the secondary event serving as background context, and the corresponding reasoning for how these events are grounded in the image. The outputs from these two phases were combined to form the final ground truth labels. ... ESi = CS((ei + bi),(Ei + Bi)) + 1/2 ... we compared the reasoning output generated by our model to the ground truth across three modalities on the TARA dataset on Gemini 1.5 pro-002, utilizing the Event Evaluation metric from proposed GREAT metric."

The gold event/background labels E_i, B_i for TARA* are produced by Gemini 1.5 Pro (Section 3.1), and the model whose output is scored in Table 2 is Gemini 1.5 Pro-002. GREAT's Event score is the cosine similarity between the predicted and gold concatenated event-background embeddings. Therefore the reported Event score (e.g., 70.3 in Table 2) measures how well Gemini's predictions agree with Gemini's own generated labels for the same images, not how well the framework recovers externally verified event context. The paper's own Limitations state that many TARA images are unrelated to their articles, so the article-derived pseudo-labels are not a trustworthy independent standard.
other [Section 3.1 'Deduction Augmentation' and Section 6.1 'Deduction Analysis']
"we used the augmented ground truths as references for the images on the VLM to generate the deduction inferences and reasoning from the images for all the information present in the augmented dataset. ... we compared its output to ground truth across three modalities on the TARA dataset using Gemini 1.5 Pro-002 and the Event Evaluation metric from GREAT. GETREASON achieves scores of 81.4%, 76.9%, and 70.2% for geospatial, temporal, and event deduction, respectively."

The gold deduction/reasoning strings are created by Gemini 1.5 Pro from the image plus the augmented ground truth, and the framework's deduction outputs are also produced by Gemini 1.5 Pro-002. The GREAT Event metric then scores the cosine similarity between these two Gemini-generated texts, so the deduction scores quantify self-agreement within the same model family rather than agreement with an externally verified human reasoning chain. Because the augmented ground truth itself is also Gemini-generated, the loop is closed without an external anchor. This is a secondary analysis, not the headline Table 2 result, but it is still a self-referential evaluation step.

full rationale

The framework derivation itself is not circular: GETReason is a layered prompting pipeline, and its claimed improvements are an empirical claim rather than a theorem derived from its own assumptions. No load-bearing self-citation chain or imported uniqueness argument appears; the multi-agent design is motivated by external prior work. The GREAT metric is a defined scoring rule and is not presented as a prediction derived from the framework. The main circularity is in the evaluation protocol: TARA* event/background labels are generated by Gemini 1.5 Pro from image-plus-article metadata (Section 3.1), and the model scored in Table 2 is Gemini 1.5 Pro-002, with the Event score being cosine similarity between the model output and those same-model labels. That makes the Event component (weight 0.4) a self-consistency check rather than a correctness measure, and the paper's own Limitations acknowledge that many TARA images are unrelated to their articles, compounding the concern. Geo and Temporal components still use the original TARA/WikiTiLo annotations, so the overall conclusion is only partially affected by this circularity. Separately, a non-circular soundness problem exists: Table 3 reports Event=37.7 and Total=47.3 for the same GETReason plus Gemini 1.5 Pro-002 configuration that Table 2 reports as Event=70.3 and Total=60.4; this is a reproducibility conflict, not a circularity, and is not counted in the circularity score. Overall circularity score 4.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central evaluation relies on VLM-generated pseudo-labels for the event component, and the GREAT metric contains several hand-chosen weights and tolerances that directly affect all reported scores. These parameters and assumptions are not validated against human judgment or tested for sensitivity, which limits the strength of the empirical claims.

free parameters (7)

Event weight in GREAT overall score (TARA) = 0.4
Hand-chosen to prioritize the event component; affects final ranking of methods.
Geospatial and temporal weights (TARA) = 0.3 each
Hand-chosen; no sensitivity analysis.
Geospatial/temporal weights (WikiTiLo) = 0.5 each
No event score in WikiTiLo, so weights split equally.
Temporal tolerance thresholds = T_decade=50, T_year=5, T_month=6, T_day=15
Chosen to reflect 'expected temporal variance' without justification or sensitivity analysis.
Temporal unit weights = w_century=1, w_decade=1, w_year=1.25, w_month=1.5, w_day=1.5
Hand-chosen to penalize finer granularities more heavily.
Geospatial maximum distance Dmax = 1000 km
Default threshold; no sensitivity analysis.
Rekognition confidence threshold = 99%
Threshold for including public figures in prompts; affects all GPT experiments and possibly others.

assumptions (5)

domain assumption Gemini 1.5 Pro-generated event and background labels are valid ground truth for event-centric image understanding.
Section 3.1 builds TARA* labels by prompting Gemini 1.5 Pro with the image and NYT article; the paper does not validate these labels against human annotations.
domain assumption Semantic similarity from SBERT all-mpnet-base-v2 captures the correctness of event descriptions.
Used in the GREAT event score (Section 5); no evidence presented that embedding similarity aligns with human judgment for this task.
domain assumption The hand-set temporal tolerances and weights reflect the difficulty of temporal reasoning.
Section 5 defines T_u and w_u without empirical grounding.
domain assumption Original TARA and WikiTiLo spatio-temporal annotations are accurate enough for evaluation.
TARA metadata is auto-extracted from NYT articles; the paper's Limitations concede many images are unrelated to their articles.
domain assumption Including AWS Rekognition-identified public figures in prompts is a fair component of the evaluated pipeline.
Section 6 describes injecting celebrity names into prompts for GPT-4o-mini; this adds external knowledge that baselines may not receive consistently.

how reviews work

0 comments

Cite this review

Pith. "Pith review of GETReason: Enhancing Image Context Extraction through Hierarchical Multi-Agent Reasoning." pith.science (2026). https://pith.science/paper/D36FICA2

@misc{pith2026250521863,
  author       = {Pith},
  title        = {Pith review of: GETReason: Enhancing Image Context Extraction through Hierarchical Multi-Agent Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D36FICA2}},
  note         = {Machine review of arXiv:2505.21863}
}

read the original abstract

Publicly significant images from events hold valuable contextual information, crucial for journalism and education. However, existing methods often struggle to extract this relevance accurately. To address this, we introduce GETReason (Geospatial Event Temporal Reasoning), a framework that moves beyond surface-level image descriptions to infer deeper contextual meaning. We propose that extracting global event, temporal, and geospatial information enhances understanding of an image's significance. Additionally, we introduce GREAT (Geospatial Reasoning and Event Accuracy with Temporal Alignment), a new metric for evaluating reasoning-based image understanding. Our layered multi-agent approach, assessed using a reasoning-weighted metric, demonstrates that meaningful insights can be inferred, effectively linking images to their broader event context.

Figures

Figures reproduced from arXiv: 2505.21863 by the authors.

**Figure 1.** Example from TARA Fu et al. (2022) dataset. Given an image, extract location and time. We add the inferred event. and encoder-decoder architectures describe visible content—objects, people, and actions—but fail to infer deeper meaning. Even advanced visual-language models (Li et al., 2022; Hu et al., 2023) tend to narrate what is seen, missing why it matters. For instance, given an image of President Joko Widodo’s… view at source ↗

**Figure 2.** GETReason architecture agents that generate outputs based on specific prompts, working collaboratively to produce comprehensive and contextually rich captions. 4.1 Scene Graph Generation The Scene Graph Generation serves as the primary process of our framework, responsible for extracting and structuring information from the input image. It comprises two main modules: Scene Graph Agent. This module identifies ent… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 19 canonical work pages

[1]

Hammad Ayyubi, Xuande Feng, Junzhang Liu, Xudong Lin, Zhecan Wang, and Shih-Fu Chang. 2025. Puzzlegpt: Emulating human puzzle-solving ability for time and location prediction. arXiv preprint arXiv:2501.14210

work page Pith review arXiv 2025
[2]

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966

arXiv 2023
[3]

Satanjeev Banerjee and Alon Lavie. 2005. https://aclanthology.org/W05-0909 Meteor: An automatic metric for mt evaluation with improved correlation with human judgments . In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65--72

work page 2005
[4]

Boyi Deng, Wenjie Wang, Fengbin Zhu, Qifan Wang, and Fuli Feng. 2024. Cram: Credibility-aware attention modification in llms for combating misinformation in rag. arXiv preprint arXiv:2406.11497

arXiv 2024
[5]

ND Dinh and TK Chan. 2025. https://www.medrxiv.org/content/medrxiv/early/2025/01/07/2025.01.01.25319863.full.pdf Entagents: Ai agents for complex knowledge otolaryngology . medRxiv

work page 2025
[6]

Hongyuan Dong, Jiawen Li, Bohong Wu, Jiacong Wang, Yuan Zhang, and Haoyuan Guo. 2024. Benchmarking and improving detail image caption. arXiv preprint arXiv:2405.19092

arXiv 2024
[7]

Xingyu Fu, Sheng Zhang, Gukyeong Kwon, Pramuditha Perera, Henghui Zhu, Yuhao Zhang, Alexander Hanbo Li, William Yang Wang, Zhiguo Wang, Vittorio Castelli, et al. 2023. Generate then select: Open-ended visual question answering guided by world knowledge. arXiv preprint arXiv:2305.18842

arXiv 2023
[8]

Xingyu Fu, Ben Zhou, Ishaan Preetam Chandratreya, Carl Vondrick, and Dan Roth. 2022. https://cogcomp.seas.upenn.edu/papers/paper-to-come.pdf There’s a Time and Place for Reasoning Beyond the Image . In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL)

work page 2022

Show all 38 references

[9]

Gao and Z

D. Gao and Z. Zhou. 2024. https://dl.acm.org/doi/abs/10.1145/3689638 Qr-clip: Introducing explicit knowledge for location and time reasoning . ACM Transactions on Multimedia Computing, Communications, and Applications

2024 doi
[10]

A Ghafarollahi and MJ Buehler. 2024. https://advanced.onlinelibrary.wiley.com/doi/pdfdirect/10.1002/adma.202413523 Sciagents: Automating scientific discovery through bioinspired multi-agent intelligent graph reasoning . Advanced Materials

2024 doi
[11]

19th Century

John William Hamilton. 19th Century. https://en.wikipedia.org/wiki/Haversine_formula Haversine formula for distance calculation . Based on Hamilton's work on spherical trigonometry
[12]

Jack Hessel, Jena D Hwang, Jae Sung Park, Rowan Zellers, Chandra Bhagavatula, Anna Rohrbach, Kate Saenko, and Yejin Choi. 2022. The abduction of sherlock holmes: A dataset for visual abductive reasoning. In European Conference on Computer Vision, pages 558--575. Springer

2022
[13]

Jia Cheng Hu, Roberto Cavicchioli, and Alessandro Capotondi. 2023. Exploiting multiple sequence lengths in fast end to end training for image captioning. In 2023 IEEE International Conference on Big Data (BigData), pages 2173--2182. IEEE

2023
[14]

Douglas M. Jennewein, Johnathan Lee, Chris Kurtz, Will Dizon, Ian Shaeffer, Alan Chapman, Alejandro Chiquete, Josh Burks, Amber Carlson, Natalie Mason, Arhat Kobwala, Thirugnanam Jagadeesan, Praful Barghav, Torey Battelle, Rebecca Belshe, Debra McCaffrey, Marisa Brazil, Chaita...

2023
[15]

u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Proc...

2020
[16]

Chenliang Li, Haiyang Xu, Junfeng Tian, Wei Wang, Ming Yan, Bin Bi, Jiabo Ye, Hehong Chen, Guohai Xu, Zheng Cao, et al. 2022. mplug: Effective and efficient vision-language learning by cross-modal skip-connections. arXiv preprint arXiv:2205.12005

2022 arXiv
[17]

Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013 Rouge: A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81

2004
[18]

H. Liu, X. Zhou, and T. Wang. 2023. https://ieeexplore.ieee.org/document/10123456 Edis: Entity-driven image search for cross-modal retrieval . IEEE Transactions on Multimedia, 25(3):482--496

2023
[19]

JY Ng, SY Liew, CH Chng, and TW Teo. 2024. https://ieeexplore.ieee.org/abstract/document/10730162/ The discovery and solution of the pseudo-reasoning issue for constructing cost-effective multi-agent frameworks in large language models . In Proceedings of the IEEE Conference o...

2024
[20]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://aclanthology.org/P02-1040 Bleu: A method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pages 311--318

2002
[21]

David M. W. Powers. 2011. https://arxiv.org/abs/2010.16061 Evaluation: From precision, recall and f-measure to roc, informedness, markedness & correlation . Journal of Machine Learning Technologies, 2(1):37–63

2011 arXiv
[22]

Arnau Ramisa, Fei Yan, Francesc Moreno-Noguer, and Krystian Mikolajczyk. 2017. https://doi.org/10.18653/v1/W17-2005 The B reaking N ews dataset . In Proceedings of the Sixth Workshop on Vision and Language, pages 38--39, Valencia, Spain. Association for Computational Linguistics

2017 doi
[23]

Singh, K

A. Singh, K. Patel, and R. Sharma. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/6b5f6e6a4f94c2f3b1c5b1e6c2a2b5c7-Paper.pdf Viphy: Evaluating vision-language models on physical commonsense knowledge . In Proceedings of NeurIPS

2023
[24]

X. Song, M. Wu, K. Zhu, and C. Zhang. 2024. https://arxiv.org/abs/2402.18409 A cognitive evaluation benchmark of image reasoning and description for large vision language models . arXiv preprint arXiv:2402.18409

2024 arXiv
[25]

DeepSeek AI Research Team. 2025 a . https://www.deepseek.ai/research/r1 Deepseek-r1: Advancements in reinforcement learning for language models

2025
[26]

Qwen Team. 2025 b . https://qwenlm.github.io/blog/qwen2.5-vl/ Qwen2.5-vl

2025
[27]

image, tell me your story!

Jonathan Tonglet, Marie-Francine Moens, and Iryna Gurevych. 2024. " image, tell me your story!" predicting the original meta-context of visual misinformation. arXiv preprint arXiv:2408.09939

2024 arXiv
[28]

Jonathan Tonglet, Gabriel Thiem, and Iryna Gurevych. 2025. Cove: Context and veracity prediction for out-of-context images. arXiv preprint arXiv:2502.01194

2025 arXiv
[29]

Lawrence Zitnick, and Devi Parikh

Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. https://arxiv.org/abs/1411.5726 Cider: Consensus-based image description evaluation . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4566--4575

2015 arXiv
[30]

HW Wang, L Birnbaum, and V Setlur. 2025. https://arxiv.org/pdf/2501.16661 Jupybara: Operationalizing a design space for actionable data analysis and storytelling with llms . arXiv preprint arXiv:2501.16661

2025 arXiv
[31]

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. Qwen2-vl: Enhancing vision-language mode...

2024 arXiv
[32]

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

2022
[33]

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Morgan Funtowicz, Nate O'Rourke, Jakob Uszkoreit, et al. 2019. https://arxiv.org/abs/1910.03771 Huggingface's transformers: State-of-the-art natural language processing . ...

2019 arXiv
[34]

Wei Xu, Shujie Liu, Minlie Huang, Si Wei, and Hang Li. 2018. https://arxiv.org/abs/1803.02577 A survey on hallucination in neural machine translation . arXiv preprint arXiv:1803.02577. Accessed: 2025-02-14

2018 arXiv
[35]

Linda Zeng, Rithwik Gupta, Divij Motwani, Diji Yang, and Yi Zhang. 2025. Worse than zero-shot? a fact-checking dataset for evaluating the robustness of rag against misleading retrievals. arXiv preprint arXiv:2502.16101

2025
[36]

Hao Zhang, Qi Liu, Jun Zhang, Ying Huang, and Liang Xu. 2024. Goodguesser: A benchmark for assessing spatio-temporal reasoning in vision-language models. arXiv preprint arXiv:2404.07312

2024 arXiv
[37]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[38]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Hammad Ayyubi, Xuande Feng, Junzhang Liu, Xudong Lin, Zhecan Wang, and Shih-Fu Chang. 2025. Puzzlegpt: Emulating human puzzle-solving ability for time and location prediction. arXiv preprint arXiv:2501.14210

work page Pith review arXiv 2025

[2] [2]

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966

arXiv 2023

[3] [3]

Satanjeev Banerjee and Alon Lavie. 2005. https://aclanthology.org/W05-0909 Meteor: An automatic metric for mt evaluation with improved correlation with human judgments . In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65--72

work page 2005

[4] [4]

Boyi Deng, Wenjie Wang, Fengbin Zhu, Qifan Wang, and Fuli Feng. 2024. Cram: Credibility-aware attention modification in llms for combating misinformation in rag. arXiv preprint arXiv:2406.11497

arXiv 2024

[5] [5]

ND Dinh and TK Chan. 2025. https://www.medrxiv.org/content/medrxiv/early/2025/01/07/2025.01.01.25319863.full.pdf Entagents: Ai agents for complex knowledge otolaryngology . medRxiv

work page 2025

[6] [6]

Hongyuan Dong, Jiawen Li, Bohong Wu, Jiacong Wang, Yuan Zhang, and Haoyuan Guo. 2024. Benchmarking and improving detail image caption. arXiv preprint arXiv:2405.19092

arXiv 2024

[7] [7]

Xingyu Fu, Sheng Zhang, Gukyeong Kwon, Pramuditha Perera, Henghui Zhu, Yuhao Zhang, Alexander Hanbo Li, William Yang Wang, Zhiguo Wang, Vittorio Castelli, et al. 2023. Generate then select: Open-ended visual question answering guided by world knowledge. arXiv preprint arXiv:2305.18842

arXiv 2023

[8] [8]

Xingyu Fu, Ben Zhou, Ishaan Preetam Chandratreya, Carl Vondrick, and Dan Roth. 2022. https://cogcomp.seas.upenn.edu/papers/paper-to-come.pdf There’s a Time and Place for Reasoning Beyond the Image . In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL)

work page 2022