Pith. sign in

REVIEW 3 major objections 5 minor 40 references

Question Answering for Diagram-Rich Technical Meeting Videos

T0 review · 3 major / 5 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM system that indexes speech and software diagrams from technical meeting videos once can answer open-ended questions with much higher accuracy and lower cost than long-video baselines.

desk verdict Real industrial multimodal QA system with large measured gains; the diagram-aware causal story is oversold relative to the end-to-end evidence. read the letter →

arxiv 2607.10494 v1 pith:M45HTYBD submitted 2026-07-11 cs.SE

Zhuoran Xu , Jia Li , Dayuan Tan , Mark Cole , Ish Ashraf , Sandeep Puri , Mehrdad Sabetzadeh , Shiva Nejati This is my paper
classification cs.SE
keywords multimodalquestionansweringtechnicalmeetingvideossoftwarediagramsUMLlargelanguagemodelsretrieval-augmentedgenerationrequirementsanddesignvideoindexing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Software engineers often need requirements, design decisions, and rationale buried in long recorded meetings that mix speech with slides and technical diagrams. This paper presents LMVQA, a two-stage multimodal question-answering system that processes each video once into a reusable, time-stamped evidence corpus of audio transcripts and diagram-aware visual descriptions, then answers questions by retrieving that evidence. On an industrial Ciena dataset and a public software-engineering lecture dataset, LMVQA raises answer accuracy from 31% to 94% and from 21% to 88% versus an adapted state-of-the-art long-video baseline, with larger gains on diagram-rich content. After indexing, average per-question response time falls to a few seconds and token-based LLM API cost drops by about 75%. Engineers who tried the system valued locating software-relevant details and tracing answers back to specific video segments without rewatching full recordings.

What carries the argument

LMVQA: a two-stage pipeline that builds a reusable time-stamped audio-visual evidence corpus with diagram-type-aware extraction, then answers queries by retrieving top chunks and generating answers that must stay grounded in that evidence and return timestamps for verification.

What would settle it

Run the same open-ended questions with an ablated LMVQA that keeps offline indexing and retrieval but replaces diagram-type-aware extraction with generic frame captions; if accuracy on diagram-related questions stays near the full system, the claim that diagram-aware processing drives the gains would not hold.

Watch

Extended reading notes

Core claim

LMVQA shows that open-ended question answering over long, diagram-rich technical meeting videos becomes accurate and practical when each video is converted once into a reusable multimodal evidence corpus that detects diagram frames, classifies software-diagram types, extracts category-specific semantics, and indexes them with time-stamped speech for retrieval-grounded generation.

Load-bearing premise

The large accuracy gain is taken as evidence that diagram-type-aware extraction works, even though the baseline was not built for open-ended technical-meeting questions and the study does not isolate diagram handling from offline indexing or model choice.

Editorial extensions

If this is right

  • Engineers can query archived technical meetings for requirements and design rationale without rewatching full recordings.
  • Diagram-heavy software engineering videos benefit more from type-aware visual extraction than from generic long-video QA.
  • One-time offline indexing makes interactive multimodal video QA practical when the same recordings are queried repeatedly.
  • Time-stamped multimodal evidence tracing can reduce reliance on ungrounded general-purpose tools for institutional technical memory.
  • Organizations that produce large volumes of technical meeting video can treat those archives as searchable requirements and design knowledge bases.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Similar diagram-aware indexing may transfer to other engineering domains whose meetings feature schematics, circuit diagrams, or process models.
  • The larger gains on diagram-rich content imply that software tooling should treat UML and architecture diagrams as first-class evidence modalities, not generic images.
  • Without component ablations, the field still needs a controlled test separating diagram classification from offline indexing and model choice before attributing most of the accuracy jump to any single piece.
  • Public software-engineering lecture videos could become a reusable open benchmark for multimodal knowledge recovery if this evaluation protocol is repeated.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents LMVQA, an industrial LLM-based multimodal QA system for long technical meeting videos that are rich in software-engineering diagrams. It processes each video once into a reusable time-stamped audio–visual evidence corpus, with SSIM keyframing, lightweight diagram detection/classification, and category-specific extraction for UML and related diagrams, then answers open-ended questions via top-k retrieval and grounded generation. On a Ciena industrial set (5 videos, 236 questions) and a public Course set (5 videos, 295 questions), LMVQA is reported to raise binary human-judged accuracy versus an adapted DrVideo baseline from 31% to 94% and from 21% to 88%, with larger relative gains on the more diagram-heavy Course data, while after one-time indexing cutting average per-question latency from 81.3s to 3.3s (Ciena) and 98.4s to 9.2s (Course) and reducing token-based LLM API cost by about 75%. Three Ciena engineers’ interviews further support usefulness for locating rationale and tracing answers to timestamps.

Significance. If the end-to-end system results hold under broader evaluation, the work is a practically significant contribution to software-engineering knowledge recovery from asynchronous meeting archives: it targets a real industrial pain point (long, diagram-rich recordings), ships a deployable two-stage design with offline corpus reuse, reports large accuracy and cost/latency gains under matched API conditions, provides dual independent labeling with high pooled Cohen’s κ (0.908), explicit correctness criteria, a public Course dataset and replication package, and qualitative practitioner feedback plus Ciena deployment notes. The combination of diagram-type-aware extraction, multimodal grounding, and amortized indexing is a useful systems template even if causal isolation of individual components remains incomplete.

major comments (3)
  1. [RQ1 / Lesson 1 / Table VI / Table II] The central attribution of accuracy gains to diagram-type-aware extraction (Abstract; §I Contributions; Lesson 1; Table VI) is not isolated. RQ1 reports only end-to-end LMVQA vs adapted DrVideo. Confounds include: (i) offline reusable indexing + RAG vs DrVideo’s per-question raw-video processing (§III, §IV-B, Table VII); (ii) GPT-5 for diagram frames vs GPT-4o for DrVideo and for non-diagram frames (Table II); (iii) different prompting and top-k retrieval. No ablation holds indexing/retrieval/model fixed while toggling diagram detection, category classification, or category-specific schemas (Listings a–e). Without that, the claim that SE diagram customization (rather than corpus reuse + stronger models + RAG) drives the 31%→94% / 21%→88% jumps is under-supported. Add ablations (e.g., no-diagram-pipeline, non-diagram-only extraction, matched LLM tier) or substantially tone the causal lang
  2. [§IV-A Table I / Validity Considerations / Abstract] External validity rests on only ten videos (five industrial, five course; Table I). Although the authors note this in Validity Considerations, the Abstract and contributions still present large absolute gains as general evidence for diagram-rich technical meeting QA. With average ~47–59 questions per video but few distinct meetings, presentation styles, and domains, the reported accuracy and “larger gains on diagram-rich videos” remain fragile. Strengthen with more videos, stratified diagram-vs-audio question breakdowns with confidence intervals, or clearer scoping as an industrial experience report rather than a general method claim.
  3. [§IV-B / Table VI / Table I] The baseline comparison is necessary but imperfectly controlled for open-ended technical-meeting QA. DrVideo was designed for multiple-choice long-video QA and is adapted via prompts/post-processing to open-ended JSON (§IV-B); it also lacks an offline index by design. Reporting only binary correctness against ground truth without error typology (hallucination, missed diagram entity, ASR failure, retrieval miss) or per-question-type accuracy (diagram vs audio vs visual; Table I averages) makes it hard to verify where LMVQA’s gains actually accrue. Provide stratified accuracy and a short error analysis on the incorrect LMVQA cases (14 Ciena, 34 Course).
minor comments (5)
  1. [§IV-E] Section heading typography: “Internal V alidity” / “External V alidity” appear with a space in “V alidity” (§IV-E); fix throughout.
  2. [Table II / §IV-B] Table II lists GPT-5 for diagram frames while the narrative sometimes refers generically to “frontier LLM”; state access date/version constraints more clearly for reproducibility, given API model churn.
  3. [Fig. 1 / Fig. 2] Figure 1 and Figure 2 are helpful but dense; a short caption note on which stages use GPT-4o-mini vs GPT-4o vs GPT-5 would reduce cross-referencing to Table II.
  4. [RQ3 / §IV-C] RQ3 rests on three engineers and self-reported accuracy during short sessions; present this more clearly as formative feedback rather than parallel evidence to the 531-question quantitative study.
  5. [Table VIII / §V] Related-work Table VIII’s partial-support marks for DrVideo (audio/diagram) are reasonable but could cite the exact DrVideo mechanisms that justify ▷ rather than × for diagram grounding.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical end-to-end system evaluation against independently authored ground-truth answers, not a fitted or definitional derivation.

full rationale

LMVQA is an industrial systems paper whose load-bearing claims are measured accuracy, latency, and cost against an adapted baseline (DrVideo) and human-labeled open-ended answers. Ground-truth answers were supplied by Ciena engineers and by non-author students; correctness is binary human judgment under explicit criteria (Tables III–IV), not a quantity recovered from parameters fitted to the same data. Offline indexing is reported separately from interactive latency and is amortized, not renamed as a prediction. There are no equations, uniqueness theorems, or ansatzes whose outputs reduce by construction to their inputs. Self-citations (replication package, related Ciena QA work) are ordinary and non-load-bearing for the accuracy/efficiency numbers. The paper is therefore self-contained against external benchmarks; the reader’s causal concern about missing ablations is a validity/attribution issue, not circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

Load-bearing content is engineering design choices and evaluation assumptions, not a mathematical derivation. Free parameters are pipeline thresholds and retrieval/model settings chosen from preliminary experiments. Axioms are standard retrieval/LLM practices plus SE domain assumptions about meeting-video redundancy and diagram taxonomies. The main invented entity is the LMVQA evidence-corpus pipeline itself.

free parameters (5)
  • SSIM keyframe threshold = 0.92
    Set to 0.92 from preliminary experiments; controls which visual changes become evidence units and thus what the QA model can see.
  • Frame sampling rate = 1 FPS
    Fixed at 1 FPS to cut cost while claiming sufficient temporal resolution for technical slides/diagrams.
  • Top-k retrieved chunks = 200
    K=200 chosen to balance evidence coverage and context limits; truncated further if GPT-4o context is exceeded.
  • LLM role assignment (4o-mini / 4o / 5) = GPT-4o-mini, GPT-4o, GPT-5 as specified in Table II
    Model choices for classification, non-diagram captioning, diagram extraction, and QA are hand-selected commercial models that directly affect accuracy and cost.
  • LLM temperature = 0
    Set to 0 for determinism; still leaves residual model nondeterminism unquantified because runs were not repeated.
assumptions (5)
  • domain assumption Technical meeting videos have high visual redundancy, so low-rate sampling plus SSIM filtering retains the semantically important frames.
    Stated in §III-A with the 30 FPS / 90-minute redundancy argument; if subtle diagram edits fall below the SSIM threshold, evidence is lost.
  • domain assumption A fixed SE diagram taxonomy (UML class/sequence/state-activity/use-case, network/topology, architecture/workflow, other) is adequate for category-specific extraction.
    Used in diagram classification and Listing d extraction fields; mis-taxonomy would degrade diagram answers.
  • domain assumption Cosine similarity over embeddings of textualized multimodal chunks is an adequate retrieval signal for open-ended SE questions.
    Core of §III-B RAG stage; no alternative retrievers evaluated.
  • domain assumption Binary semantic-equivalence labeling against author-provided ground truth is a valid accuracy metric for open-ended multimodal answers.
    Defined in Tables III–IV and used for all RQ1 claims; partial-credit or multi-aspect quality is not measured.
  • standard math Standard vector-space retrieval and commercial LLM generation behave as black-box oracles under temperature 0.
    Cosine similarity and API model calls are treated as given infrastructure rather than proved components.
invented entities (1)
  • LMVQA time-stamped multimodal evidence corpus with diagram-type-aware extraction
    purpose: Reusable offline index that fuses ASR sentences and structured diagram/non-diagram visual descriptions for grounded QA.
    The paper’s primary system artifact; evaluated only inside this work’s datasets and deployment narrative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Question Answering for Diagram-Rich Technical Meeting Videos." pith.science (2026). https://pith.science/paper/M45HTYBD

@misc{pith2026260710494,
  author       = {Pith},
  title        = {Pith review of: Question Answering for Diagram-Rich Technical Meeting Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M45HTYBD}},
  note         = {Machine review of arXiv:2607.10494}
}
read the original abstract

Software engineering increasingly relies on asynchronous communication artifacts, including recorded meetings where stakeholders discuss concerns, rationale, and decisions. These meetings often include diagram-based representations of requirements, system behavior, component interactions, and trace dependencies. Accessing knowledge from these meetings is challenging because recordings are long and relevant evidence is distributed across speech, slides, and technical diagrams. This paper reports our industrial experience developing and evaluating LMVQA, an LLM-based multimodal question-answering system for technical meeting videos. Developed in collaboration with engineers at Ciena, LMVQA supports the understanding of requirements and design intent by grounding answers in audio and visual evidence, with explicit handling of diagram-rich content such as requirements and UML diagrams. It processes each video once to build a reusable time-stamped evidence corpus for grounded question answering. Across a Ciena dataset and a public dataset, we show that LMVQA significantly improves answer accuracy compared to a state-of-the-art baseline, from 31% to 94% on the Ciena dataset and from 21% to 88% on the public dataset, with larger gains on diagram-rich videos. We further show that, after one-time indexing, LMVQA reduces average response time from 81.3s to 3.3s on Ciena and from 98.4s to 9.2s on the public dataset, while lowering average token-based LLM API cost by about 75%. Finally, our interviews with three domain experts show that engineers particularly value LMVQA for locating software-engineering-relevant information, revisiting rationale, and tracing answers to specific video segments.

Figures

Figures reproduced from arXiv: 2607.10494 by the authors.

Figure 2
Figure 2. Excerpt of an example showing how visual and audio content from [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 1
Figure 1. Overview of our LLM-based multimodal approach for technical video [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 6 linked inside Pith

  1. [1]

    Requirements engineering in the year 00: A re- search perspective,

    A. van Lamsweerde, “Requirements engineering in the year 00: A re- search perspective,” inProceedings of the 22nd International Conference on Software Engineering (ICSE), 2000, pp. 5–19

    2000

  2. [2]

    Effectiveness of elicitation techniques in distributed requirements engineering,

    W. J. Lloyd, M. B. Rosson, and J. D. Arthur, “Effectiveness of elicitation techniques in distributed requirements engineering,” in10th Anniversary IEEE Joint International Conference on Requirements Engineering (RE 2002), 2002, pp. 311–318

    2002

  3. [3]

    Hy- perMeeting: Supporting asynchronous meetings with hypervideo,

    A. Girgensohn, J. Marlow, F. M. Shipman, and L. Wilcox, “Hy- perMeeting: Supporting asynchronous meetings with hypervideo,” in Proceedings of the 23rd Annual ACM Conference on Multimedia, 2015, pp. 611–620

    2015

  4. [4]

    Turning asyn- chronicity into an opportunity: asynchronous communication for shared understanding with vision videos,

    L. Nagel, O. Karras, S. M. Amiri, and K. Schneider, “Turning asyn- chronicity into an opportunity: asynchronous communication for shared understanding with vision videos,”Requirements Engineering, vol. 29, pp. 49–71, 2024

    2024

  5. [5]

    A comparison of asynchronous online text-based lectures and synchronous interactive web conferencing lectures,

    A. Skylar, “A comparison of asynchronous online text-based lectures and synchronous interactive web conferencing lectures,”Issues in Teacher Education, vol. 18, no. 2, 2009

    2009

  6. [6]

    Team-based learning in asynchronous online settings,

    S. Palsole and C. Awalt, “Team-based learning in asynchronous online settings,”New Directions for Teaching and Learning, vol. 2008, no. 116, pp. 87–95, 2008

    2008

  7. [7]

    CodeTube: Extracting relevant fragments from software development video tutorials,

    L. Ponzanelli, G. Bavota, A. Mocci, M. Di Penta, R. Oliveto, B. Russo, S. Haiduc, and M. Lanza, “CodeTube: Extracting relevant fragments from software development video tutorials,” inProceedings of the 38th International Conference on Software Engineering Companion, 2016, pp. 645–648

    2016

  8. [8]

    Text retrieval-based tagging of software engineering video tutorials,

    J. Escobar-Avila, E. Parra, and S. Haiduc, “Text retrieval-based tagging of software engineering video tutorials,” in2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE- C), 2017

    2017

Show all 40 references
  1. [9]

    Automatic tag recommen- dation for software development video tutorials,

    E. Parra, J. Escobar-Avila, and S. Haiduc, “Automatic tag recommen- dation for software development video tutorials,” inProceedings of the 26th IEEE/ACM International Conference on Program Comprehension (ICPC), 2018

    2018

  2. [10]

    psc2code: Denois- ing code extraction from programming screencasts,

    L. Bao, Z. Xing, X. Xia, D. Lo, M. Wu, and X. Yang, “psc2code: Denois- ing code extraction from programming screencasts,”ACM Transactions on Software Engineering and Methodology, 2020

    2020

  3. [11]

    VT-Revolution: Interactive programming video tutorial authoring and watching system,

    L. Bao, Z. Xing, X. Xia, and D. Lo, “VT-Revolution: Interactive programming video tutorial authoring and watching system,”IEEE Transactions on Software Engineering, 2019

    2019

  4. [12]

    Improving code extraction from coding screencasts using a code-aware encoder-decoder model,

    A. Malkadi, A. Tayeb, and S. Haiduc, “Improving code extraction from coding screencasts using a code-aware encoder-decoder model,” inProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2023

    2023

  5. [13]

    Efficient search of live-coding screen- casts from online videos,

    C. Yang, F. Thung, and D. Lo, “Efficient search of live-coding screen- casts from online videos,” inIEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), 2022, pp. 73–77

    2022

  6. [14]

    Semantic GUI scene learning and video alignment for detecting du- plicate video-based bug reports,

    Y . Yan, N. Cooper, O. Chaparro, K. Moran, and D. Poshyvanyk, “Semantic GUI scene learning and video alignment for detecting du- plicate video-based bug reports,” inProceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE), 2024, pp. 2868–2880

    2024

  7. [15]

    Question answering for multi-release systems: A case study at ciena,

    P. Khamsepour, M. Cole, I. Ashraf, S. Puri, M. Sabetzadeh, and S. Nejati, “Question answering for multi-release systems: A case study at ciena,” in33rd IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), 2026

    2026

  8. [16]

    Developing a llama-based chatbot for CI/CD question answer- ing: A case study at ericsson,

    D. Chaudhary, S. L. Vadlamani, D. Thomas, S. Nejati, and M. Sabet- zadeh, “Developing a llama-based chatbot for CI/CD question answer- ing: A case study at ericsson,” inIEEE International Conference on Software Maintenance and Evolution (ICSME), 2024, pp. 707–718

    2024

  9. [17]

    Experiences developing an AI chatbot in the pharmaceutical industry,

    S. Abedu, A. Yuen, A. Abdellatif, S. Owolabi, M. S. Ruiz Rodriguez, C. Lim Ah Tock, A. Zaraket, E. Shihab, and N. Nasseri, “Experiences developing an AI chatbot in the pharmaceutical industry,” in7th Interna- tional Workshop on Bots and Agents in Software Engineering (BoatSE), 2026

    2026

  10. [18]

    Evaluating large language models for requirements question answering in industrial aerospace software,

    L. Yang, Y . Luo, H. Gao, Y . Fan, J. Zhang, X. Li, X. Dong, B. Gu, Z. Jin, and M. Yang, “Evaluating large language models for requirements question answering in industrial aerospace software,” inProceedings of the 33rd ACM International Conference on the Foundations of Softwa...

    2025

  11. [19]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Hao, M. M. Botvinick, O. Vinyals, A. Zisserman, and K. Simonyan, “Flamingo: a visual language model for few-shot learning...

    2022 arXiv

  12. [20]

    Video-LLaMA: An instruction-tuned audio-visual language model for video understanding,

    H. Zhang, X. Li, and L. Bing, “Video-LLaMA: An instruction-tuned audio-visual language model for video understanding,”arXiv preprint arXiv:2306.02858, 2023. [Online]. Available: https://arxiv.org/abs/2306. 02858

    2023 arXiv

  13. [21]

    Video- LLaV A: Learning united visual representation by alignment before projection,

    B. Lin, B. Zhu, Y . Ye, M. Ning, P. Jin, and L. Yuan, “Video- LLaV A: Learning united visual representation by alignment before projection,”arXiv preprint arXiv:2311.10122, 2023. [Online]. Available: https://arxiv.org/abs/2311.10122

    2023 arXiv

  14. [22]

    Video-ChatGPT: Towards detailed video understanding via large vision and language models,

    M. Maaz, H. Rasheed, S. Khan, and F. S. Khan, “Video-ChatGPT: Towards detailed video understanding via large vision and language models,”arXiv preprint arXiv:2306.05424, 2023. [Online]. Available: https://arxiv.org/abs/2306.05424

    2023 arXiv

  15. [23]

    DrVideo: Document retrieval based long video understanding,

    Z. Ma, C. Gou, H. Shi, B. Sun, S. Li, H. Rezatofighi, and J. Cai, “DrVideo: Document retrieval based long video understanding,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2025, pp. 18 936–18 946

    2025

  16. [24]

    A simple LLM framework for long-range video question- answering,

    C. Zhang, T. Lu, M. M. Islam, Z. Wang, S. Yu, M. Bansal, and G. Bertasius, “A simple LLM framework for long-range video question- answering,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). Miami, Florida, USA: Association for ...

    2024

  17. [25]

    LongVLM: Efficient long video understanding via large language models,

    Y . Weng, M. Han, H. He, X. Chang, and B. Zhuang, “LongVLM: Efficient long video understanding via large language models,” arXiv preprint arXiv:2404.03384, 2024. [Online]. Available: https: //arxiv.org/abs/2404.03384

    2024 arXiv

  18. [26]

    LMVQA: Prompts, resources, codes, and interviews for the LMVQA system,

    Z. Xu, “LMVQA: Prompts, resources, codes, and interviews for the LMVQA system,” https://github.com/ZhuoRanRan/LMVQA, 2026, [On- line]

    2026

  19. [27]

    Frame rates explained: Why FPS matters in broadcasting,

    Medialooks, “Frame rates explained: Why FPS matters in broadcasting,” 2025. [Online]. Available: https://medialooks.com/ articles/frame-rates-explained-why-fps-matters-in-broadcasting

    2025

  20. [28]

    Image quality assessment: From error visibility to structural similarity,

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,”IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004

    2004

  21. [29]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V . Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” inProceedings of the 36th International Conference on Neural Information Processing Systems, ser. NeurIPS ’2...

    2022

  22. [30]

    Language models are few-shot learners,

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B...

    2020

  23. [31]

    Robust speech recognition via large-scale weak supervi- sion,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervi- sion,” inProceedings of the 40th International Conference on Machine Learning (ICML), ser. Proceedings of Machine Learning Research, vol. 202, 202...

    2023

  24. [32]

    A vector space model for automatic indexing,

    G. Salton, A. Wong, and C. S. Yang, “A vector space model for automatic indexing,”Communications of the ACM, vol. 18, no. 11, pp. 613–620, 1975

    1975

  25. [33]

    The measurement of observer agreement for categorical data,

    J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data,”Biometrics, vol. 33, no. 1, pp. 159–174, 1977

    1977

  26. [34]

    PlotQA: Reasoning over scientific plots,

    N. Methani, P. Ganguly, M. M. Khapra, and P. Kumar, “PlotQA: Reasoning over scientific plots,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 1520–1529

    2020

  27. [35]

    ChartQA: A benchmark for question answering about charts with visual and logical reasoning,

    A. Masry, D. X. Long, J. Q. Tan, S. Joty, and E. Hoque, “ChartQA: A benchmark for question answering about charts with visual and logical reasoning,” inFindings of the Association for Computational Linguistics: ACL 2022, 2022, pp. 2263–2279. [Online]. Available: https://aclant...

    2022

  28. [36]

    ChartVQA: A benchmark for question answering on charts using visual reasoning,

    Z. Yanget al., “ChartVQA: A benchmark for question answering on charts using visual reasoning,”arXiv preprint arXiv:2203.10244, 2022. [Online]. Available: https://arxiv.org/abs/2203.10244

    2022 arXiv

  29. [37]

    A diagram is worth a dozen images,

    A. Kembhavi, M. Salvato, E. Kolve, M. J. Seo, H. Hajishirzi, and A. Farhadi, “A diagram is worth a dozen images,” inComputer Vision – ECCV 2016, ser. Lecture Notes in Computer Science, vol. 9908. Springer, 2016, pp. 235–251. [Online]. Available: https://doi.org/10.1007/978-3-3...

    2016 doi

  30. [38]

    DocVQA: A dataset for VQA on document images,

    M. Mathew, D. Karatzas, and C. V . Jawahar, “DocVQA: A dataset for VQA on document images,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021, pp. 2200–2209

    2021

  31. [39]

    Test input validation for vision-based dl systems: An active learning approach,

    D. Ghobari, M. H. Amini, D. Q. Tran, S. Park, S. Nejati, and M. Sabetzadeh, “Test input validation for vision-based dl systems: An active learning approach,” inIEEE/ACM 47th International Con- ference on Software Engineering: Software Engineering in Practice (ICSE 2025). IEEE,...

    2025 doi

  32. [40]

    Milvus: A purpose-built vector data management system,

    J. Wang, X. Yi, R. Guo, H. Jin, P. Xu, S. Li, X. Wang, X. Guo, C. Li, X. Xu, K. Yu, Y . Yuan, Y . Zou, J. Long, Y . Cai, Z. Li, Z. Zhang, Y . Mo, J. Gu, R. Jiang, Y . Wei, and C. Xie, “Milvus: A purpose-built vector data management system,” inProceedings of the 2021 Internatio...

    2021

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.