Pith. sign in

REVIEW 4 major objections 6 minor 66 references

Beyond the Textual: Generating Coherent Visual Options for MCQs

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CmOS generates educational MCQs with visual answer and distractor options and reports state-of-the-art results on ScienceQA.

desk verdict New task, sensible framework, but the visual-option SOTA rests on an undefined SSIM and a retrieval database that may leak test images; needs a clean re-run before the headline claim holds. read the letter →

arxiv 2508.18772 v1 pith:5VCKA7CF submitted 2025-08-26 cs.CV cs.CL

Wanqiang Wang , Longzhu He , Wei Zheng This is my paper
classification cs.CVcs.CL
keywords multiple-choicequestiongenerationvisualoptiondistractormultimodalchain-of-thoughtretrieval-augmentedtext-to-imageeducationalassessmentScienceQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Writing multiple-choice questions is useful but expensive, and existing automatic generators nearly always produce text-only options. This paper argues that option quality can be improved by generating images as options, and proposes a cross-modal synthesis pipeline, CmOS, that turns an input image, context, and correct answer into a question, a visual answer, and visual distractors. The pipeline has four stages: it judges whether the content can be pictured, produces several question–reason pairs with multimodal chain-of-thought reasoning, selects the best pair by internal and external consistency scores, and finally generates visual options by retrieving image templates and iteratively tuning a text-to-image model. If the reported results hold, educators could scale visual distractor authoring instead of hand-crafting images, which would make visual MCQs available in subjects where they are currently rare.

What carries the argument

The load-bearing mechanism is the four-stage CmOS pipeline: (a) a discriminator with retrieved exemplars and multimodal chain-of-thought (MCoT) decides whether content is convertible; (b) question and reason generators produce candidate pairs; (c) OQRM (Optimal Question-Reason Match) ranks the pairs by a Total Match Score, a weighted sum of internal consistency (closeness to the centroid of all candidates) and external consistency (alignment with the original image and the phrase 'a photo of A'); (d) an option generator writes descriptions, an image database is searched by a mixed image-caption similarity score, and an MLLM evaluates and tunes the T2I output up to three rounds. The OQRM scor

What would settle it

Check whether any of the 812 evaluation instances appear in the image database used for template retrieval, then recompute structural similarity under both option-to-option and option-to-template reference settings; if overlap or template-reference scoring appears, the reported visual gains shrink or vanish.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that visual options are not an afterthought to MCQ generation but a separate cross-modal synthesis problem with its own solution. CmOS reports 88.2% accuracy in deciding which content is convertible to visual options, 75.5 BLEU-4 for question generation, and 59.5 in structural-similarity score (SSIM) for visual options, beating every baseline across subjects, modalities, and grade levels on ScienceQA. The ablations ground these gains in specific mechanisms: removing retrieved exemplars lowers discrimination accuracy by 19.5 points, removing the optimal question–reason matching module lowers BLEU-4 by 33.1 points, and removing template-ba

Load-bearing premise

The visual-option gains rest on two assumptions: the retrieval database contains none of the evaluation images, and the structural-similarity metric compares generated options against each other rather than against the retrieved template; if either fails, the visual scores overstate CmOS's advantage.

Editorial extensions

If this is right

  • If CmOS is right, an educator can input a lesson image, context, and correct answer and receive a question plus image-based answer and distractor options, reducing manual visual authoring.
  • Content discrimination becomes a reusable front gate: the model flags which content is worth converting, so visual-option generation is applied only where it helps.
  • The OQRM selection rule gives a concrete way to choose among candidate questions by consistency, not by surface likelihood, and could carry over to other generative tasks.
  • The template-and-tune loop enforces option-to-option structural similarity, which is the property that makes visual distractors look like a coherent set rather than isolated images.
  • Reported gains are consistent across subjects, modalities, and grade levels, suggesting the framework is not tuned to a single content type.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's tests, the discriminator's notion of 'convertible' could transfer to diagram-rich disciplines such as anatomy, geography, or chemistry, where textual distractors are hard to author; that transfer is unproven.
  • A natural next experiment is whether higher SSIM and CLIP-T scores predict actual student confusion; the paper's human ratings only partially bridge this, so a behavioral study would be a testable extension of its claims.
  • Because the paper builds its exemplar pool from ScienceQA, out-of-domain content would likely degrade discrimination; an inference worth testing is whether a continually growing, domain-general exemplar database closes that gap.
  • The retrieve-then-tune mechanism might also serve as a general way to force text-to-image models to respect within-set visual consistency, beyond education.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Cross-modal Options Synthesis (CmOS), a four-stage framework for generating multiple-choice questions (MCQs) with visual options from a multimodal input (context, image, answer). The stages are: (a) content discrimination with exemplar retrieval, (b) MCoT-based generation of candidate question-reason pairs, (c) optimal question-reason matching (OQRM) via a weighted similarity score, and (d) visual option generation using retrieved image templates and iterative T2I refinement. Experiments on ScienceQA claim state-of-the-art results on content discrimination (88.2% accuracy), question generation (75.5 BLEU-4), and visual option generation (59.5 SSIM, 40.2 CLIP-T), with ablations showing the contribution of each module.

Significance. If the empirical claims are reliable, this is the first systematic treatment of visual-option MCQ generation and offers a plausible architecture combining retrieval-augmented generation with multimodal chain-of-thought reasoning. The paper's strengths include a clear modular design, extensive ablations, human evaluation with explicit guidelines, and a hyperparameter analysis. However, the evaluation contains unresolved issues that directly affect the headline numeric claims: the image database used for template retrieval is not shown to be disjoint from the evaluation instances, the SSIM metric's reference is undefined, and the few-shot exemplar set is drawn from the same test split used for evaluation. These issues are correctable, and the underlying framework remains plausible, so the work is worth revising rather than rejecting.

major comments (4)
  1. [Sec. 3.4 and Sec. 4.1] The image database D is described only as 'collecting images from the ScienceQA' (Sec. 3.4), while the evaluation dataset DQ is sampled from DC, which is the ScienceQA test set minus the 482 exemplars (Sec. 4.1). No statement establishes that D is disjoint from DQ. If D contains test-set images, the retrieval stage can return the correct option image (or a near-duplicate) as the template, and the final 'generated' option is an image-to-image transformation of the target rather than a synthesis from the description. This would inflate the SSIM and CLIP-T values in Table 3. The w/o Template ablation showing an 11.2-point SSIM drop is consistent with the template carrying direct target information. Please re-run the evaluation with D restricted to the training split (or otherwise prove disjointness) and report the resulting numbers.
  2. [Sec. 4.1 (Metrics)] SSIM is a full-reference metric, yet the reference image is never specified. The statement 'SSIM evaluates the perceptual quality of visual options by jointly modeling luminance, contrast, and structural consistency' is insufficient. It is unclear whether the score is computed (a) between each generated option and the original ScienceQA image, (b) between a generated option and its retrieved template, or (c) pairwise among the generated options. Each choice leads to a different interpretation of Table 3. Please define the exact computation, the reference, and the parameters used (e.g., window size). Without this, the main visual-option result is not reproducible.
  3. [Sec. 3.2 and Sec. 4.1] The exemplar set DE is explicitly 'extracted from the ScienceQA test dataset' (Sec. 3.2), and the content discriminator is evaluated on the remaining test instances DC (Sec. 4.1). Using few-shot exemplars from the same benchmark split can leak distributional information, and the retrieval mechanism (Eq. 2) may retrieve near-duplicate exemplars whose convertibility labels are directly informative. This puts the reported 88.2% discrimination accuracy in question. The authors should construct DE from the training split (or a held-out development split) and verify that the retrieval similarity does not expose the convertibility label.
  4. [Tables 1, 3, and Fig. 3] All automatic results are reported as point estimates without error bars, confidence intervals, or significance tests, despite the text claiming superiority across every subject, modality, and grade level. With per-category sample sizes that are necessarily small (e.g., LAN questions), the observed gaps may not be statistically reliable. Please report variance over multiple runs or bootstrap intervals, and provide significance tests for the headline comparisons against the strongest baselines.
minor comments (6)
  1. [Appendix B / Fig. 8] The dataset statistics are internally inconsistent. For DC, subject counts sum to 3746 (1990+969+787), while modality counts sum to 3759 (1795+1964); grade counts give 3759 as well. Similar discrepancies appear for DQ, where only NAT and SOC are listed but IMG/TXT and grade counts are provided. Please reconcile these numbers.
  2. [Eq. (5)] The summation symbol appears as 'X' in the equation, and the TMS variable is not explicitly defined. This makes the selection criterion hard to parse.
  3. [Figure 1] The label 'MPQG' in the figure is unexplained and appears to be a leftover from a previous version.
  4. [Sec. 5] The conclusion refers to a 'three-stage Multimodal-CoT framework,' but the method section describes four stages. Please correct the inconsistency.
  5. [Table 4] There is a typo: 'DELLE-3' should be 'DALLE-3'.
  6. [Sec. 4.1] The text says 'Our method adopts QWEN 2.1-TURBO as the backbone,' but Appendix C and Table 3 identify the image generator as WANX2.1-turbo. Please clarify the naming.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CmOS is an empirical pipeline evaluated against human labels and external metrics; no central claim reduces to its inputs or to self-citation.

full rationale

The paper's claims are empirical benchmark results rather than derivations from first principles. Content discrimination accuracy is measured against human-annotated convertibility labels on D_C; question generation is measured against human-authored references using BLEU-4, ROUGE-L, and METEOR; visual option generation is measured with SSIM, CLIP-T, and human ratings. The hyperparameters α and β are selected by explicit sweeps, and for α the paper states that tuning was performed on 300 instances sampled from D_C with no overlap with D_Q, which is standard practice and not circular. There is no load-bearing self-citation: the cited external tools (Qwen, FARE, Wanx, baselines) are used as components or comparators, not as proof of CmOS's own claims. The main concerns are data-splitting and metric-definition omissions: the image database D is said to be built by 'collecting images from the ScienceQA' without an explicit statement that it excludes the evaluation set D_Q, and the SSIM metric is described only as modeling 'luminance, contrast, and structural consistency' without specifying the reference image. These are potential leakage/evaluation-validity risks, not circularity: even if the template database overlaps the test instances or SSIM uses the template as reference, the reported numbers are not equivalent to the inputs by construction. The framework remains an independently evaluated pipeline whose components are compared against external baselines and human judgments.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new theoretical entities are introduced. The load-bearing commitments are the free parameters alpha, beta, and sigma, plus several empirical assumptions about embedding similarity, annotator reliability, dataset disjointness, and the validity of the image-based evaluation.

free parameters (3)
  • alpha (alpha) = 0.6
    Balance weight between internal and external consistency in the OQRM total match score; chosen by maximizing BLEU/ROUGE/METEOR on a 300-instance subset of DC (Appendix E).
  • beta (beta) = 1.4
    Balance weight between image and caption similarity in template retrieval (Eq. 6); chosen by maximizing SSIM/CLIP-T on an evaluation subset (Appendix E).
  • sigma (sigma) = 0.8
    Acceptance threshold for generated visual option similarity; set by hand without sensitivity analysis (Section 3.4).
assumptions (5)
  • domain assumption Embedding-space consistency measured by cosine similarity correlates with human-judged question quality
    OQRM selects question-reason pairs based on vector similarities to the center of candidate embeddings and to the source content; no evidence validates this proxy against human preferences beyond the reported metrics.
  • domain assumption Annotator labels for 'convertible' content are reliable ground truth
    Three annotators labeled convertibility and created reference questions; inter-annotator agreement is not reported, and the label definition is subjective.
  • domain assumption The retrieval image database is independent from the test set
    Section 3.4 says the database is built from ScienceQA images but never states a train/test split; if it includes test images, template retrieval leaks information into visual option generation.
  • domain assumption SSIM is computed as pairwise similarity between generated options
    The metric description in Section 4.1 does not specify the reference image for SSIM; the interpretation changes the validity of the visual option comparison.
  • domain assumption Text-to-image models can produce educationally valid visual options from descriptions and templates
    This is the premise of the generation stage, yet human evaluation scores are low (3.55/5) and the limitations admit hallucinations and inconsistent style.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond the Textual: Generating Coherent Visual Options for MCQs." pith.science (2026). https://pith.science/paper/5VCKA7CF

@misc{pith2026250818772,
  author       = {Pith},
  title        = {Pith review of: Beyond the Textual: Generating Coherent Visual Options for MCQs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5VCKA7CF}},
  note         = {Machine review of arXiv:2508.18772}
}
read the original abstract

Multiple-choice questions (MCQs) play a crucial role in fostering deep thinking and knowledge integration in education. However, previous research has primarily focused on generating MCQs with textual options, but it largely overlooks the visual options. Moreover, generating high-quality distractors remains a major challenge due to the high cost and limited scalability of manual authoring. To tackle these problems, we propose a Cross-modal Options Synthesis (CmOS), a novel framework for generating educational MCQs with visual options. Our framework integrates Multimodal Chain-of-Thought (MCoT) reasoning process and Retrieval-Augmented Generation (RAG) to produce semantically plausible and visually similar answer and distractors. It also includes a discrimination module to identify content suitable for visual options. Experimental results on test tasks demonstrate the superiority of CmOS in content discrimination, question generation and visual option generation over existing methods across various subjects and educational levels.

Figures

Figures reproduced from arXiv: 2508.18772 by the authors.

Figure 1
Figure 1. An example of the CmOS Framework, showing how it generates a visual MCQ to identify the part re￾sponsible for photosynthesis, using content that includes an image, an answer, and the context about plant cell. bilities in scientific question answering, researchers have begun to explore their potential in the auto￾matic generation of educational questions to alle￾viate human labor, including objective and open￾ended f… view at source ↗
Figure 2
Figure 2. Overview of multimodal educational questions and visual options generation: (a) Evaluate content [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Automatic evaluation results of content discrimination. In terms of accuracy, regardless of subject, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Case in terms of the OQRM module. Option: A long bench Description: A long wooden bench placed horizontally in a minimal indoor space with a smooth gray floor and white wall. The bench is made of light brown wood, supported by four legs, and spans almost the full width…
Figure 5
Figure 5. Figure 5: Case regarding the Template and Tuning. 5 Conclusion In this paper, we present a novel framework called Cross-modal Options Synthesis (CmOS), which combines retrieved similar exemplars and Multi￾modal Chain-of-Thought (MCoT) reasoning to gen￾erate educational multiple-…
Figure 6
Figure 6. Figure 6: The overall performance of question genera [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: The overall performance of question genera [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Dataset statistics of ScienceQA test benchmark and our test sets. Question types: NAT = natural science, [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Guideline of human evaluation for question and visual option generation quality. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 46 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Naveed Afzal and Ruslan Mitkov. 2014. Automatic generation of multiple choice questions using dependency-based semantic relations. Soft Computing, 18:1269--1281

    work page 2014

  4. [4]

    Naveed Afzal, Ruslan Mitkov, and Atefeh Farzindar. 2011. Unsupervised relation extraction using dependency trees for automatic generation of multiple-choice questions. In Advances in Artificial Intelligence: 24th Canadian Conference on Artificial Intelligence, Canadian AI 2011, St. John’s, Canada, May 25-27, 2011. Proceedings 24, pages 32--43. Springer

    work page 2011

  5. [5]

    Renlong Ai, Sebastian Krause, Walter Kasper, Feiyu Xu, and Hans Uszkoreit. 2015. Semi-automatic generation of multiple-choice tests from mentions of semantic relations. In Proceedings of the 2nd Workshop on Natural Language Processing Techniques for Educational Applications, pages 26--33

    work page 2015

  6. [6]

    Somaiya Al Shuraiqi, Abdulrahman Aal Abdulsalam, Ken Masters, Hamza Zidoum, and Adhari AlZaabi. 2024. Automatic generation of medical case-based multiple-choice questions (mcqs): a review of methodologies, applications, evaluation, and future directions. Big Data and Cognitive Computing, 8(10):139

    work page 2024

  7. [7]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  8. [8]

    Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65--72

    2005

Show all 66 references
  1. [9]

    Jonathan Brown, Gwen Frishkoff, and Maxine Eskenazi. 2005. Automatic question generation for vocabulary assessment. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 819--826

    2005

  2. [10]

    Sahan Bulathwela, Hamze Muse, and Emine Yilmaz. 2023. Scalable educational question generation with pre-trained language models. In International Conference on Artificial Intelligence in Education, pages 327--339. Springer

    2023

  3. [11]

    Andrew C Butler. 2018. Multiple-choice testing in education: Are the best practices for assessment also good for learning? Journal of Applied Research in Memory and Cognition, 7(3):323--331

    2018

  4. [12]

    Shuyang Cao and Lu Wang. 2021. Controllable open-ended question generation with a new question type ontology. arXiv preprint arXiv:2107.00152

    2021 arXiv

  5. [13]

    Rudeema Chughtai, Farooque Azam, Muhammad Waseem Anwar, Wasi Haider But, and Muhammad Umar Farooq. 2022. A lecture centric automated distractor generation for post-graduate software engineering courses. In 2022 International Conference on Frontiers of Information Technology (F...

    2022

  6. [14]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. https://arxiv.org/abs/2305.06500 Instructblip: Towards general-purpose vision-language models with instruction tuning . Preprint, arXiv:2305.06500

    2023 arXiv

  7. [15]

    Barbara Gross Davis. 2009. Tools for teaching. John Wiley & Sons

    2009

  8. [16]

    Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. https://arxiv.org/abs/2103.10360 Glm: General language model pretraining with autoregressive blank infilling . Preprint, arXiv:2103.10360

    2022 arXiv

  9. [17]

    Fares Fawzi, Sarang Balan, Mutlu Cukurova, Emine Yilmaz, and Sahan Bulathwela. 2024. Towards human-like educational question generation with small language models. In International Conference on Artificial Intelligence in Education, pages 295--303. Springer

    2024

  10. [18]

    Yifan Gao, Piji Li, Irwin King, and Michael R Lyu. 2019. Interconnected question generation with coreference alignment and conversation flow modeling. arXiv preprint arXiv:1906.06893

    2019 arXiv

  11. [19]

    Mark J Gierl, Okan Bulut, Qi Guo, and Xinxin Zhang. 2017. Developing, analyzing, and using distractors for multiple-choice tests in education: A comprehensive review. Review of educational research, 87(6):1082--1116

    2017

  12. [20]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  13. [21]

    Jing Gu, Mostafa Mirshekari, Zhou Yu, and Aaron Sisto. 2021. Chaincqg: Flow-aware conversational question generation. arXiv preprint arXiv:2102.02864

    2021 arXiv

  14. [22]

    Michael Heilman. 2011. Automatic factual question generation from text. Carnegie Mellon University

    2011

  15. [23]

    Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. 2024. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. arXiv preprint arXiv:2406.09403

    2024 arXiv

  16. [24]

    Inthrani Raja Indran, Priya Paranthaman, Neelima Gupta, and Nurulhuda Mustafa. 2024. Twelve tips to leverage ai for efficient and effective medical question generation: a guide for educators using chat gpt. Medical Teacher, 46(8):1021--1026

    2024

  17. [25]

    Zixi Jia, Jiqiang Liu, Hexiao Li, Qinghua Liu, and Hongbin Gao. 2024. Dcot: Dual chain-of-thought prompting for large multimodal models. In The 16th Asian Conference on Machine Learning (Conference Track)

    2024

  18. [26]

    Minchi C Kim and Michael J Hannafin. 2011. Scaffolding problem solving in technology-enhanced learning environments (teles): Bridging research and theory with practice. Computers & Education, 56(2):403--417

    2011

  19. [27]

    Myo-Kyoung Kim, Rajul A Patel, James A Uchizono, and Lynn Beck. 2012. Incorporation of bloom’s taxonomy into multiple-choice examination questions for a pharmacotherapeutics course. American journal of pharmaceutical education, 76(6):114

    2012

  20. [28]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199--22213

    2022

  21. [29]

    Devang Kulshreshtha, Muhammad Shayan, Robert Belfer, Siva Reddy, Iulian Vlad Serban, and Ekaterina Kochmar. 2022. Few-shot question generation for personalized feedback in intelligent tutoring systems. In PAIS 2022, pages 17--30. IOS Press

    2022

  22. [30]

    Archana Praveen Kumar, Ashalatha Nayak, Manjula Shenoy, Shashank Goyal, and 1 others. 2023. A novel approach to generate distractors for multiple choice questions. Expert Systems with Applications, 225:120022

    2023

  23. [31]

    Girish Kumar, Rafael E Banchs, and Luis Fernando D'Haro. 2015. Automatic fill-the-blank question generator for student self-assessment. In 2015 IEEE Frontiers in Education Conference (FIE), pages 1--3. IEEE

    2015

  24. [32]

    Ghader Kurdi, Jared Leo, Bijan Parsia, Uli Sattler, and Salam Al-Emari. 2020. A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30:121--204

    2020

  25. [33]

    Selvia Ferdiana Kusuma, Daniel Oranova Siahaan, and Chastine Fatichah. 2022. Automatic question generation with various difficulty levels based on knowledge ontology using a query template. Knowledge-Based Systems, 249:108906

    2022

  26. [34]

    Selvia Ferdiana Kusuma and 1 others. 2018. Automatic question generation for 5w-1h open domain of indonesian questions by using syntactical template-based features from academic textbooks. Journal of Theoretical and Applied Information Technology (JATIT), 96(12):3908--3923

    2018

  27. [35]

    Salima Lamsiyah, Abdelkader El Mahdaouy, Aria Nourbakhsh, and Christoph Schommer. 2024. Fine-tuning a large language model with reinforcement learning for educational question generation. In International Conference on Artificial Intelligence in Education, pages 424--438. Springer

    2024

  28. [36]

    Guillaume Le Berre, Christophe Cerisara, Philippe Langlais, and Guy Lapalme. 2022. Unsupervised multiple-choice question generation for out-of-domain q&a fine-tuning. In 60th annual meeting of the association for computational linguistics, volume 2, pages 732--738. Association...

    2022

  29. [37]

    Wei Li, Xue Xu, Jiachen Liu, and Xinyan Xiao. 2024. https://doi.org/10.18653/v1/2024.acl-long.335 UNIMO - G : Unified image generation through multimodal conditional diffusion . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume ...

    2024 doi

  30. [38]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  31. [39]

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2...

    2022

  32. [40]

    Haohao Luo, Yang Deng, Ying Shen, See-Kiong Ng, and Tat-Seng Chua. 2024. Chain-of-exemplar: enhancing distractor generation for multimodal educational question generation. ACL

    2024

  33. [41]

    Richard E Mayer. 2005. Cognitive theory of multimedia learning. The Cambridge handbook of multimedia learning, 41(1):31--48

    2005

  34. [42]

    Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. 2024. Compositional chain-of-thought prompting for large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14420--14431

    2024

  35. [43]

    Nikahat Mulla and Prachi Gharpure. 2023. Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications. Progress in Artificial Intelligence, 12(1):1--32

    2023

  36. [44]

    Philip Newton and Maira Xiromeriti. 2024. Chatgpt performance on multiple choice question examinations in higher education. a pragmatic scoping review. Assessment & Evaluation in Higher Education, 49(6):781--798

    2024

  37. [45]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 Bleu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL '02, page 311...

    2002

  38. [46]

    Ricardo Rodriguez-Torrealba, Eva Garcia-Lopez, and Antonio Garcia-Cabot. 2022. End-to-end generation of multiple-choice questions using text-to-text transfer transformer models. Expert Systems with Applications, 208:118258

    2022

  39. [47]

    Umme Sara, Morium Akter, and Mohammad Shorif Uddin. 2019. Image quality assessment through fsim, ssim, mse and psnr—a comparative study. Journal of Computer and Communications, 7(3):8--18

    2019

  40. [48]

    Christian Schlarmann, Naman Deep Singh, Francesco Croce, and Matthias Hein. 2024. Robust clip: Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models. arXiv preprint arXiv:2402.12336

    2024 arXiv

  41. [49]

    Rahul Singhal and Martin Henz. 2014. Automated generation of region based geometric questions. In 2014 IEEE 26th International Conference on Tools with Artificial Intelligence, pages 838--845. IEEE

    2014

  42. [50]

    Rahul Singhal, Martin Henz, Shubham Goyal, and FE Browder. 2015. A framework for automated generation of questions across formal domains. In the 17th international conference on artificial intelligence in education, pages 776--780

    2015

  43. [51]

    Linfeng Song and Lin Zhao. 2016. Question generation from a knowledge base with web exploration. arXiv preprint arXiv:1610.03807

    2016 arXiv

  44. [52]

    Will Thalheimer. 2014. https://www.worklearning.com/wp-content/uploads/2017/10/Learning-Benefits-of-Questions-2014-v2.0.pdf Learning benefits of questions . Technical report, Work-Learning Research. Version 2.0

    2014

  45. [53]

    Toyin Tofade, Jamie Elsner, and Stuart T Haines. 2013. Best practice strategies for effective use of questions as a teaching tool. American journal of pharmaceutical education, 77(7):155

    2013

  46. [54]

    Youness Touissi, Ghita Hjiej, Abderrazak Hajjioui, Azeddine Ibrahimi, and Maryam Fourtassi. 2022. Does developing multiple-choice questions improve medical students’ learning? a systematic review. Medical Education Online, 27(1):2005505

    2022

  47. [55]

    Masaki Uto, Yuto Tomikawa, and Ayaka Suzuki. 2023. Difficulty-controllable neural question generation for reading comprehension using item response theory. In Proceedings of the 18th workshop on innovative use of NLP for building educational applications (BEA 2023), pages 119--129

    2023

  48. [56]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    2022 arXiv

  49. [57]

    Zichao Wang and Richard Baraniuk. 2023. Multiqg-ti: Towards question generation from multi-modal sources. arXiv preprint arXiv:2307.04643

    2023 arXiv

  50. [58]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  51. [59]

    Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Jia-Jun Li, Nora Bradford, Branda Sun, and 1 others. 2022. Fantastic questions and where to find them: Fairytaleqa--an authentic dataset for narrative comprehension. arXiv preprint arXi...

    2022 arXiv

  52. [60]

    Gautam Yadav, Ying-Jui Tseng, and Xiaolin Ni. 2023. Contextualizing problems to student interests at scale in intelligent tutoring system using large language models. arXiv preprint arXiv:2306.00190

    2023 arXiv

  53. [61]

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2024. https://arxiv.org/abs/2304.14178 mplug-owl: Modulari...

    2024 arXiv

  54. [62]

    Min-Hsuan Yeh, Vicent Chen, Ting-Hao'Kenneth' Haung, and Lun-Wei Ku. 2022. Multi-vqg: Generating engaging questions for multiple images. arXiv preprint arXiv:2211.07441

    2022 arXiv

  55. [63]

    Nikki L Bibler Zaidi, Karri L Grob, Seetha M Monrad, Joshua B Kurtz, Andrew Tai, Asra Z Ahmed, Larry D Gruppen, and Sally A Santen. 2018. Pushing critical thinking skills with multiple-choice questions: does bloom’s taxonomy work? Academic Medicine, 93(6):856--859

    2018

  56. [64]

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493

    2022 arXiv

  57. [65]

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2023. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923

    2023 arXiv

  58. [66]

    Zhenjie Zhao, Yufang Hou, Dakuo Wang, Mo Yu, Chengzhong Liu, and Xiaojuan Ma. 2022. Educational question generation of children storybooks via question type distribution learning and event-centric summarization. arXiv preprint arXiv:2203.14187

    2022 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.