Pith. sign in

REVIEW 4 major objections 5 minor 45 references

"There Is No Such Thing as a Dumb Question," But There Are Good Ones

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that a good question is both appropriate and effective, and that a six-criterion rubric with two context variables lets an LLM judge score question quality automatically.

desk verdict A clearly argued rubric framework for question quality, but the empirical support rests on a three-question validity test and an unverified LLM judge. read the letter →

arxiv 2505.09923 v1 pith:OUGGBL5G submitted 2025-05-15 cs.AI

Minjung Shin , Donghyun Kim , Jeh-Kwang Ryu This is my paper · ORCID
classification cs.AI
keywords questionqualityevaluationlargelanguagemodelLLM-as-judgerubricscoringappropriatenesseffectivenessgenerationpragmatics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Questioning is everywhere in human discourse and increasingly central to AI, yet until now almost no one has tried to define what makes a question good rather than merely encouraging people to ask more. The paper proposes that a good question is one that is both appropriate and effective: appropriate means it fits the conversational context and social norms, effective means it achieves the asker's intended goal. To operationalize this, the authors build a five-point rubric with six sub-criteria (cohesion, answerability, respectfulness; clarity, coherence, informativeness) and inject two dynamic context variables—who the answerer is and what the discourse goal is—into the rubric descriptions. They then use claude-3-5-sonnet as a judge at temperature 0 to score follow-up questions from two datasets, one of well-formed uncertainty-resolving questions and one of intentionally problematic sensitive questions. The score profiles separate the two groups and shift when the goal variable changes, which the authors read as evidence that the framework can assess question quality in a context-sensitive way.

What carries the argument

The load-bearing mechanism is a five-point analytic rubric with six sub-components, each anchored by one of five ascending descriptions from 'complete deficiency' to 'full achievement'. Two dynamic placeholder variables are embedded in the rubric text: ${answerer} (who will respond), inserted into the Answerability criterion, and ${goal} (the discourse purpose), inserted into Clarity and Informativeness. The rubric is delivered as a prompt to the claude-3-5-sonnet-20240620 model at temperature 0, which scores each follow-up question on all six criteria; the placeholders make the evaluation semi-adaptive, so the same sentence can be judged differently depending on who is answering and what the conversation is trying to accomplish.

What would settle it

Take the same 300 questions from the CAUS and SQUARE datasets, have a panel of human judges apply the paper's rubric to them, and compare the human scores with the LLM's scores using a weighted agreement measure. If the LLM's scores diverge systematically from human ratings—or if a question that human judges unanimously consider excellent scores low on all six criteria—the framework's claim to measure question quality collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that question quality reduces to two measurable dimensions: appropriateness (sociolinguistic competence in context) and effectiveness (strategic competence in goal achievement), with 'a good question is both appropriate and effective' as the summary definition. Each dimension decomposes into three rubric sub-components, and the rubric is made context-dependent through two placeholder variables, ${answerer} and ${goal}, that modify the scoring descriptions for answerability, clarity, and informativeness. The authors claim that this rubric, used as a prompt for an LLM judge, produces scores that distinguish a legitimate follow-up question from deliberately misleading or off-topic versions, and that the same question receives different scores under different goals, demonstrating adaptability. They also claim that applying the rubric to two contrasting datasets reveals distinct, interpretable quality profiles—high clarity and respectfulness for well-formed questions, and specific deficits (e.g., low respectfulness and cohesion) for ethically problematic questions.

Load-bearing premise

The entire validation depends on the assumption that claude-3-5-sonnet, prompted with the hand-written rubric at temperature 0, gives scores that match human judgments of question quality; the paper reports it was selected for highest agreement with human evaluators but does not report any agreement numbers or the human-evaluation protocol.

Editorial extensions

If this is right

  • Question generation systems can be selected or fine-tuned against rubric scores, giving them an explicit quality signal instead of similarity to reference questions.
  • The same six-criterion rubric transfers across domains by simply re-instantiating the ${answerer} and ${goal} variables.
  • Score profiles become explainable diagnostics: for example, low respectfulness combined with low cohesion identifies ethically problematic questions.
  • AI systems can be evaluated on their questioning ability separately from their answering ability, supporting a shift toward question-centric interaction design.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the LLM judge's agreement with humans is as high as the paper implies, the framework offers a scalable proxy for human evaluation of open-ended dialogue questions, which the paper does not directly demonstrate with agreement evidence.
  • The two-dimensional structure suggests a testable typology: questions can be appropriate-but-ineffective (polite and on-topic but missing the goal) or effective-but-inappropriate (goal-achieving but rude or norm-violating); the paper's FQ#1 illustrates the former, and constructing a rude-but-informative example would stress-test the latter.
  • The framework's reliance on explicit goal and answerer variables implies that practical deployment requires a separate step—inferring the user's goal and the answerer's identity—before scores are meaningful, a step the paper treats as given.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a rubric-based framework for evaluating question quality in human-AI interaction. It defines two main dimensions—appropriateness, understood as sociolinguistic fit to context, and effectiveness, understood as strategic goal achievement—and breaks each into three subcomponents: cohesion, answerability, and respectfulness for appropriateness; clarity, coherence, and informativeness for effectiveness. A five-point scoring rubric is provided for each subcomponent, with two dynamic contextual variables, ${answerer} and ${goal}, inserted into selected rubric descriptions. The authors report a small validity test using one legitimate and two deliberately invalid follow-up questions (Figures 1-2), then apply the rubric via claude-3-5-sonnet-20240620 to 150 questions from the CAUS dataset and 150 questions from the SQUARE dataset, reporting means and standard deviations in Tables 3-4. The paper concludes that the framework can assess both well-formed and problematic questions while adapting to varied contexts.

Significance. If the measurement instrument were properly validated, the framework would be a useful contribution: the conceptual distinction between appropriateness and effectiveness is clearly motivated by pragmatics and speech-act theory, the rubric is detailed and operationalized, and the contextual variables offer a sensible mechanism for semi-adaptive scoring. The authors also release code and detailed evaluation procedures in a public repository, which supports reproducibility. However, the current empirical evidence does not establish that the LLM judge's scores correspond to human judgments of question quality, so the central claim is not yet supported. At this stage, the main value of the paper is the rubric design and theoretical framing; the validation is too thin to support the abstract's claims of demonstrated assessment ability.

major comments (4)
  1. [Methods, Evaluation Method] The manuscript states that claude-3-5-sonnet-20240620 "was selected after comparative testing with GPT-4, GPT-4-turbo, and Claude Opus, showing the highest agreement with human evaluators," but no agreement coefficients, number of raters, annotation instructions, or comparison table are provided. Since all reported score patterns in Tables 3 and 4 are produced by this single LLM judge at temperature 0, the central claim that the framework "assesses" question quality cannot be separated from the claim that this model's scores match human judgments. Without human-agreement statistics, the results could equally reflect the model's stylistic preferences. Please report the full human evaluation protocol, inter-rater agreement (e.g., Cohen's kappa or ICC) for each rubric dimension, and the comparative agreement results that motivated the model choice.
  2. [Validity Test of the Evaluation Metric] The validity test in Figures 1-2 uses only three author-written questions, and the rubric descriptions already encode the intended verdicts. For example, Informativeness level 1 is defined as "seeks irrelevant or speculative information," which is exactly the property assigned to FQ#1 and FQ#2, and Cohesion level 1 is defined as "contextually misused cohesive markers." The observed ordering is therefore largely a restatement of the scoring rubric rather than an independent test of the metric. The paper's own statement that "we validated only three questions" is not an adequate validity argument. Please validate on a larger, independently annotated item set that includes non-extreme cases near the boundaries of the scale, with human judgments collected under a transparent protocol.
  3. [Methods and Results, CAUS/SQUARE datasets] The empirical evaluation is statistically underpowered and faces a circularity risk. The CAUS dataset is authored by the same research group (Shin, Kim, & Ryu, 2024) and consists of LLM-generated questions, so if the judge favors LLM-like phrasing, the high clarity and respectfulness scores in Table 3 may reflect that preference rather than question quality. Tables 3 and 4 report only means and standard deviations, with no inferential tests comparing the first/third/fifth question sets or the three SQUARE categories, no comparison against existing QG metrics or human baselines, and no inter-rater reliability. Consequently, conclusions such as "the ethical question set needs improvement" (Results, Applying to Irrelevant and Ineffective Questions) are not statistically supported. Please add pre-defined hypotheses, appropriate significance tests or effect sizes, and at least one baseline metric.
  4. [Abstract and Discussion] The abstract's claim that the framework can adapt "to varied contexts" is not supported by the evidence. The only context manipulation is the two goal values in the three-question validity test (Figure 2); in the CAUS and SQUARE applications the answerer and goal variables are fixed within each dataset. No evaluation shows how scores behave across a range of answerer/goal values, and no reliability analysis of the semi-adaptive criteria is presented. Please either provide systematic context-variation experiments or soften the claim to match the actual scope of the demonstration.
minor comments (5)
  1. [Abstract] The abstract says the framework can "access both well-formed and problematic questions"; this should be "assess".
  2. [Table 1] The definition of effectiveness contains the typo "sucessfully"; it should be "successfully."
  3. [Methods and Results] The dataset name is given as "SQAURE" in the Methods and Results but as "SQuARe" in the reference list and "SQUARE" elsewhere; please standardize the spelling.
  4. [Methods, Evaluation Method] "maximum token of 1500" should be "maximum tokens of 1500," and the sampling procedure for the 150-question subsets (e.g., random seed, inclusion criteria) should be reported.
  5. [Footnotes] The footnote numbering is inconsistent: Footnote 1 for the speech-act definition appears in the Introduction, but footnotes 3-5 are mentioned in the Methods without corresponding numbered markers in the text; please align the footnote markers.

Circularity Check

2 steps flagged · score 4.0 of 10

The validity test is self-definitional and the CAUS validation set is self-authored, but the rubric itself has independent theoretical content.

  1. self definitional [Validity Test of the Evaluation Metric (Figures 1-2); rubric anchors in Table 2]
    "We then created two invalid versions of questions, 1)“FQ#1” misleads the user’s intent by asking about ownership changes - while professionally courteous, it ineffectively diverts from the original purpose. 2)“FQ#2” completely leads the context to personal/social matters, neither clarifying nor contributing to the inquiry."

    The two 'invalid' questions are labeled invalid by the same effectiveness/appropriateness notions the rubric operationalizes: FQ#1 'diverts from the original purpose' and FQ#2 'neither clarifying nor contributing' map directly onto Table 2 anchors such as Informativeness=1 'Seeks irrelevant or speculative information, hindering ${goal}' and Coherence=1 'Irrelevant to topic with unclear purpose.' Because the LLM judge is prompted with that rubric, the observed low scores are a direct consequence of the rubric definitions rather than evidence from an independent criterion; the validity test cannot fail if the judge follows its instructions.

  2. self citation load bearing [Methods, CAUS Dataset; Results, Applying to Relevant and Effective Questions; Abstract]
    "The CAUS dataset (Shin, Kim, & Ryu, 2024) was developed by generating questions that could arise in uncertain scenes using LLMs. As the questions were generated using a chain of thought approach (Wei et al., 2022) aimed at resolving uncertainty, they formed a logical and straightforward question set."

    CAUS is the authors' own prior dataset, as Shin, Kim, and Ryu are the present authors. The paper then sets the evaluation goal to 'resolving uncertainty by acquiring useful information,' matching the dataset's generation goal. The high CAUS scores (e.g., Clarity 5.00, Coherence 5.00 for the first set) are therefore partly built into the data-generation objective, so CAUS provides limited independent support for the abstract's claim that the framework was 'validated using the CAUS and SQUARE datasets, demonstrating the ability ... to access both well-formed and problematic questions.'

full rationale

The core framework is definitional and theory-driven rather than fitted to outcome data: the paper defines a good question as both appropriate and effective, and no parameter is fitted to a target and then re-predicted. However, the paper's validity demonstration is self-definitional because the 'invalid' questions were constructed by the authors using the same criteria that are encoded verbatim in the rubric's low-anchor descriptions, so the observed score separation is an inevitable consequence of the rubric rather than an external confirmation. In addition, the well-formed side of the validation relies heavily on CAUS, a dataset created by the same three authors with a generation goal that matches the rubric's ${goal} variable, reducing independence. The SQUARE dataset is external and provides some independent content, and the rubric itself is grounded in communication-competence theory, so the central claim is not wholly circular. The unverified claim about the LLM judge's agreement with human evaluators is a serious evidence gap but is a validity concern, not a circularity, and is therefore not scored here.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The framework rests on hand-designed rubric criteria and context placeholders rather than on fitted parameters. The main burden is the unvalidated assumption that LLM rubric scores correspond to human judgments.

assumptions (4)
  • domain assumption Question quality is exhaustively decomposable into appropriateness and effectiveness, each with the three listed subcomponents.
    Introduced in the Metric Development section without empirical or theoretical proof. The grouping of Cohesion, Answerability, and Respectfulness under appropriateness and Clarity, Coherence, and Informativeness under effectiveness is a modeling choice.
  • domain assumption An LLM applying the rubric at temperature 0 produces scores that reflect human judgments of question quality.
    The Evaluation Method section relies on claude-3-5-sonnet-as-judge. The claimed human agreement is not quantified, so this is a load-bearing assumption.
  • domain assumption The CAUS and SQUARE datasets provide representative contexts for validating question evaluation.
    CAUS is the authors' own uncertainty-question dataset and SQUARE is a safety benchmark. Both are convenience samples, and only 150 questions from each were used.
  • domain assumption Speech act theory and the formal/functional competence distinction justify evaluating questions through pragmatic criteria rather than truth values.
    Searle, Huddleston, and Mahowald et al. are cited as background, but their applicability to LLM output is assumed rather than demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of "There Is No Such Thing as a Dumb Question," But There Are Good Ones." pith.science (2026). https://pith.science/paper/OUGGBL5G

@misc{pith2026250509923,
  author       = {Pith},
  title        = {Pith review of: "There Is No Such Thing as a Dumb Question," But There Are Good Ones},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OUGGBL5G}},
  note         = {Machine review of arXiv:2505.09923}
}
read the original abstract

Questioning has become increasingly crucial for both humans and artificial intelligence, yet there remains limited research comprehensively assessing question quality. In response, this study defines good questions and presents a systematic evaluation framework. We propose two key evaluation dimensions: appropriateness (sociolinguistic competence in context) and effectiveness (strategic competence in goal achievement). Based on these foundational dimensions, a rubric-based scoring system was developed. By incorporating dynamic contextual variables, our evaluation framework achieves structure and flexibility through semi-adaptive criteria. The methodology was validated using the CAUS and SQUARE datasets, demonstrating the ability of the framework to access both well-formed and problematic questions while adapting to varied contexts. As we establish a flexible and comprehensive framework for question evaluation, this study takes a significant step toward integrating questioning behavior with structured analytical methods grounded in the intrinsic nature of questioning.

Figures

Figures reproduced from arXiv: 2505.09923 by the authors.

Figure 1
Figure 1. Original script for validity test (FQ: Follow-up Ques [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Result of the validity test. Colored lines represent [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Evaluation of the SQUARE dataset. The three radial graphs show the re￾sults of the analysis for 50 questions each: (Left) con￾tentious questions, (Middle) questions asking for ethical judgments, (Right) predic￾tive questions [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 30 canonical work pages

  1. [1]

    write newline

    " write newline " cite write " FUNCTION editor.postfix editor num.names #1 > "( )" "( )" if FUNCTION editor.trans.postfix editor num.names #1 > "( )" "( )" if FUNCTION trans.postfix translator num.names #1 > "( )" "( )" if FUNCTION authors.editors.reflist.apa5 'field := 'dot := field num.names 'numnames := numnames 'format.num.names := format.num.names na...

  2. [2]

    , Olah, C

    amodei2016concrete APACrefauthors Amodei, D. , Olah, C. , Steinhardt, J. , Christiano, P. , Schulman, J. \ Man \'e , D. APACrefauthors \ 2016 . Concrete problems in AI safety Concrete problems in ai safety . arXiv preprint arXiv:1606.06565

    arXiv 2016

  3. [3]

    \ Lavie, A

    banerjee2005meteor APACrefauthors Banerjee, S. \ Lavie, A. APACrefauthors \ 2005 . METEOR: An automatic metric for MT evaluation with improved correlation with human judgments Meteor: An automatic metric for mt evaluation with improved correlation with human judgments . Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Mac...

    work page 2005

  4. [4]

    , Gebru, T

    Bender2021parrots APACrefauthors Bender, E M. , Gebru, T. , McMillan-Major, A. \ Shmitchell, S. APACrefauthors \ 2021 . On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency Proceedings ...

    arXiv 2021

  5. [5]

    \ Koller, A

    bender2020climbing APACrefauthors Bender, E M. \ Koller, A. APACrefauthors \ 2020 07 . Climbing towards NLU : On Meaning, Form, and Understanding in the Age of Data Climbing towards NLU : On meaning, form, and understanding in the age of data . D. Jurafsky, J. Chai, N. Schluter \ J. Tetreault\ ( ), Proceedings of the 58th Annual Meeting of the Association...

    2020

  6. [6]

    APACrefauthors \ 2018

    Brookhart2018rubric APACrefauthors Brookhart, S M. APACrefauthors \ 2018 . Appropriate Criteria: Key to Effective Rubrics Appropriate criteria: Key to effective rubrics . Frontiers in Education 3 1-12 . APACrefDOI doi:10.3389/feduc.2018.00022 APACrefDOI

    arXiv 2018

  7. [7]

    APACrefauthors \ 2014

    canale2014communicative APACrefauthors Canale, M. APACrefauthors \ 2014 . From communicative competence to communicative language pedagogy1 From communicative competence to communicative language pedagogy1 . Language and communication Language and communication \ ( \ 2--27). Routledge

    work page 2014

  8. [8]

    APACrefauthors \ 2020

    damassino2020QTT APACrefauthors Damassino, N. APACrefauthors \ 2020 . The questioning turing test The questioning turing test . Minds and Machines 30 4 563--587

    work page 2020

Show all 45 references
  1. [9]

    APACrefauthors \ 2006

    dillon2006effect APACrefauthors Dillon, J T. APACrefauthors \ 2006 . Effect of questions in education and other enterprises Effect of questions in education and other enterprises . Rethinking schooling Rethinking schooling \ ( \ 145--174). Routledge

    2006

  2. [10]

    \ Dietz, L

    farzi2024pencils APACrefauthors Farzi, N. \ Dietz, L. APACrefauthors \ 2024 . Pencils Down! Automatic Rubric-based Evaluation of Retrieve/Generate Systems Pencils down! automatic rubric-based evaluation of retrieve/generate systems . Proceedings of the 2024 ACM SIGIR Internati...

    2024

  3. [11]

    APACrefauthors \ 1981

    flammer1981towards APACrefauthors Flammer, A. APACrefauthors \ 1981 . Towards a theory of question asking Towards a theory of question asking . Psychological Research 43 4 407--420

    1981

  4. [12]

    , Ozuru, Y

    Graesser2009whatGQ APACrefauthors Graesser, A. , Ozuru, Y. \ Sullins, J. APACrefauthors \ 2009 . What is a good question? What is a good question? M G. McKeown\ L. Kucan\ ( ), Bringing reading research to life Bringing reading research to life \ ( \ 170--193). New York, NY Gui...

    2009

  5. [13]

    APACrefauthors \ 1985

    graesser1985introduction APACrefauthors Graesser, A C. APACrefauthors \ 1985 . An introduction to the study of questioning An introduction to the study of questioning . A C. Graesser\ J B. Black\ ( ), The psychology of questions The psychology of questions \ ( \ 1--14). London...

    1985

  6. [14]

    \ Black, J B

    graesser1985psychology APACrefauthors Graesser, A C. \ Black, J B. APACrefauthors \ ( ). \ 1985 . The Psychology of Questions The psychology of questions . Lawrence Erlbaum Associates, Inc

    1985

  7. [15]

    , Millis, K K

    graesser1997discourse APACrefauthors Graesser, A C. , Millis, K K. \ Zwaan, R A. APACrefauthors \ 1997 . Discourse comprehension Discourse comprehension . Annual review of psychology 48 1 163--189

    1997

  8. [16]

    , Person, N

    graesser1992mechanisms APACrefauthors Graesser, A C. , Person, N. \ Huber, J. APACrefauthors \ 1992 . Mechanisms that generate questions Mechanisms that generate questions . T W. Lauer, E. Peacock \ A C. Graesser\ ( ), Questions and information systems Questions and informatio...

    1992

  9. [17]

    APACrefauthors \ 2002

    Huddleston2002ClauseType APACrefauthors Huddleston, R. APACrefauthors \ 2002 . Clause type and illocutionary force Clause type and illocutionary force . R. Huddleston\ G K. Pullum\ ( ), The Cambridge Grammar of the English Language The cambridge grammar of the english language...

    2002

  10. [18]

    , Zhang, Z

    Jain2017CVPR APACrefauthors Jain, U. , Zhang, Z. \ Schwing, A G. APACrefauthors \ 2017 July . Creativity: Generating Diverse Questions Using Variational Autoencoders Creativity: Generating diverse questions using variational autoencoders . Proceedings of the IEEE Conference on...

    2017

  11. [19]

    , Lyu, C

    ji2022qascore APACrefauthors Ji, T. , Lyu, C. , Jones, G. , Zhou, L. \ Graham, Y. APACrefauthors \ 2022 . QAScore—an unsupervised unreferenced metric for the question generation evaluation Qascore—an unsupervised unreferenced metric for the question generation evaluation . Ent...

    2022 doi

  12. [20]

    kim2022prosocialdialog APACrefauthors Kim, H. , Yu, Y. , Jiang, L. , Lu, X. , Khashabi, D. , Kim, G. Sap, M. APACrefauthors \ 2022 12 . P rosocial D ialog: A Prosocial Backbone for Conversational Agents P rosocial D ialog: A prosocial backbone for conversational agents . Y. Go...

    2022

  13. [21]

    , Ullman, T D

    lake2017building APACrefauthors Lake, B M. , Ullman, T D. , Tenenbaum, J B. \ Gershman, S J. APACrefauthors \ 2017 . Building machines that learn and think like people Building machines that learn and think like people . Behavioral and brain sciences 40 e253 . APACrefDOI doi:1...

    2017 doi

  14. [22]

    , Hong, S

    lee2023square APACrefauthors Lee, H. , Hong, S. , Park, J. , Kim, T. , Cha, M. , Choi, Y. Ha, J W. APACrefauthors \ 2023 07 . SQ u AR e: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created through Human-Machine Collaboration SQ u AR e: A large-scale d...

    2023

  15. [23]

    , Jiang, B

    li2024llmjudge APACrefauthors Li, D. , Jiang, B. , Huang, L. , Beigi, A. , Zhao, C. , Tan, Z. others APACrefauthors \ 2024 . From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge From generation to judgment: Opportunities and challenges of llm-as-a-judge ...

    2024

  16. [24]

    APACrefauthors \ 2004

    lin2004rouge APACrefauthors Lin, C Y. APACrefauthors \ 2004 . Rouge: A package for automatic evaluation of summaries Rouge: A package for automatic evaluation of summaries . Proc. of Workshop on Text Summarization Branches Out, Post Conference Workshop of ACL 2004 Proc. of wor...

    2004

  17. [25]

    , Ivanova, A A

    mahowald2024dissociating APACrefauthors Mahowald, K. , Ivanova, A A. , Blank, I A. , Kanwisher, N. , Tenenbaum, J B. \ Fedorenko, E. APACrefauthors \ 2024 . Dissociating language and thought in large language models Dissociating language and thought in large language models . ...

    2024

  18. [26]

    \ Davis, E

    marcus2020gpt APACrefauthors Marcus, G. \ Davis, E. APACrefauthors \ 2020 . GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about Gpt-3, bloviator: Openai’s language generator has no idea what it’s talking about . Technology Review 294

    2020

  19. [27]

    \ Gharpure, P

    mulla2023automatic APACrefauthors Mulla, N. \ Gharpure, P. APACrefauthors \ 2023 . Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications Automatic question generation: a review of methodologies, datasets, evaluation metrics, a...

    2023 doi

  20. [28]

    oh2023evaluation APACrefauthors Oh, S. , Go, H. , Moon, H. , Lee, Y. , Jeong, M. , Lee, H S. \ Choi, S. APACrefauthors \ 2023 jul . Evaluation of Question Generation Needs More References Evaluation of question generation needs more references . Findings of the Association for...

    2023 doi

  21. [29]

    , Roukos, S

    papineni2002bleu APACrefauthors Papineni, K. , Roukos, S. , Ward, T. \ Zhu, W J. APACrefauthors \ 2002 . Bleu: A method for automatic evaluation of machine translation Bleu: A method for automatic evaluation of machine translation . Proceedings of the 40th Annual Meeting of th...

    2002

  22. [30]

    , Cacioppo, J T

    petty1981effects APACrefauthors Petty, R E. , Cacioppo, J T. \ Heesacker, M. APACrefauthors \ 1981 . Effects of rhetorical questions on persuasion: A cognitive response analysis. Effects of rhetorical questions on persuasion: A cognitive response analysis. Journal of personali...

    1981

  23. [31]

    APACrefauthors \ 1997

    popham1997rubric3-5 APACrefauthors Popham, W J. APACrefauthors \ 1997 . What's wrong-and what's right-with rubrics What's wrong-and what's right-with rubrics . Educational leadership 55 72--75

    1997

  24. [32]

    , Zhang, Y

    qi2020stay APACrefauthors Qi, P. , Zhang, Y. \ Manning, C D. APACrefauthors \ 2020 nov . Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations Stay hungry, stay focused: Generating informative and specific questions in in...

    2020

  25. [33]

    \ Di Vesta, F J

    rickards1974type APACrefauthors Rickards, J P. \ Di Vesta, F J. APACrefauthors \ 1974 . Type and frequency of questions in processing textual material. Type and frequency of questions in processing textual material. Journal of Educational Psychology 66 3 354 . APACrefDOI doi:1...

    1974 doi

  26. [34]

    , Lake, B M

    Rothe2017NIPS APACrefauthors Rothe, A. , Lake, B M. \ Gureckis, T. APACrefauthors \ 2017 . Question Asking as Program Generation Question asking as program generation . I. Guyon\ . \ ( ), Advances in Neural Information Processing Systems Advances in neural information processi...

    2017

  27. [35]

    , Lake, B M

    rothe2018people APACrefauthors Rothe, A. , Lake, B M. \ Gureckis, T M. APACrefauthors \ 2018 . Do people ask good questions? Do people ask good questions? Computational Brain & Behavior 1 1 69--89 . APACrefDOI doi:10.1007/s42113-018-0005-5 APACrefDOI

    2018 doi

  28. [36]

    APACrefauthors \ 2011

    sagan2011demon APACrefauthors Sagan, C. APACrefauthors \ 2011 . The demon-haunted world: Science as a candle in the dark The demon-haunted world: Science as a candle in the dark . Ballantine books

    2011

  29. [37]

    APACrefauthors \ 1969

    searle1969speech APACrefauthors Searle, J R. APACrefauthors \ 1969 . Speech acts: An essay in the philosophy of language Speech acts: An essay in the philosophy of language . London Cambridge U.P

    1969

  30. [38]

    , Jang, M

    shin2023uncertainty APACrefauthors Shin, M. , Jang, M. , Cho, M. \ Ryu, J K. APACrefauthors \ 2023 . Uncertainty-Resolving Questions for Social Robots Uncertainty-resolving questions for social robots . Companion of the 2023 ACM/IEEE International Conference on Human-Robot Int...

    2023

  31. [39]

    , Kim, D

    shin2024caus APACrefauthors Shin, M. , Kim, D. \ Ryu, J K. APACrefauthors \ 2024 . CAUS: A Dataset for Question Generation based on Human Cognition Leveraging Large Language Models Caus: A dataset for question generation based on human cognition leveraging large language model...

    2024

  32. [40]

    , Canary, D J

    spitzberg1994competence APACrefauthors Spitzberg, B H. , Canary, D J. \ Cupach, W R. APACrefauthors \ 1994 . A competence-based approach to the study of interpersonal conflict A competence-based approach to the study of interpersonal conflict . Conflict in personal relationshi...

    1994

  33. [41]

    , Chandel, S

    sultan2020diverseQG APACrefauthors Sultan, M A. , Chandel, S. , Fernandez Astudillo, R. \ Castelli, V. APACrefauthors \ 2020 07 . On the Importance of Diversity in Question Generation for QA On the importance of diversity in question generation for QA . D. Jurafsky, J. Chai, N...

    2020

  34. [42]

    , Huang, Y

    toles2023good APACrefauthors Toles, M. , Huang, Y. , Yu, Z. \ Gravano, L. APACrefauthors \ 2023 . What is a good question? Task-oriented asking with fact-level masking What is a good question? task-oriented asking with fact-level masking . arXiv preprint arXiv:2310.11571

    2023 arXiv

  35. [43]

    , Wang, X

    wei2022chain APACrefauthors Wei, J. , Wang, X. , Schuurmans, D. , Bosma, M. , Ichter, B. , Xia, F. Zhou, D. APACrefauthors \ 2022 . Chain-of-thought prompting elicits reasoning in large language models Chain-of-thought prompting elicits reasoning in large language models . Pro...

    2022

  36. [44]

    , Kim, D

    ye2024flask APACrefauthors Ye, S. , Kim, D. , Kim, S. , Hwang, H. , Kim, S. , Jo, Y. Seo, M. APACrefauthors \ 2024 . FLASK : Fine-grained Language Model Evaluation based on Alignment Skill Sets FLASK : Fine-grained language model evaluation based on alignment skill sets . ICLR...

    2024

  37. [45]

    \ Bak, J

    yoon2023storybook APACrefauthors Yoon, H. \ Bak, J. APACrefauthors \ 2023 12 . Diversity Enhanced Narrative Question Generation for Storybooks Diversity enhanced narrative question generation for storybooks . H. Bouamor, J. Pino \ K. Bali\ ( ), Proceedings of the 2023 Conferen...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.