Pith. sign in

REVIEW 3 major objections 6 minor 45 references

CoRE: Condition-based Reasoning for Identifying Outcome Variance in Complex Events

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By pairing story outcomes with counterfactual condition pairs, this paper shows that outcome-variant conditions—states whose alternatives would flip a goal's success—improve LLM outcome-validation accuracy by 2–6% macro F1 and can…

desk verdict Useful new dataset and tasks for condition-based outcome reasoning, but the central RQ3 claim is weakened by a circular label-construction pipeline. read the letter →

arxiv 2506.01253 v1 pith:6LH5CTYM submitted 2025-06-02 cs.CL

Sai Vallurupalli , Francis Ferraro This is my paper
classification cs.CL
keywords condition-basedreasoningoutcomevariancecounterfactualconditionslargelanguagemodelsnarrativeunderstandinggoalachievementvalidationimperfectiveparadox
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether knowing which latent conditions would change a story's outcome can help language models validate that outcome when the story context is incomplete. The authors combine two existing narrative-annotation datasets to define three relationships between conditions and outcomes—outcome-variant, outcome-invariant, and outcome-irrelevant—and evaluate six open and closed LLMs on generating, identifying, and using such conditions. Their central finding is that outcome-variant conditions (counterfactual pairs where one condition makes the outcome true and the other makes it false) improve outcome-validation accuracy by 2–6% macro F1 over outcome-invariant conditions, and can partially substitute for missing story context. They also find that models are much better at recognizing whether a condition is relevant than at judging whether it changes the outcome, and that larger models become more cautious (answer 'Unsure') when context is removed. The paper extends the same pipeline to news articles, where performance drops 2–18%, which the authors attribute to domain shift and pretraining memory of real events.

What carries the argument

The central machinery is the outcome relation taxonomy: a condition pair is outcome-variant if the outcome is true in the original story and false in the alternate story (or vice versa), outcome-invariant if the outcome's truth value is unchanged across the pair, and outcome-irrelevant if the condition has no bearing on the outcome. The paper also relies on the contrastive story-pair structure of PASTA (an original story and a minimally altered alternate story supporting a counterfactual state) combined with SAGA's per-story goal-achievement annotations, to assign these labels automatically. The three research questions then test LLMs on (RQ1) aspectual sensitivity via Dowty's imperfective paradox, (RQ2) generating and identifying outcome-variant pairs with single-step, standard chain-of-thought, and alternate chain-of-thought prompts, and (RQ3) using annotated or generated conditions to validate outcomes under full, partial, intent-only, and no-context settings.

What would settle it

Independently re-annotate a sample of condition pairs and outcomes with fresh annotators who do not see the SAGA goal-achievement scores, then re-run RQ3; if the macro-F1 gap between variant and invariant conditions vanishes or reverses, the result is an artifact of shared label construction.

Watch

Extended reading notes

Core claim

The paper's central claim is that explicit, outcome-variant conditions—pairs of contrastive states such as 'Sam trusts his Dad' versus 'Sam does not trust his Dad,' where the outcome is true under one and false under the other—carry information about goal achievement that language models can exploit to validate narrative outcomes, and that this information is most valuable precisely when the surrounding story context is missing or partial. Using 951 condition pairs built from the PASTA and SAGA datasets, the authors show that pairing an outcome-variant condition with story context raises macro-F1 outcome validation by 2–6% relative to using outcome-invariant conditions or context alone, and that the benefit persists—though attenuated—when models generate the conditions themselves rather than receiving crowd-annotated ones.

Load-bearing premise

The labels that say which conditions are outcome-variant are computed automatically by comparing goal-achievement scores across an original and an alternate story, and those same goal-achievement scores are converted into the outcome labels the models are asked to predict; if the story pairs differ in more than the intended state, or the goal-achievement labels are noisy, the reported usefulness of variant conditions may partly reflect label construction rather than an independent measure of causal influence.

Editorial extensions

If this is right

  • Outcome-variant conditions can substitute for missing narrative context in outcome validation, with macro F1 gains of 2–6% over outcome-invariant conditions across the evaluated models.
  • Models are consistently better at recognizing whether a condition is relevant to an outcome than at judging whether a contrastive pair actually flips the outcome, so counterfactual reasoning is the binding constraint for this approach.
  • Chain-of-thought prompting does not reliably improve identification of outcome-variant pairs; providing intermediate relevance labels as extra context can even hurt performance.
  • The usefulness of conditions is strongest in partial-context settings, while some larger models (GPT-4o, Llama-70BI, Mistral-7BI) become more cautious and answer 'Unsure' when context is removed.
  • Transferring the same condition-and-outcome pipeline to news stories yields a 2–18% drop in performance, which the authors attribute to domain complexity and models drawing on pretraining knowledge of real events.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 2–6% gain may be an upper bound, because the outcome-variant labels are computed automatically from the same goal-achievement scores that are converted into the outcome labels models are asked to predict; an independent re-annotation could shrink the apparent benefit.
  • The variant/invariant distinction suggests a cheap annotation scheme for counterfactual probing: any paired narratives with outcome labels could yield condition pairs without additional crowd work, which could be used to scale counterfactual reasoning benchmarks.
  • Outcome-variant conditions could double as contrastive explanations for model predictions—pointing to the specific state whose flip would change a model's verdict could support error analysis beyond simply validating outcomes.
  • A testable extension would apply the same condition-generation and outcome-validation pipeline to non-English narratives or to automatically generated counterfactual stories, to isolate whether the observed gains depend on the specific PASTA/SAGA story-pair structure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces CoRE, a Condition-based Reasoning framework built by augmenting PASTA and SAGA annotations. It defines conditions as outcome-variant, outcome-invariant, or outcome-irrelevant relative to a goal outcome, and studies three research questions: (RQ1) whether LLMs distinguish ongoing vs. completed outcome descriptions, (RQ2) whether models can generate and identify outcome-variant contrastive condition pairs, and (RQ3) whether outcome-variant conditions help models validate outcomes when story context is incomplete. The authors evaluate six open and closed LLMs on SAGA and news stories, and report that outcome validation improves by 2–6% macro F1 when outcome-variant conditions are used instead of outcome-invariant ones.

Significance. The central claim is potentially useful: if explicit outcome-changing conditions can substitute for missing narrative context in LLM outcome validation, this would have implications for explainable and counterfactual NLP. The paper contributes a new augmented dataset, a multi-model comparison across open and closed LLMs, a cross-domain news extension, and manual error analyses. However, the headline RQ3 result is undermined by a label-construction loop: the outcome-variant/invariant labels are derived from the same goal-achievement annotations that produce the outcome labels RQ3 asks models to predict. Until the condition-type labels and outcome labels are obtained independently, the empirical contribution is not sufficient to support the abstract's claim.

major comments (3)
  1. [Appendix A.5 (steps 4 and 6); §5.3, Table 18] The central claim that outcome-variant conditions improve outcome validation by 2–6% macro F1 is confounded by the way the two label sets are derived. Step 4 converts SAGA goal-achievement Likert scores into the True/False/Unsure outcome labels that RQ3 uses as gold labels; step 6 defines a condition pair as outcome-variant precisely when those same goal-achievement-derived labels differ across the original and alternate story, and outcome-invariant when they agree. The variant/invariant partition is therefore selected to maximize dependence between condition and outcome label. Under this construction, a model given a variant condition and asked to predict the outcome is being evaluated on a subset chosen to make the condition informative; the reported 2–6% gap is at least partly an artifact of the label construction rather than an independent measurement of the condition's ability to replace missing context. Additionally, step 3's automatic correction of goal-achievement scores on test/validation splits ties the two sides even more tightly. Please re-run RQ3 with outcome-variant/invariant labels obtained independently of the outcome labels (e.g., expert judgments of condition–outcome influence on held-out stories, or a separate outcome prediction dataset) and report the comparison under that protocol.
  2. [§5.3, Table 18; §6, Tables 19/22] The headline performance differences are reported without confidence intervals or significance tests, and the magnitude is small relative to per-model variance. For example, in Table 18 GPT-4o's full-context macro F1 is .66 with variant conditions and .64 with all conditions, while the baseline without a condition is .69; this does not support the claimed improvement, and the text's note that GPT-4o is an exception is insufficient. With six models and one constructed test set, differences of 2–8% are within plausible noise. Please provide bootstrap confidence intervals or paired significance tests (e.g., McNemar's test or paired bootstrap) for the variant-vs-invariant contrast, and either revise the claim or characterize precisely which models and contexts support it.
  3. [§5.2.1 (manual evaluation)] The RQ2 evaluation of generated condition pairs uses manual inspection of 100 pairs per model with a binary rubric, but no inter-annotator agreement is reported, and the rubric includes a Variant judgment that is likely as difficult for humans as it is for models. Since the same variant judgments feed the 'generated vs. annotated conditions' comparison in Fig. 4 and §6, the reliability of these manual labels is load-bearing for the cross-domain comparison. Please report Cohen's kappa or a similar measure over at least a double-annotated subset, and include the rubric guidelines used for the manual judgments.
minor comments (6)
  1. [§3] The text says '5-sentence ROC Story (Mostafazadeh et al., 2016)' but the resource is commonly called 'ROCStories'; please correct the name and citation style.
  2. [Appendix A.2] The sentence 'Run time for a model ranges from a .52 hours' appears to contain a typo; please clarify the intended runtime range.
  3. [Table 18] The column headers 'Baseline #1 +Condition' are ambiguous; it would be clearer to separate the baseline macro F1 from the +Condition macro F1 in distinct subcolumns or to explicitly define the ordering in the caption.
  4. [§4.1] When introducing the four condition categories, the paper says it expands the 3 categories used in PASTA, but the original three categories are never defined; please briefly describe them for context.
  5. [Abstract and §5.3] The claim 'conditions are useful when not all context is available' is not precisely operationalized: the paper compares several context settings, but there is no single policy for when a condition should be considered 'useful' versus 'not useful.' Please state the exact comparison rule used to substantiate this claim.
  6. [§8 Limitations] The limitations section does not mention the circular derivation of outcome-variant labels relative to the outcome labels used in RQ3; adding an explicit discussion of this issue would strengthen the paper's transparency.

Circularity Check

1 steps flagged · score 6.0 of 10

RQ3's 2-6% variant-over-invariant benefit is forced by constructing variant labels from the same goal-achievement outcomes that define the prediction targets.

  1. self definitional [Appendix A.5, step 6 with step 4, evaluated in §5.3 and Table 18]
    "6. We automatically identify if a pair of contrastive conditions are outcome- variant, invariant or unsure using the goal achievement annotations from the two stories. ... 4. We convert the 5-point likert scale to the 3 outcome labels of True/False/Unsure (Fully & Moderately Successful labels are converted to an outcome label of true and Less & UnSuccessful labels are converted to an outcome label of false)."

    The outcome-variant/invariant partition (A.5 step 6) is computed from SAGA goal-achievement annotations across the original and alternate story; those same annotations are converted (A.5 step 4) into the True/False/Unsure outcome labels that RQ3 asks models to predict. A pair is therefore labeled variant exactly when its gold outcome label flips across the story pair, and invariant when the gold label stays fixed. Giving a model the condition text and asking it to predict the gold label mechanically favors variant pairs, because that subset was selected to maximize outcome-label dependence. The 2-6% macro-F1 advantage is thus an artifact of label construction, not an independent measurement that conditions replace missing context.

full rationale

The paper's RQ1 and RQ2 analyses are largely self-contained benchmarks: RQ1 tests aspectual understanding against linguistic gold labels, and RQ2 evaluates generation and identification of variant/invariant pairs against manual rubrics and gold labels; those are legitimate prediction tasks. The circularity is concentrated in RQ3 (Section 5.3 and the cross-domain Section 6), where the central quantitative claim is that outcome-variant conditions improve outcome validation by 2-6% macro F1 over invariant conditions. Because Appendix A.5 derives both the gold outcome labels (step 4, via Likert-to-True/False/Unsure conversion) and the variant/invariant status (step 6, via comparing goal-achievement annotations between stories) from the same SAGA goal-achievement signal, the comparison is self-referential: a model using the condition to predict the outcome should trivially do better on pairs whose gold label differs by construction. The condition-only setting, in particular, cannot separate the condition's informativeness from the way the variant partition was built. The paper itself acknowledges noise in the automatic mapping (A.5 step 5) and in the goal-achievement annotations, but the core issue is not noise; it is that the predictor and the grouping variable share the same source annotation. Other contributions, such as the condition-category analysis and the news-domain transfer, retain some independent interest, so the circularity is partial rather than total. The appropriate score is 6, not 8 or 10, because the label-construction equivalence affects the headline variant-vs-invariant comparison but not the entire paper's task formulations.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central empirical contribution depends on a chain of hand-chosen annotation mappings (Likert conversion, support/oppose rules, variant/invariant definition) and domain assumptions about PASTA and SAGA. The most notable burden is that outcome-variant labels are constructed from the same outcome labels later used as prediction targets.

free parameters (5)
  • temperature = 0.6 for non-GPT models, 1 for GPT models
    Chosen by hand in A.3 to generate diverse conditions; affects all generation and classification outputs.
  • nucleus sampling top-p = 0.1 (top 10%)
    Chosen by hand in A.3; shapes output diversity and likely affects all reported F1 numbers.
  • number of in-context examples = 2 per label
    Chosen by hand in A.3 for the few-shot prompts in RQ2; results may be sensitive to this choice.
  • Likert-to-outcome-label mapping = Fully/Moderately Successful -> True; Less/Unsuccessful -> False; else Unsure
    Introduced in A.5 step 4 to convert SAGA's 5-point goal achievement scores into the three outcome labels used as gold labels.
  • RQ1 gold-label derivation rule = use the ongoing-action label when the completed-action label is true, otherwise use the completed-action label
    Defined in Section 5.1 to create gold labels for the imperfective paradox task; a hand-chosen rule that affects all RQ1 scores.
assumptions (4)
  • domain assumption PASTA alternate stories minimally alter the original story so that the only meaningful change is the perturbed state condition.
    The entire outcome-variant/invariant definition in Section 4.2 depends on comparing original and alternate stories as if the condition pair is the operative difference. PASTA's 'minimal alteration' is trusted rather than checked per instance.
  • domain assumption SAGA goal achievement annotations reliably indicate whether the outcome statement is true in a given story.
    Outcome labels in A.5 are converted directly from SAGA's 5-point goal achievement scores, and the same scores are used to define outcome-variant labels. No independent verification of goal-outcome validity is provided.
  • ad hoc to paper A condition's support/oppose relation to the outcome can be inferred automatically from SAGA sentence-goal relations, with the counterfactual condition assumed to have exactly the opposite relation.
    A.5 step 5 derives condition-outcome relations this way and the authors explicitly call it noisy. The assumption is load-bearing for the outcome-oriented relation annotations used in RQ2 and RQ3.
  • ad hoc to paper The same outcome label can serve both as the signal that defines whether a condition is outcome-variant and as the prediction target for evaluating whether that condition helps predict outcomes.
    This circularity is inherent in the dataset construction (A.5 steps 4 and 6) and central to the RQ3 evaluation design.
invented entities (1)
  • Outcome-variant condition
    purpose: A condition whose contrastive counterpart flips the truth value of a goal outcome; used to define new tasks and to evaluate LLM counterfactual reasoning.
    The label is constructed by comparing outcome labels on two alternate stories from the same dataset, not from an externally falsifiable source. It is a conceptual/annotation construct rather than an independently measured entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CoRE: Condition-based Reasoning for Identifying Outcome Variance in Complex Events." pith.science (2026). https://pith.science/paper/6LH5CTYM

@misc{pith2026250601253,
  author       = {Pith},
  title        = {Pith review of: CoRE: Condition-based Reasoning for Identifying Outcome Variance in Complex Events},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6LH5CTYM}},
  note         = {Machine review of arXiv:2506.01253}
}
read the original abstract

Knowing which latent conditions lead to a particular outcome is useful for critically examining claims made about complex event outcomes. Identifying implied conditions and examining their influence on an outcome is challenging. We handle this by combining and augmenting annotations from two existing datasets consisting of goals and states, and explore the influence of conditions through our research questions and Condition-based Reasoning tasks. We examine open and closed LLMs of varying sizes and intent-alignment on our reasoning tasks and find that conditions are useful when not all context is available. Models differ widely in their ability to generate and identify outcome-variant conditions which affects their performance on outcome validation when conditions are used to replace missing context. Larger models like GPT-4o, are more cautious in such less constrained situations.

Figures

Figures reproduced from arXiv: 2506.01253 by the authors.

Figure 1
Figure 1. Conditions and their relationship to an out [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our research questions explore outcome validation through our condition-based reasoning tasks. We [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 4
Figure 4. Model generated conditions are not as good as Human annotated conditions leading to lower per￾formance on outcome identification. We compare task settings with varying SAGA story contexts paired with an annotated or a generated condition. Results are ag￾gregated across all models. tences from which the condition was derived (Par SC+C, Par SC+GC), 3. indicating the participant’s intent of achieving their goal (Goal I… view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: Models are able to utilize conditions in addi [PITH_FULL_IMAGE:figures/full_fig_p008_3.png]
Figure 6
Figure 6. Figure 6: We compare the performance of outcome iden [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Models are able to utilize conditions in addition to the story context to improve performance when [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 36 canonical work pages

  1. [1]

    Malihe Alikhani and Matthew Stone. 2019. https://doi.org/10.18653/v1/W19-1806 caption as a coherence relation: Evidence and implications . In Proceedings of the Second Workshop on Shortcomings in Vision and Language, pages 58--67, Minneapolis, Minnesota. Association for Computational Linguistics

    work page doi:10.18653/v1/w19-1806 2019

  2. [2]

    Nadav Amir, Yael Niv, and Angela J Langdon. 2024. https://openreview.net/forum?id=MwYgU2eXIE States as goal-directed concepts: an epistemic approach to state-representation learning . In Reinforcement Learning Conference

    work page 2024

  3. [3]

    Filippos Bellos, Yayuan Li, Wuao Liu, and Jason Corso. 2024. https://aclanthology.org/2024.scalellm-1.3/ Can large language models reason about goal-oriented tasks? In Proceedings of the First edition of the Workshop on the Scaling Behavior of Large Language Models (SCALE-LLM 2024), pages 24--34, St. Julian ' s, Malta. Association for Computational Linguistics

    work page 2024

  4. [4]

    Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen-tau Yih, and Yejin Choi. 2020. Abductive commonsense reasoning. In Proceedings of the 8th International Conference on Learning Representations, ICLR 2020 - Addis Ababa, Ethiopia

    work page 2020

  5. [5]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    work page 2020

  6. [6]

    Ruth M.J Byrne. 2016. https://doi.org/10.1146/annurev-psych-122414-033249 Counterfactual thought. Annual Review of Psychology, 67:135–157

    work page doi:10.1146/annurev-psych-122414-033249 2016

  7. [7]

    Ruth MJ Byrne. 2019. Counterfactuals in explainable artificial intelligence (xai): Evidence from human reasoning. In IJCAI, pages 6276--6282. California, CA

    work page 2019

  8. [8]

    Ruth MJ Byrne and Alessandra Tasso. 1999. Deductive reasoning with factual, possible, and counterfactual conditionals. Memory & cognition, 27:726--740

    work page 1999

Show all 45 references
  1. [9]

    Gergely Csibra and György Gergely. 2007. https://doi.org/https://doi.org/10.1016/j.actpsy.2006.09.007 ‘obsessed with goals’: Functions and mechanisms of teleological interpretation of actions in humans . Acta Psychologica, 124(1):60--78. Becoming an Intentional Agent: Early De...

    2007 doi

  2. [10]

    Ruud Custers. 2023. https://doi.org/10.1037/mot0000306 Thoughts about actions and outcomes (and what they lead to) . Motivation Science, 9

    2023 doi

  3. [11]

    David R. Dowty. 1977. http://www.jstor.org/stable/25000028 Toward a semantic analysis of verb aspect and the english 'imperfective' progressive . Linguistics and Philosophy, 1(1):45--77

    1977

  4. [12]

    Yi Fang, Moxin Li, Wenjie Wang, Lin Hui, and Fuli Feng. 2025. https://aclanthology.org/2025.coling-main.703/ Counterfactual debating with preset stances for hallucination elimination of LLM s . In Proceedings of the 31st International Conference on Computational Linguistics, p...

    2025

  5. [13]

    Oswaldo Filho. 2012. https://doi.org/10.5007/1808-1711.2011v15n3p383 Goodman and parry on counterfactual . Principia: an international journal of epistemology, 15

    2012

  6. [14]

    Annemarie Friedrich, Nianwen Xue, and Alexis Palmer. 2023. https://doi.org/10.18653/v1/2023.eacl-main.44 A kind introduction to lexical and grammatical aspect, with a survey of computational approaches . In Proceedings of the 17th Conference of the European Chapter of the Asso...

    2023 doi

  7. [15]

    Sayontan Ghosh, Mahnaz Koupaee, Isabella Chen, Francis Ferraro, Nathanael Chambers, and Niranjan Balasubramanian. 2023. https://doi.org/10.1162/tacl_a_00600 PASTA : A dataset for modeling PA rticipant STA tes in narratives . Transactions of the Association for Computational Li...

    2023 doi

  8. [16]

    T. Givón. 1992. https://doi.org/doi:10.1515/ling.1992.30.1.5 The grammar of referential coherence as mental processing instructions . Linguistics, 30(1):5--56

    1992 doi

  9. [17]

    Nelson Goodman. 1947. http://www.jstor.org/stable/2019988 The problem of counterfactual conditionals . The Journal of Philosophy, 44(5):113--128

    1947

  10. [18]

    William Hart and Dolores Albarrac \' n. 2011. Learning about what others were doing: Verb aspect and attributions of mundane and criminal intent for past actions. Psychological Science, 22(2):261--266

    2011

  11. [19]

    Bernhard Hommel, Jochen Müsseler, Gisa Aschersleben, and Wolfgang Prinz. 2001. https://doi.org/10.1017/S0140525X01000103 The theory of event coding (tec): A framework for perception and action planning . The Behavioral and brain sciences, 24:849--78; discussion 878

    2001 doi

  12. [20]

    Tianyu Jiang and Ellen Riloff. 2018. https://doi.org/10.18653/v1/P18-1120 Learning prototypical goal activities for locations . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1297--1307, Melbourne, Aust...

    2018 doi

  13. [21]

    Thomas Kober, Malihe Alikhani, Matthew Stone, and Mark Steedman. 2020. https://doi.org/10.18653/v1/2020.coling-main.401 Aspectuality across genre: A distributional semantics approach . In Proceedings of the 28th International Conference on Computational Linguistics, pages 4546...

    2020 doi

  14. [22]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013 ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  15. [23]

    Carol J Madden and Rolf A Zwaan. 2003. How does verb aspect constrain event representations? Memory & cognition, 31(5):663--672

    2003

  16. [24]

    Joseph P Magliano and Michelle C Schleich. 2000. Verb aspect and situation models. Discourse processes, 29(2):83--112

    2000

  17. [25]

    Rolison, Marta Stragà, Donatella Ferrante, Clare R

    Hugo Mercier, Jonathan J. Rolison, Marta Stragà, Donatella Ferrante, Clare R. Walsh, and Vittorio Girotto. 2017. https://doi.org/10.3758/s13421-016-0660-5 Questioning the preparatory function of counterfactual thinking . Memory and Cognition

    2017 doi

  18. [26]

    Marc Moens and Mark Steedman. 1988. https://aclanthology.org/J88-2003/ Temporal ontology and temporal reference . Computational Linguistics, 14(2):15--28

    1988

  19. [27]

    Nasrin Mostafazadeh, Lucy Vanderwende, Wen-tau Yih, Pushmeet Kohli, and James Allen. 2016. https://doi.org/10.18653/v1/W16-2505 Story cloze evaluator: Vector space representation evaluation by predicting what happens next . In Proceedings of the 1st Workshop on Evaluating Vect...

    2016 doi

  20. [28]

    Yael Niv. 2019. Learning task-state representations. In Nature neuroscience, pages 1544--1553

    2019

  21. [29]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022 arXiv

  22. [30]

    Katarzyna Pru \'s , Mark Steedman, and Adam Lopez. 2024. https://aclanthology.org/2024.eacl-long.115 Human temporal inferences go beyond aspectual class . In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: ...

    2024

  23. [31]

    Lianhui Qin, Antoine Bosselut, Ari Holtzman, Chandra Bhagavatula, Elizabeth Clark, and Yejin Choi. 2019. https://doi.org/10.18653/v1/D19-1509 Counterfactual story reasoning and generation . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processi...

    2019 doi

  24. [32]

    Elahe Rahimtoroghi, Jiaqi Wu, Ruimin Wang, Pranav Anand, and Marilyn Walker. 2017. https://doi.org/10.18653/v1/W17-5543 Modelling protagonist goals and desires in first-person narrative . In Proceedings of the 18th Annual SIG dial Meeting on Discourse and Dialogue , pages 360-...

    2017 doi

  25. [33]

    Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A

    Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith, and Yejin Choi. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.418 Thinking like a skeptic: Defeasible inference in natural language . In Findings of the As...

    2020 doi

  26. [34]

    Meghan M Salomon, Joseph P Magliano, and Gabriel A Radvansky. 2013. Verb aspect and problem solving. Cognition, 128(2):134--139

    2013

  27. [35]

    Siegel and Kathleen R

    Eric V. Siegel and Kathleen R. McKeown. 2000. https://aclanthology.org/J00-4004/ Learning methods to combine linguistic indicators:improving aspectual classification and revealing linguistic insights . Computational Linguistics, 26(4):595--627

    2000

  28. [36]

    Carlota S. Smith. 1983. http://www.jstor.org/stable/413899 A theory of aspectual choice . Language, 59(3):479--501

    1983

  29. [37]

    Carlota S. Smith. 1999. http://www.jstor.org/stable/25001754 Activities: States or events? Linguistics and Philosophy, 22(5):479--508

    1999

  30. [38]

    Niket Tandon, Bhavana Dalvi, Keisuke Sakaguchi, Peter Clark, and Antoine Bosselut. 2019. https://doi.org/10.18653/v1/D19-1629 WIQA : A dataset for what if... reasoning over procedural text . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Process...

    2019 doi

  31. [39]

    Sai Vallurupalli, Katrin Erk, and Francis Ferraro. 2024. https://aclanthology.org/2024.findings-acl.910 SAGA : A participant-specific examination of story alternatives and goal applicability for a deeper understanding of complex events . In Findings of the Association for Comp...

    2024

  32. [40]

    Zeno Vendler. 1957. http://www.jstor.org/stable/2182371 Verbs and times . The Philosophical Review, 66(2):143--160

    1957

  33. [41]

    Chi, Quoc Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. 2022. http://arxiv.org/abs/2201.11903 Chain of thought prompting elicits reasoning in large language models . CoRR, abs/2201.11903

    2022 arXiv

  34. [42]

    Li Zhang, Qing Lyu, and Chris Callison-Burch. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.374 Reasoning about goals, steps, and temporal ordering with W iki H ow . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 46...

    2020 doi

  35. [43]

    Kaitlyn Zhou, Dan Jurafsky, and Tatsunori Hashimoto. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.335 Navigating the grey area: How expressions of uncertainty and overconfidence affect language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural...

    2023 doi

  36. [44]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.