Pith. sign in

REVIEW 4 major objections 4 minor 13 references

MLD-EA: Check and Complete Narrative Coherence by Introducing Emotions and Actions

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a large language model trained to track characters' emotions and actions can locate the position of a deleted sentence in a short story and generate a coherent replacement, outperforming all prompting baselines.

desk verdict A clearly described, honest pipeline for a genuinely new task, but the headline F1 is mostly the model replicating the emotion-change rule that defined the ground truth. read the letter →

arxiv 2412.02897 v1 pith:WFMBALDZ submitted 2024-12-03 cs.CL cs.AI

Jinming Zhang , Yunfei Long This is my paper · ORCID
classification cs.CLcs.AI
keywords narrativecoherencestorygenerationemotionclassificationactionabstractionmissingsentencedetectionlargelanguagemodelsfine-tuningPlutchikemotions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces MLD-EA, a model that uses large language models to detect and repair gaps in narrative logic. Its central claim is that annotating each sentence with characters' emotions and actions, then fine-tuning an LLM on those annotations, lets the model identify the position from which a sentence was deleted in a five-sentence story and write a replacement that restores emotional and logical continuity. On the missing-sentence-position task it reports a micro-averaged F1 of 76.04, against 38.39 for the best prompting baseline. A reader should care because story generation systems still struggle to check their own coherence, and this work offers a mechanism for that check grounded in observable actions and emotions.

What carries the argument

The central machinery is the emotion–action abstraction pipeline: action abstraction writes each character's behavior as <Character>Action(Target, Object)</Character>, and emotion classification assigns one of Plutchik's eight emotions plus 'none' per character per sentence. The narrative logic checker then looks for discontinuity in these sequences, informed by the psychological link that emotions drive actions and actions alter emotions. The paper defines the training target in Appendix A: the missing sentence is the one whose removal maximizes aggregate emotional distance across characters (equations 6–7), so the model's logic-gap prediction is tied to that operationalization.

What would settle it

Construct test stories where a sentence is deleted because a physical action or causal connection is missing but the surrounding emotions stay nearly flat, and check whether MLD-EA locates those gaps above chance; if it does not, its reported F1 reflects the emotion-change rule rather than narrative logic. A cleaner test is to compare MLD-EA's predicted missing positions against human-annotated logic gaps on held-out five-sentence stories.

Watch

Extended reading notes

Core claim

In the paper's terms, MLD-EA treats narrative logic as the continuity of each character's emotion–action chain. It first abstracts actions and classifies emotions (from Plutchik's eight basic types plus 'none') for every sentence, then infers the index k where a missing sentence should be inserted, predicts the actions and emotions that the missing sentence should contain, and finally generates the sentence. The empirical core is Table 3: with actions and emotions included, MLD-EA reaches 76.04 average F1 on predicting the missing index versus 38.39 for the best baseline, and the ablation in Table 6 shows removing emotions (−13.45 F1) or actions (−10.93 F1) lowers accuracy. The paper also reports that including predicted actions and emotions improves BLEU and ROUGE scores for generation and brings the generated sentence closer to the reference on valence–arousal–dominance measures.

Load-bearing premise

The load-bearing assumption is that a narrative logic gap is exactly the sentence whose removal causes the largest aggregate emotion change across characters, as defined by the rule in Appendix A; if real logic gaps differ from this operationalization, the model's high accuracy would not demonstrate narrative logic detection.

Editorial extensions

If this is right

  • If correct, a lightweight fine-tune on emotion and action annotations can turn an instruction-tuned LLM into a sentence-level story coherence checker, a task that prompting alone handles poorly.
  • The same emotion–action representation could be applied to longer narratives, since the mechanism does not by itself depend on the five-sentence format (the paper's limitation acknowledges this remains untested).
  • The generation module shows a concrete way to use predicted emotions and actions as a bridge between gap detection and gap filling, improving n-gram overlap with the original text.
  • The large gains from adding actions and emotions suggest these features carry coherence signal that is not accessible from surface text alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit is a direct comparison of MLD-EA's predicted missing positions against human-annotated logic gaps; the current labels are generated by the emotion-change rule, so the high F1 may partly reflect learning that rule.
  • A testable extension is to replace the emotion-distance rule with a richer coherence signal, such as missing causal links or contradictions, to see whether the same architecture detects other narrative defects.
  • Because Plutchik's emotions and basic action frames are not English-specific, the pipeline could transfer to other languages or domains, though the paper does not claim this.
  • The emotion classifier is trained on the same dataset that defines the missing-sentence labels, so evaluating on held-out stories with independently human-marked gaps would separate genuine logic detection from label-rule memorization.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces MLD-EA, an LLM-based model for detecting "narrative logic gaps" in short stories and generating sentences to fill those gaps. It uses the Story Commonsense dataset, constructs missing-sentence labels by identifying the sentence whose removal maximizes aggregate emotional change (Appendix A, Eqs. 6-7), then fine-tunes a Llama-3 model to predict that index using character emotion and action annotations. The model is also used to predict actions/emotions of the missing sentence and generate a filler. Experiments report high F1 (76.04) for gap position prediction, improvements from emotion/action features, and competitive BLEU/ROUGE/BERTScore in generation.

Significance. If the evaluation were sound, the approach would be a noteworthy application of emotion/action modeling to narrative coherence. The paper makes a concrete, reproducible task definition and provides detailed prompts and ablations. However, the central evaluation is confounded: the ground truth is defined by an emotion-discontinuity rule, and the model is trained and evaluated on exactly that rule. Consequently, the reported results do not support the claim that the model detects narrative logic gaps in a general sense. The contribution is instead a demonstration that a fine-tuned LLM can predict a hand-crafted emotion-change heuristic from emotion labels. Without human validation or an independent definition of narrative gaps, the paper's main claim is unsupported.

major comments (4)
  1. [Appendix A, Eqs. 6-7; Section 6.2, Table 3] The ground-truth missing position is defined in Appendix A as the sentence that maximizes aggregate emotional discontinuity (Eq. 7), and the narrative logic checker consumes character emotion labels and is fine-tuned to predict that index. The 76.04 average F1 in Table 3 therefore measures how accurately MLD-EA reproduces the Appendix A emotion-change rule, not whether the model identifies narrative gaps in any independent sense. No human annotation study or alternative test set is provided to validate that maximum-emotion-change positions correspond to narrative incoherence, so the abstract and Section 6.2 claims about 'narrative logic detection' and 'logical coherence' are not supported by the reported experiments.
  2. [Appendix A, Eq. 6] The emotion distance function d in Eq. 6 is never defined, and the condition for when D is nonzero is truncated ('if esi,c and esj ,c' with no predicate). Since every positive example in the training and test sets is generated by this formula, the dataset construction is irreproducible and the task itself is underspecified. This is a load-bearing issue because an arbitrary choice of d would change the ground-truth labels and, in turn, the reported F1 scores.
  3. [Section 6.3, Table 4] The generation task is scored against the single original deleted sentence as the reference. Because that sentence is selected by the emotion-change heuristic and many plausible sentences could fill the narrative gap, BLEU-1/2/4, ROUGE, and BERTScore against one reference are not valid measures of narrative coherence or even of generation quality. The absence of any human evaluation further weakens the claim that MLD-EA completes stories 'coherently both logically and emotionally' (Section 1).
  4. [Section 6.4, Table 6] The ablation study shows that removing emotions causes the largest F1 drop (13.45 points), which is exactly what one would expect if the model is reproducing the emotion-discontinuity rule that generated the labels. This result does not provide evidence for the general utility of actions and emotions in narrative logic checking; it is fully explained by the circularity of the evaluation design. A meaningful ablation would require an independent ground truth for what constitutes a missing or incoherent sentence.
minor comments (4)
  1. [Appendix A, Eq. 6] The condition for D = 0 is an incomplete sentence fragment ('if esi,c and esj ,c'); the intended predicate (e.g., both emotions being present and non-'none') should be stated explicitly.
  2. [Appendix C, Table 10] In the Gemma2-9B-it zero-shot row, the reported F1 of 6.49 for k=2 is inconsistent with the precision of 3.36 and recall of 1.00 (the harmonic mean of those values is 1.54), suggesting a numerical error that should be corrected.
  3. [Section 2] The related work does not discuss existing story-completion and cloze-style evaluations that also involve locating a missing sentence (e.g., ROCStories-based cloze tasks); discussing them would help position the claimed novelty of the 'narrative logic detection' task.
  4. [Throughout] There are numerous typographical and formatting issues, including 'border goals' in the abstract, 'SW AG' instead of 'SWAG' in Section 2, and an inconsistent citation for NPN (Bosselut et al., 2017 in the text versus Rashkin et al., 2018 in Table 2).

Circularity Check

2 steps flagged · score 7.0 of 10

The narrative-logic-checker benchmark defines a 'gap' as the maximum emotion-discontinuity point (Appendix A, Eq. 7) and then feeds the same emotion labels to the model, so the headline F1 measures reproduction of that heuristic rather than independent logic detection.

  1. self definitional [Appendix A, Eqs. (6)-(7); Section 5.1]
    "Given a sequence of emotions attributed to characters in a narrative, we determine where emotional changes are most pronounced. Specifically, we analyze the emotions expressed by each character at different steps, calculate the 'distance' between emotions in sentences, and identify the step where the aggregate emotional change across all characters is the greatest. ... Then we remove this simax from the original story."

    The gold labels for the 'narrative logic checker' are not independent human judgments of narrative incoherence; they are generated by Eq. (7), which deletes the sentence whose removal maximizes aggregate emotion change. The model is then given character emotion labels as input and is fine-tuned or prompted to output exactly this index. Therefore Table 3's F1 measures how well MLD-EA reproduces the emotion-discontinuity labeling rule, not whether it detects narrative gaps in any sense independent of that rule. The abstract's claim that the model can 'identify narrative gaps' is, by construction, equivalent to 'find the maximum emotion-change point.'

  2. fitted input called prediction [Section 4.3, Eqs. (2)-(3); Section 5.3; Table 3]
    "MLD-EA then predicts the missing sentence index k, which is determined by evaluating the continuity and logical consistency of the sequences with the interaction of characters' actions and emotions: (E, A) = sum_{s∈S,c∈C} [e(c, s), a(c, s)], k = InfIndex[(S ⊕ (E, A)), C]."

    The predictor consumes the same emotion/action annotation stream that was used in Appendix A to create the target index. Section 5.3 confirms that baselines and MLD-EA use 'original input labels from the dataset' when actions and emotions are included. Thus the 'With EA' variants are not performing an independent logic check; they are supervised or prompted to recover the Appendix A heuristic from the very features that defined the label. The ablation in Table 6, where removing emotions drops F1 by 13.45, further shows that the checker's score is driven by the emotion feature that generated the ground truth.

full rationale

The central evaluation of the 'narrative logic checker' is circular in benchmark construction: Appendix A defines every missing-sentence label by Eq. (7), deleting the sentence with maximum aggregate emotion discontinuity, and Section 4.3 then feeds the characters' emotion labels (plus extracted actions) to the model and asks it to output that same index. The 76.04 average F1 in Table 3 therefore quantifies agreement with the emotion-change heuristic that generated the labels, not detection of narrative gaps under any independent operationalization. The 'With EA' prompting baselines are handed the same original labels, which explains part of the improvement when emotions and actions are added. The paper offers no human validation, alternative test set, or analysis showing that the Appendix A heuristic corresponds to perceived narrative incoherence; the claimed ability to 'identify narrative gaps' and 'check logical coherence' reduces, for this dataset, to reproducing a maximum-emotion-change rule. Some surrounding components are independently evaluated: emotion classification is compared with annotated labels (Table 2), and generation is scored against original sentences via BLEU/ROUGE/BERTScore/VAD (Tables 4-5). The circularity is therefore concentrated in the headline logic-checker result rather than the entire paper, but it is load-bearing for the paper's main contribution. Score 7 reflects this central definitional reduction while acknowledging the non-circular auxiliary evaluations.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The paper contributes a pipeline and results, but its central evaluation leans on a self-defined gap heuristic. No new entities are introduced. The free parameters are not fitted to an external benchmark; they define the target.

free parameters (2)
  • emotion distance function d in Eq. 6 = unspecified
    The rule that decides which sentence is 'missing' depends on an unspecified distance function between emotion labels; different choices would change every ground-truth label.
  • emotion-change aggregation in Eq. 7 = argmax over sentences
    The missing sentence is defined as the maximum aggregate emotion change, with no threshold or validation against human judgments.
assumptions (5)
  • domain assumption Plutchik's eight basic emotions plus 'none' are sufficient to represent characters' narrative emotions (Section 4.2).
    Emotion classification is constrained to this taxonomy; narrative emotions outside it are collapsed or labeled none.
  • domain assumption Action-emotion interaction rules (emotions drive actions, actions influence emotions, actions reflect emotions) are sufficient to detect logical gaps (Section 4.3 and prompt in Table 13).
    The logic checker uses only these rules to compare chains; other sources of incoherence such as causal, temporal, or spatial gaps are not modeled.
  • ad hoc to paper The maximum aggregate emotion-change sentence is the missing sentence (Appendix A, Eq. 6-7).
    Ground-truth gaps are defined by this heuristic, not by human judgment of narrative logic.
  • domain assumption The first and last sentences of every story are always complete and never missing (Section 3).
    The model only outputs missing indices k in {2, 3, 4, -1}.
  • domain assumption Story Commonsense emotion labels are accurate enough to serve as supervision (Section 5.1).
    All emotion classification and logic checker training uses these labels without re-validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MLD-EA: Check and Complete Narrative Coherence by Introducing Emotions and Actions." pith.science (2026). https://pith.science/paper/WFMBALDZ

@misc{pith2026241202897,
  author       = {Pith},
  title        = {Pith review of: MLD-EA: Check and Complete Narrative Coherence by Introducing Emotions and Actions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WFMBALDZ}},
  note         = {Machine review of arXiv:2412.02897}
}
read the original abstract

Narrative understanding and story generation are critical challenges in natural language processing (NLP), with much of the existing research focused on summarization and question-answering tasks. While previous studies have explored predicting plot endings and generating extended narratives, they often neglect the logical coherence within stories, leaving a significant gap in the field. To address this, we introduce the Missing Logic Detector by Emotion and Action (MLD-EA) model, which leverages large language models (LLMs) to identify narrative gaps and generate coherent sentences that integrate seamlessly with the story's emotional and logical flow. The experimental results demonstrate that the MLD-EA model enhances narrative understanding and story generation, highlighting LLMs' potential as effective logic checkers in story writing with logical coherence and emotional consistency. This work fills a gap in NLP research and advances border goals of creating more sophisticated and reliable story-generation systems.

Figures

Figures reproduced from arXiv: 2412.02897 by the authors.

Figure 1
Figure 1. A task example. "Identify logical coherence [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. MLD-EA model overview. Each Input Story contains [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Emotion classification. This figure illustrates [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages

  1. [1]

    Love, Loved, Loves, See, Saw, Attack, Attacks, Attacked, Move, Moves, Moved, Move to, Come, Came, etc.)

    The action form looks like this: Action(Target, ActionObject), where (Action: The action performed by the character (i.e. Love, Loved, Loves, See, Saw, Attack, Attacks, Attacked, Move, Moves, Moved, Move to, Come, Came, etc.). Target: The target of the action (who or what the action is directed towards) (i.e. A Love B -> A Love(B)). ActionObject: The spec...

    work page

  2. [2]

    If ’none’ means the characters do not have a discernible emotion or will not appear at this point

    The emotions are ONLY from Plutchik’s eight basic emotions (joy, trust, fear, surprise, sadness, disgust, anger, anticipation) for the characters based on their likely emotional state based on the context and characters’ actions. If ’none’ means the characters do not have a discernible emotion or will not appear at this point. UserInput will provide a sto...

    work page

  3. [3]

    Love, Loved, Loves, See, Saw, Attack, Attacks, Attacked, Move, Moves, Moved, Move to, Come, Came, etc.)

    The action form looks like this: Action(Target, ActionObject), where (Action: The action performed by the character (i.e. Love, Loved, Loves, See, Saw, Attack, Attacks, Attacked, Move, Moves, Moved, Move to, Come, Came, etc.). Target: The target of the action (who or what the action is directed towards) (i.e. A Love B -> A Love(B)). ActionObject: The spec...

    work page

  4. [5]

    Examine the provided story and identify the events leading to the specified index; if the index is -1, no missing sentence needs to be generated here; stop responding and give ’none’

    work page

  5. [6]

    **Action Prediction**:

    Focus on the actions and emotions of the characters in the story to understand their progression. **Action Prediction**:

    work page

  6. [7]

    This prediction should be based on strong evidence from the surrounding context and reflect a logical progression in the narrative

    Predict the most likely action that would occur before the specified index. This prediction should be based on strong evidence from the surrounding context and reflect a logical progression in the narrative

    work page

  7. [8]

    The action should be in the open-text format and reflect what the character would logically do next based on previous actions, emotions and the situation

    work page

  8. [10]

    Assign an emotion ONLY from Plutchik’s eight basic emotions (joy, trust, fear, surprise, sadness, disgust, anger, anticipation) to the characters based on their likely emotional state based on the context and characters’ actions

    work page

Show all 13 references
  1. [11]

    **Reasoning**:

    If the characters do not have a discernible emotion or will not appear at this point, use ’none’. **Reasoning**:

  2. [12]

    Provide the predicted action and emotion for each character(s) that should appear in the missing sentence before the specified index

  3. [13]

    Table 15: Prompt template: Actions and Emotions Prediction

    Ensure that the predicted actions and emotions consistently follow logical flow. Table 15: Prompt template: Actions and Emotions Prediction

  4. [14]

    Also, Table 15 shows how we predict the actions and emotions for the missing part

    The prompt templates of ’With EA’ means the LLMs have to consider the characters’ emotions and actions during those tasks, which the zero-shot prompt template for the narrative logic checker is shown in Table 13 and the generation template is in Table 14. Also, Table 15 shows ...

  5. [2021]

    distance

    Natural language processing (nlp) based text summarization-a survey. In 2021 6th International Conference on Inventive Computation Technologies (ICICT), pages 1310–1317. IEEE. Albert Bandura. 1977. Social learning theory. Prentice- Hall google schola, 2:101–123. Antoine Bossel...

    2021 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.