Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech

Gopala Anumanchipalli; Henry Hong; Jiachen Lian; Xinyi Yang; Xuanru Zhou

REVIEW 4 major objections 6 minor 11 references

Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech

T0 review · 4 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A causal graph over four speech states steers speech language models toward human-aligned reasoning, cutting hallucination and matching far larger systems at a fraction of the training cost.

desk verdict The causal-graph training and ablation results are worth a look, but the headline "explicit reasoning beats open SLMs" claim rests on a circular LLM-as-judge setup and needs human-validated labels before it can be believed. read the letter →

arxiv 2512.05933 v2 pith:OJDOILEO submitted 2025-12-05 eess.AS

Xuanru Zhou , Jiachen Lian , Henry Hong , Xinyi Yang , Gopala Anumanchipalli This is my paper · ORCID

classification eess.AS

keywords speechunderstandingworldmodelcausalgraphexplicitreasoningtheoryofmindpragmaticintentinstructiontuningsemi-supervisedlearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that speech understanding should not be a single black-box pass from audio to text, but an explicit chain of four cognitive states—situational context, speaker mental state, speech act, and pragmatic intent—organized as a causal graph. It claims that conditioning a language model on these graph-inferred states steers its reasoning into human-aligned spaces, improving accuracy and reducing hallucinations compared with standard speech language models and chain-of-thought prompting. The experiments report that an 8-billion-parameter model with this graph outperforms open-source speech LLMs on a model-as-judge reasoning metric and approaches proprietary systems, while training in roughly 20 GPU hours.

What carries the argument

The central object is a probabilistic causal graph with four categorical latent nodes—WMA (situational context), ToM (affective state), SA (communicative function), and Prag (underlying intent)—whose joint posterior factorizes as p(WMA|X)·p(ToM|X)·p(SA|WMA,ToM,X)·p(Prag|WMA,ToM,SA,X). Each node is a neural classifier receiving parents' states plus fused acoustic/text/prosody features; teacher forcing mixes ground-truth and predicted parent states; semi-supervised training lets gradients from labeled children update unlabeled parents, turning them into latent-variable generators. In the second stage, the graph's inferred states are fed as explicit context to an instruction-tuned LLM, which is

What would settle it

Have human raters, blind to model identity, score the generated reasoning chains and responses on the same test instances; if human scores do not show a gap between the graph-guided model and the chain-of-thought-tuned baseline that the model-as-judge reports, the claimed explicit-reasoning advantage is an artifact of the judge.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a predefined causal graph over four speech modules—World Model Activation (WMA), Theory of Mind (ToM), Speech Act (SA), and Pragmatic Intent (Prag)—can be trained with partial labels and then used to guide instruction tuning of a speech language model. The graph infers latent states via a DAG factorization, and the resulting state trace, serialized into the prompt, produces explicit reasoning chains and responses that the authors show surpass current open-source speech language models on a reasoning metric, with particularly strong emotion recognition. The graph also converges about five times faster than a fully connected random graph

Load-bearing premise

The whole pipeline and evaluation assume that the pseudo-labels generated by a teacher language model for the four speech modules are accurate enough to serve as ground truth, and that a judge model scoring against those labels reflects genuine reasoning quality; if those labels are biased, the graph learns from noise and the judge rewards internally consistent hallucination.

Editorial extensions

If this is right

If the graph states are reliable, speech language models can produce transparent, step-by-step reasoning that exposes which context, emotion, act, and intent drove the interpretation, making errors easier to diagnose.
Under partial supervision, the causal graph can infer missing modules from labeled ones (e.g., emotion from speech act), reducing annotation cost for new speech datasets.
The roughly five-fold training speedup of the causal graph over the random graph suggests that cognitive priors can substitute for data in learning state dependencies.
The strong emotion-recognition gain indicates that explicit state disentanglement may counteract text-dominance bias in audio-language models.
If the results hold, small models with structured priors could close part of the gap with much larger proprietary models at a fraction of the training compute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same four-state factorization may transfer to other modalities, such as vision-language or embodied agents, where situational context, mental state, act, and intent drive behavior; the causal graph could serve as a control variate to steer reasoning in those settings too.
The method's ceiling is likely set by the teacher model's label accuracy; a small human-verified subset of labels could calibrate how much label noise degrades the graph and the downstream reasoning.
Discrete categorical states are a coarse approximation of continuous cognitive states; allowing hierarchical or continuous states might better handle ambiguity such as mixed emotions or nested intents.
If the model-as-judge scores were replaced by human raters blind to model identity, the ranking between the graph-guided model and a chain-of-thought baseline could shift, isolating whether the graph's guidance or the data quality drives the improvement.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things worth knowing. First, the causal graph itself—the four-module DAG, the edge ablations, the random-graph comparison, the training-time difference—is a real, useful contribution. Second, the paper's headline claim that graph-guided reasoning beats open SLMs is not supported by the evaluation in Table 3. The stress-test note lands: the GPT-4o judge is handed the final.STATE labels, told to reward exact alignment with them, and only SWM is conditioned on those states. That scoring protocol measures echo fidelity, not reasoning quality. The pipeline's training targets are also Vicuna-generated from Vicuna-generated pseudo-labels, and the authors' own limitation section concedes inaccuracies propagate. So the M.J. margin (7.81 vs 2.92) is an internal-consistency score, not external evidence of human-aligned reasoning.

What the paper does well: graph training convergence speed (~5x vs random graph), robustness across teacher-forcing values, ablations showing causal edge removal hurts SA accuracy, and the semi-supervised setup with localized degradation patterns. Those results are not touched by the circularity concern and are worth building on.

Soft spots, in proportion. The semi-supervised latent-module accuracies (34–43%) undercut the claim that the graph reliably infers missing labels; the "world model" framing is overstatement—there is no temporal dynamics, just a static factorized classifier (and the action notion is causal influence, not time-step transition); the variance-reduction argument is informal and could be cut. None of that matters as much as the evaluation problem.

Bottom line: a serious referee should be assigned—the architecture idea and the graph-efficiency finding deserve attention—but the paper needs major revision: human-annotated labels on a held-out set, a judge that is blind to the conditioning states or at least identical for all models, and human judging of reasoning quality. As written, I would not accept the central claim.

Referee Report

4 major / 6 minor

Summary. The paper proposes Speech World Model (SWM), an instruction-tuned speech-language model conditioned on four categorical latent states — WMA, ToM, SA, and Prag — inferred by a predefined causal DAG. The graph is trained in fully- and semi-supervised settings, and the inferred states are incorporated as prompts for a Llama-3.1-8B or Qwen2-Audio-7B backbone via Eq. (7)-(8). The authors claim faster graph convergence, stable causal edge estimates, and superior speech understanding/reasoning over open-source SLMs, with Table 3 reporting an overall M.J. score of 7.81 for SWM versus 2.92 for the best open-source baseline. The final reasoning trace and response are generated from these graph states, with the stated hypothesis that conditioning on explicit graph states guides reasoning toward human-aligned spaces.

Significance. If the reported results were independently verified, the work would be a useful demonstration that structured speech states can improve instruction-tuned SLM reasoning while providing interpretable interventions. The architecture is clearly specified, the graph-training ablations (Tables 5-6) are informative, and the inclusion of a tuned Qwen2-Audio-CoT baseline plus explicit training-cost numbers are strengths. However, the central comparison is not an external evaluation: the GPT-4o judge receives exactly the four state labels that SWM is trained and conditioned on, and those labels themselves are generated by Vicuna-13b and used to build the reasoning targets. The core claim that graph states produce human-aligned reasoning is therefore not currently supported by the evidence presented.

major comments (4)

[Appendix A.9; Table 3] The evaluation protocol is circular. The GPT-4o judge prompt supplies the final.STATE labels and explicitly requires that the analysis 'align with final.STATE labels exactly.' SWM is conditioned on exactly these states in Eq. (7)-(8), while the baselines are not. The Table 3 margin (7.81 vs. 2.92) therefore largely measures whether the model echoes the answer key it was given, not its reasoning quality. No human evaluation or standard speech-reasoning benchmark is reported. This is load-bearing for the Sec. 5 claim of surpassing open-source SLMs and for the Sec. 3.3 hypothesis about human-aligned reasoning.
[Sec. 4.2; Appendix A.4.2] The ground-truth construction is unvalidated and self-referential. In Stage 1, Vicuna-13b imputes missing module labels; in Stage 2, Vicuna generates the reasoning and response targets conditioned on those same labels. The causal graph is trained on these pseudo-labels, and the instruction-tuning targets are Vicuna's rationalization of Vicuna's labels. The paper's own limitation section (Sec. 5) concedes that inaccuracies can propagate to reasoning and response. No agreement statistics against human-annotated subsets (e.g., MELD/IEMOCAP emotion labels) are provided, so the claim of 'human-aligned' reasoning is unsupported.
[Table 1; Sec. 4.5] The semi-supervised results undermine the latent-inference claim. When WMA, ToM, or SA is left unlabeled during training, its accuracy drops to 34.8%, 43.3%, and 34.4%, respectively, compared with 69.4%, 73.5%, and 65.3% in the fully supervised setting. The text interprets this as successful inference of latent states via causal structure, but these numbers are near chance for several label spaces and are not compared to majority-class baselines or to a no-causal-structure semi-supervised baseline. This weakens the 'reliably infer unlabeled modules' claim in Sec. 1 and Sec. 3.1.4.
[Appendix A.8.1; Eq. (24)-(25)] ACE and ICS are computed by intervening on the model's own internal state distributions, not by validating effects on held-out data or human judgments. These scores therefore show that the trained graph has certain internal dynamics; they do not establish that the predefined DAG matches causal structure in the speech signal. The ICS threshold tau is not specified, and the random-graph baseline is not compared on these metrics, so the claim of stable, interpretable causal dependencies is only partially supported.

minor comments (6)

[Section 4.5/4.6] Table 2 (random graph) appears before Table 1 (causal graph) in the text; renumber or reorder for clarity.
[Throughout] The text and figures repeatedly use 'casual graph' where 'causal graph' is intended; please correct.
[Appendix A.4.2] The worked example uses labels 'Office-work,' 'grateful,' 'Thanks/Appreciation,' and 'Task-completion' that are not present in the listed label spaces; this is confusing and should be made consistent.
[Appendix A.3.1, Eq. (19)] The redundancy term uses an undefined importance indicator Pi_i; define or remove.
[Reproducibility Statement] The statement says code will be made public at a github.io URL, but no actual repository or release is provided; please include a working link with artifact instructions.
[Table 3] Column header 'R-Len(Inference)' is described as 'average length of the generated response in words'; align the caption with the column meaning.

Circularity Check

2 steps flagged · score 8.0 of 10

Central evaluation is self-definitional: GPT-4o judge rewards exact alignment with final.STATE labels that only SWM receives as conditioning input, so Table 3's margin reflects answer-key leakage rather than independently measured reasoning.

self definitional [Appendix A.9 (GPT-4o judge rubric); Eqs. 7–8 in Sec. 3.3]
"You are given: the ASR transcript, the final.STATE with four labels (WMA, ToM, SA, Prag), plus the sample's analysis and response. ... Accuracy: does it align with final.STATE labels exactly? No inventing or changing labels."

The judge's primary 'Accuracy' criterion defines good reasoning as exact equality to the final.STATE labels. SWM is trained and prompted with exactly those labels (Eq. 7: p(y|Instr, I(G(x))); Eq. 8: p(y|Instr, x, SWMA, SToM, SSA, SPrag)), while the compared open-source baselines are not given them. Hence the M.J. score partially measures whether SWM echoes the answer key it was handed; Table 3's margin (SWM 7.81 vs Voxtral CoT 2.92) is, by construction, not evidence that the graph states produce more human-aligned reasoning.
fitted input called prediction [Sec. 4.2 Label-Generation, Stage 1/Stage 2; Eqs. 7–8; Sec. 5 limitation]
"To construct complete instances for fully-supervised graph training and instruction tuning, we implemented a robust two-stage label generation pipeline leveraging Vicuna-13b-v1.5 (Chiang et al., 2023) as the teacher model. ... Conditioned on the complete labels (from stage 1) and transcription, we query the LLM again to synthesize the downstream instruction data: (i) a comprehensive reasoning analysis ... and (ii) a contextually appropriate response aligned with these states."

The same LLM teacher (Vicuna-13b) imputes missing WMA/ToM/SA/Prag labels (Stage 1) and then generates the reasoning/response targets conditioned on those labels (Stage 2). These targets are the y in Eqs. 7–8 and the graph is trained on the Stage-1 labels, so the 'human-aligned' reasoning trace is Vicuna rationalizing Vicuna's labels. The paper's own limitation concedes 'inaccuracies can propagate to both the reasoning and response stages.' Calling the resulting graph states a human-aligned cognitive search space is therefore a fitted teacher output renamed as ground truth for the downstream evaluation.

full rationale

The graph-training comparisons (causal vs random graph convergence, ACE/ICS, edge ablations) are internally controlled and not intrinsically circular; those parts would support a much lower score. However, the paper's central claim—that conditioning on explicit graph states guides reasoning toward human-aligned spaces and that SWM surpasses open-source SLMs—rests on Table 3, whose Model-as-Judge protocol is structurally biased: the judge is given final.STATE labels, its rubric explicitly rewards exact alignment with them, and only SWM receives those states as inputs. The instruction-tuning targets are also generated by the same LLM that produced the labels. Thus the main 'prediction' (human-aligned reasoning advantage) reduces, in large part, to fidelity to an LLM teacher and to the answer key. Score 8 rather than 10 because graph-training efficiency and module-quality results have independent content and are not affected by the circular evaluation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The central claim rests on a set of hand-set structural choices (graph edges, module taxonomies) and a fully LLM-generated labeling and evaluation pipeline. The numeric 'free parameters' are ordinary hyperparameters, but the structural and evaluative assumptions are load-bearing and unvalidated. The semi-supervised latent-state inference is weak (accuracy around 34-43% for unlabeled modules), which further suggests the invented cognitive search space is not capturing true latent dynamics.

free parameters (4)

Teacher-forcing probability p = 0.3
Chosen for the causal graph training and ablated (Table 5); affects gradient flow and is not derived from data.
Fusion mechanism choice = Gated fusion
Selected among gated/attention/transformer based on validation; an architectural hyperparameter, not an invariant constant.
LoRA ranks and learning rates = r=64, lr=5e-5 (Llama); r=16, lr=2e-4 (Qwen)
Standard tuning hyperparameters that affect the instruction-tuning results but are not central to the causal graph claim.
M.J. score weights = 0.6 * Rs + 0.4 * Rp
The overall metric combines reasoning and response scores with hand-picked weights; changing them changes rankings.

assumptions (5)

ad hoc to paper The predefined DAG with edges WMA->SA, ToM->SA, WMA->Prag, ToM->Prag, SA->Prag is the correct causal structure for speech understanding.
Stated in Sec. 3.1 and in the system prompts (Appendix A.5.2); it is justified by cognitive science references but not learned or validated against data. The ablation removes edges, but no alternative graph is tested.
ad hoc to paper Four modules (WMA, ToM, SA, Prag) are necessary and sufficient to represent the communicative process.
Sec. 3.1.1 claims these modules form a 'complete cognitive perceptual subspace', but this is an assertion; the paper admits in Sec. 5 that more modules could help.
ad hoc to paper Vicuna-13b's generated labels for SA and missing modules are ground truth.
Sec. 4.2: label generation uses Vicuna-13b with one-shot prompting; no human validation or agreement metric is provided.
domain assumption GPT-4o's Model-as-Judge score is a valid measure of reasoning quality.
Appendix A.9: the judge is explicitly instructed to reward alignment with the predicted state labels, which are the same labels used in training and generation, making the metric circular.
domain assumption WavLM and OpenSmile features, combined with text, are sufficient to predict all four speech states.
Appendix A.2.1: the acoustic and prosodic features are fixed; the paper does not test whether other features or encoders would change the graph's conclusions.

invented entities (2)

Cognitive state search space over four modules
purpose: Serves as an explicit intermediate representation to guide LLM reasoning and enable counterfactual interventions.
The four modules and their label taxonomies (30, 7, 24, 14 classes) are introduced by the authors; no external benchmark or prediction outside this paper validates that this specific decomposition is correct.
Action A_{u->v} as causal influence between modules
purpose: Reinterprets state transitions as actions in a world model (Sec. 3.1.1, Fig. 3).
This is a conceptual relabeling of the graph edges, not a measurable entity. No independent evidence is provided that these 'actions' correspond to real-world communicative interventions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech." pith.science (2026). https://pith.science/paper/OJDOILEO

@misc{pith2026251205933,
  author       = {Pith},
  title        = {Pith review of: Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OJDOILEO}},
  note         = {Machine review of arXiv:2512.05933}
}

read the original abstract

Current speech-language models (SLMs) typically use a cascade of speech encoder and large language model, treating speech understanding as a single black box. They analyze the content of speech well but reason weakly about other aspects, especially under sparse supervision. Thus, we argue for explicit reasoning over speech states and actions with modular and transparent decisions. Inspired by cognitive science we adopt a modular perspective and a world model view in which the system learns forward dynamics over latent states. We factorize speech understanding into four modules that communicate through a causal graph, establishing a cognitive state search space. Guided by posterior traces from this space, an instruction-tuned language model produces a concise causal analysis and a user-facing response, enabling counterfactual interventions and interpretability under partial supervision. We present a graph-based modular speech model for explicit reasoning, highlighting a path toward more transparent and controllable speech understanding.

Figures

Figures reproduced from arXiv: 2512.05933 by the authors.

**Figure 1.** Speech World Model. Demo audio link: http://bit.ly/4pBJuWP. ABSTRACT Current speech-language models (SLMs) typically use a cascade of speech encoder and large language model, treating speech understanding as a single black box. They analyze the content of speech well but reason weakly about other aspects, especially under sparse supervision. Thus, we argue for explicit reasoning over speech states and actions with m… view at source ↗

**Figure 2.** The Speech World Model pipeline with a running example, illustrating the “Causal GraphGuided Explicit Reasoning” process. (1) Causal Graph Training: multimodal inputs (text x, acoustic a, prosody z) are encoded and fused to g = ϕ(hx, ha, hz). Each node state Sv is inferred from its parents Pa(v) and fused feature g via Sv = softmax(fv(g, {Su}u∈Pa(v))), yielding structured reasoning. (2) Instruction Tuning: these st… view at source ↗

**Figure 3.** A unified perspective on world models. Both the Generative (left (Garrido et al., 2024)) and [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Comparison of gradient flow under different training scenarios.(A) Semi-supervised causal [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: ACE and ICS of each casual edge under both fully-supervised and semi-supervised training. Instruction tuning. To provide a scalable and consistent evaluation of our instruction-tuned models, we employ the Model-as-Judge (M.J.) methodology (Zheng et al., 2023), where GP…

**Figure 6.** Figure 6: ACE and ICS of each casual edge for ablation study (fusion mechanisms and teacher-forcing probabilities) on casual graph under fully-supervised setting [PITH_FULL_IMAGE:figures/full_fig_p023_6.png]

**Figure 7.** Figure 7: ACE and ICS of each casual edge for ablation study (removal of specific causal edge) on casual graph under fully-supervised setting. System Prompt (Llama3.1-8b): You are an expert in speech analysis. The four speech modules are: WMA: World Model Activation-what concret…

**Figure 8.** Figure 8: ACE and ICS of each casual edge for ablation study on casual graph under semi-supervised setting. overall performance remains remarkably stable, with node quality and ACE scores comparable to the baseline. This demonstrates the robustness of our semi-supervised framewo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 1 linked inside Pith

[1]

•INPUT: with ASR TRANSCRIPT, KNOWN LABELS and MISSING MODULES (list of modules to predict)

You will receive: •LABEL SPACE: a JSON listing the allowed labels for{WMA, ToM, SA, Prag}. •INPUT: with ASR TRANSCRIPT, KNOWN LABELS and MISSING MODULES (list of modules to predict)
[2]

For each module in **MISSING MODULES**, select EXACTLY ONE label from LABEL SPACE[module]: - Decide your choice based on given ASR TRANSCRIPT and KNOWN LABELS - Select EXACTLY ONE label from LABEL SPACE[module] for MISSING MODULES - Label strings must match EXACTLY (case-sensitive, character-perfect) - When choosing between multiple plausible options, sel...
[3]

ASR_TRANSCRIPT

OUTPUT FORMAT - for each module in MISSING MODULES: <ModuleName>CHOICE</ModuleName> EXAMPLE: Input: { "ASR_TRANSCRIPT": "Oh my God, he's lost it. He's totally lost it.", "KNOWN_LABELS": { "ToM": "sadness", "SA": "Statement-non-opinion" }, "MISSING_MODULES": ["WMA", "Prag"] } Output: <WMA>Calendar</WMA> <Prag>Request-Info</Prag> 20 Preprint The prompt for ...
[7]

You will receive: •INPUT: with ASR TRANSCRIPT and COMPLETE STATE containing all four module values (WMA, ToM, SA, Prag)
[8]

Generate two components: •<Analysis> | A detailed causal reasoning chain (4-8 sentences) explaining: - How the world context (WMA) shapes the situation - How the speaker’s mental state (ToM) influences their communication - How these factors lead to the specific speech act (SA) - How the pragmatic intent (Prag) emerges from this combination - What this su...
[9]

ASR_TRANSCRIPT

OUTPUT FORMAT: <Analysis>YOUR DETAILED CAUSAL REASONING CHAIN</Analysis> <Response>YOUR NATURAL USER-FACING REPLY</Response> EXAMPLE: Input: { "ASR_TRANSCRIPT": "Thanks so much for helping me find that document!", "COMPLETE_STATE": { "WMA": "Office-work", "ToM": "grateful", "SA": "Thanks/Appreciation", "Prag": "Task-completion" } } Output: <Analysis>Withi...

2022
[10]

This represents the behavior of MT without direct information fromM S

Baseline (Without Source):We compute the target module’s output logits, denoted as Lwithout T , by replacing the input from MS with a zero vector. This represents the behavior of MT without direct information fromM S
[11]

analysis

With Source Influence:We compute the target module’s output logits, Lwith T , by providing it with the actual probability output from MS. This represents the behavior of MT when influenced by MS. The influence is then calculated by comparing the probability distributions derived from these two sets of logits,P without =softmax(L without T )andP with =soft...

2025

Show all 11 references

[2017]

the- ory of mind

URLhttps://api.semanticscholar.org/CorpusID:3338320. Bruno Bara. Cognitive pragmatics the mental processes of communication.Intercultural Pragmatics, 8, 09 2011. doi: 10.1515/iprg.2011.020. Simon Baron-Cohen, Alan M. Leslie, and Uta Frith. Does the autistic child have a “the- ...

2011
[2024]

URL https://openreview.net/forum?id=mqoxLkX210

ISSN 2835-8856. URL https://openreview.net/forum?id=mqoxLkX210. Featured Certification. Zhifeng Kong, Arushi Goel, Rohan Badlani, Wei Ping, Rafael Valle, and Bryan Catanzaro. Audio flamingo: A novel audio language model with few-shot learning and dialogue abilities.arXiv prepr...

2024 arXiv
[2025]

state” is the context provided by the preceding text, and the “action

URLhttps://openreview.net/forum?id=246rHKUnnf. John R. Searle.Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press, 1969. Sanjit A Seshia, Dorsa Sadigh, and S Shankar Sastry. Toward verified artificial intelligence.Commu- nications of the ACM, 65(7):...

1969

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

•INPUT: with ASR TRANSCRIPT, KNOWN LABELS and MISSING MODULES (list of modules to predict)

You will receive: •LABEL SPACE: a JSON listing the allowed labels for{WMA, ToM, SA, Prag}. •INPUT: with ASR TRANSCRIPT, KNOWN LABELS and MISSING MODULES (list of modules to predict)

[2] [2]

For each module in **MISSING MODULES**, select EXACTLY ONE label from LABEL SPACE[module]: - Decide your choice based on given ASR TRANSCRIPT and KNOWN LABELS - Select EXACTLY ONE label from LABEL SPACE[module] for MISSING MODULES - Label strings must match EXACTLY (case-sensitive, character-perfect) - When choosing between multiple plausible options, sel...

[3] [3]

ASR_TRANSCRIPT

OUTPUT FORMAT - for each module in MISSING MODULES: <ModuleName>CHOICE</ModuleName> EXAMPLE: Input: { "ASR_TRANSCRIPT": "Oh my God, he's lost it. He's totally lost it.", "KNOWN_LABELS": { "ToM": "sadness", "SA": "Statement-non-opinion" }, "MISSING_MODULES": ["WMA", "Prag"] } Output: <WMA>Calendar</WMA> <Prag>Request-Info</Prag> 20 Preprint The prompt for ...

[4] [7]

You will receive: •INPUT: with ASR TRANSCRIPT and COMPLETE STATE containing all four module values (WMA, ToM, SA, Prag)

[5] [8]

Generate two components: •<Analysis> | A detailed causal reasoning chain (4-8 sentences) explaining: - How the world context (WMA) shapes the situation - How the speaker’s mental state (ToM) influences their communication - How these factors lead to the specific speech act (SA) - How the pragmatic intent (Prag) emerges from this combination - What this su...

[6] [9]

ASR_TRANSCRIPT

OUTPUT FORMAT: <Analysis>YOUR DETAILED CAUSAL REASONING CHAIN</Analysis> <Response>YOUR NATURAL USER-FACING REPLY</Response> EXAMPLE: Input: { "ASR_TRANSCRIPT": "Thanks so much for helping me find that document!", "COMPLETE_STATE": { "WMA": "Office-work", "ToM": "grateful", "SA": "Thanks/Appreciation", "Prag": "Task-completion" } } Output: <Analysis>Withi...

2022

[7] [10]

This represents the behavior of MT without direct information fromM S

Baseline (Without Source):We compute the target module’s output logits, denoted as Lwithout T , by replacing the input from MS with a zero vector. This represents the behavior of MT without direct information fromM S

[8] [11]

analysis

With Source Influence:We compute the target module’s output logits, Lwith T , by providing it with the actual probability output from MS. This represents the behavior of MT when influenced by MS. The influence is then calculated by comparing the probability distributions derived from these two sets of logits,P without =softmax(L without T )andP with =soft...

2025