Pith. sign in

REVIEW 3 major objections 41 references

CausalDS shows that LLM data-science agents master graph reading and many estimates but part ways on uncertainty, abstention, and tool efficiency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 13:11 UTC pith:EYPEJR3B

load-bearing objection Solid, carefully engineered agent benchmark that actually measures multi-axis dissociation (structure/content vs UQ/abstention/tool use); exam size and graph-level ID policy are real limits but not load-bearing against the central claim. the 3 major comments →

arxiv 2607.08093 v1 pith:EYPEJR3B submitted 2026-07-09 cs.AI cs.CLcs.LG

CausalDS: Benchmarking Causal Reasoning in Data-Science Agents

Andrej Leban , Yuekai Sun This is my paper
classification cs.AI cs.CLcs.LG
keywords causal reasoningdata-science agentsstructural causal modelsPearl's ladderabstentionuncertainty quantificationLLM benchmarksobservation layer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces CausalDS, a generator of synthetic causal data-science scenes: each scene hides a structural causal model, releases observational tables (sometimes through a noisy measurement layer), and narrates a free-form domain story that is audited against the graph. From each scene it derives tasks across Pearl's three rungs, including prediction, identification, effect estimation, bias diagnostics, and counterfactuals, and treats deliberate non-identifiability as a scored abstention target. On a realistic-composition exam of 100 scenes, six contemporary agents largely recover structure and produce many correct content answers, yet they dissociate on uncertainty quantification, knowing when no answer is warranted, and how efficiently they use tools. Claude Opus 4.8 leads the aggregate score; the open-weight models trail mainly on the epistemic axes. The point is that a competent causal data-science agent must clear all five axes at once, and existing benchmarks that isolate symbolic reasoning or open-ended coding cannot measure that joint skill.

Core claim

On a realistically composed CausalDS exam, the five evaluation axes do not collapse into one capability. Models recover story-implied structure and often get discrete content right, but they diverge sharply on calibrated uncertainty, epistemic abstention on non-identifiable estimands, and tool-use efficiency, with Claude Opus 4.8 closest to a well-rounded causal data-science agent.

What carries the argument

The CausalDS scene: a sampled DAG and SCM, optional noisy observation bundles that leave conceptual identifiability unchanged, a graph-audited free-form story, and a task suite spanning Rung 1–3 with deterministic scoring that routes non-identifiable targets into an abstention pool.

Load-bearing premise

The load-bearing premise is that the hidden graph's nonparametric identifiability labels, together with the audited story, unambiguously define what a competent agent should recover from the released prose and files, even when an agent invents parametric or proximal identification arguments the graph does not support.

What would settle it

If a controlled verbalization-swap or observation-matched re-exam showed that the same formal problem produced large, systematic rank reversals for the strongest models solely because of story wording or measurement view, or if frontier agents stopped failing the non-identifiable abstention cases while still matching the reported content accuracy, the dissociation claim would weaken.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. CausalDS is a synthetic, SCM-grounded benchmark for agentic causal data science. Each scene samples a DAG/SCM (optionally motif-grafted), generates observational data, optionally replaces conceptual variables by noisy measurement bundles with a calibration split, maps nodes to domain variables (CauseNet-seeded, LLM-audited), and produces a free-form story. From each scene the authors derive R1–R3 tasks (prediction, association, graph recovery, identification, effect estimation, bias diagnostics, counterfactuals/mediation), with private ground truth from Monte Carlo and DoWhy/y0 ID*/IDC* and first-class abstention scoring on non-identifiable queries. Evaluation uses a sandboxed mini-swe-agent harness with deterministic graders and composite metrics (Pass Rate, SNR, CausalDSScore). On a 100-scene realistic-composition exam, six models largely master structure recovery and many content estimates, but dissociate on uncertainty quantification, abstention, and tool-use efficiency, with Claude Opus 4.8 leading.

Significance. The paper fills a real gap between symbolic causal QA and open-ended data-science agent benchmarks by integrating hidden SCMs, graph-audited free-form stories, a measurement observation layer, full Pearl-ladder tasks, file-backed tool use, and scored abstention in one generator. Strengths that raise the contribution above a typical leaderboard paper include: deterministic scoring with mutually exclusive abstention routing; Fisher-information admissibility for observation bundles; verbalization-swap and matched observation-layer ablations; pass@k stability analysis; and trajectory-level case studies (e.g., Fig. 2 / App. A.14). Code and full datasets are released. If the multi-axis dissociation holds under broader exams, CausalDS becomes a useful diagnostic for whether agents can act as causal data scientists rather than causal parrots or pure coders.

major comments (3)
  1. §5 / Table 3 and App. A.10: the central multi-axis dissociation claim is supported, but several headline slices are thin (e.g., ATE-UQ95 coverage n=7 per model, 5 for Qwen; mediation n=2; several motif and abstention-family cells n=2–5). The qualitative pattern (structure/content largely saturated; UQ coverage 20–71%; abstention 18.8–75%; tool styles separable) is credible and ablations help, yet ranking precision and some per-family claims are overstated relative to sample size. Either enlarge the realistic exam / report bootstrap CIs on CausalDSScore and coverage, or clearly demote small-n slices to exploratory.
  2. §3.4, 3.7 and App. A.12–A.14: ground truth for identification and abstention is graph-level nonparametric population-ATE / R3 ID*/IDC* under the story-implied graph. The paper documents this policy and shows that parametric/proximal “rescues” (Fig. 2 trajectories) are scored as failed abstentions by design. That choice is defensible for a controlled benchmark, but it is load-bearing for the abstention axis and can be harsh relative to applied practice. The manuscript should state more prominently in the main text (not only appendices) that scores measure agreement with this fixed policy, not unique correctness of every applied identification argument, and discuss sensitivity of the frontier/open abstention gap to that policy.
  3. §5 and App. A.15: open-weight models are run at serving defaults (no standardized reasoning-effort control), while frontier models use high reasoning effort; Qwen’s low valid-continuous rate is partly tool-output mismanagement that persists under re-runs. The dissociation claim does not require identical compute, but the cost–quality separation and open-model rankings are partly confounded by inference settings and harness interaction. Report matched-effort or matched-token comparisons where feasible, or frame open vs. closed results as under released defaults rather than as pure capability.

Circularity Check

0 steps flagged

No significant circularity: CausalDS is an evaluation benchmark whose ground truth and scores are fixed by private SCMs and ID algorithms, not by re-labeling fitted model outputs as predictions.

full rationale

This paper introduces a synthetic causal data-science benchmark and reports multi-axis agent performance; it does not claim a first-principles derivation of a physical or statistical law. Scene ground truth is generated from sampled SCMs, Monte Carlo effects, and DoWhy/y0 ID*/IDC* identifiability labels before any agent is run (Secs. 3.1–3.4, 3.7). Public stories and data are scored against that private truth; abstention is a first-class target when the estimand is non-identifiable by construction. The CausalDSScore (Eq. 4) is an explicit weighted aggregate of Pass Rate, SNR, and Med. F1-Loss for leaderboard use—not a quantity presented as independently predicted from nature. Optional empirical composition priors (App. A.7–A.8) and the difficulty knob are disclosed as design choices (including hand tilts and editorial dials), not as fitted parameters later rebranded as out-of-sample predictions. Verbalization-swap and matched observation ablations (App. A.12–A.13) hold formal structure fixed and vary only story or measurement view, which is anti-circular methodology rather than self-definition. There is no load-bearing self-citation uniqueness theorem, no ansatz smuggled in as external fact, and no renaming of a known empirical pattern as a new derivation. Mild residual concerns about exam composition or nonparametric scoring policy are external-validity / policy choices, not circular reductions of claims to their inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The central empirical claim rests on standard SCM/do-calculus machinery plus several design choices that define what counts as a correct agent answer. Free parameters are mostly composition and hardness knobs, not fitted scientific constants. Invented entities are benchmark constructs (scenes, observation bundles, composite scores), not physical postulates.

free parameters (5)
  • exam difficulty knob d and tilt strengths β,δ,γ
    Editorial dial that reweights rung, output variant, and observation hardness; realistic exam uses d=0.5 but other mixes are free design choices (App. A.8).
  • non-identifiable scene fraction / rejection-sampling target (~10% deliberate non-ID; ~30% exam non-ID)
    Hand-chosen stress level for abstention, not estimated from a well-defined population frequency of non-identifiable real studies (App. A.7).
  • observation Fisher-information thresholds (e.g. τ_min=0.03) and proxy_hard corruption settings
    Admissibility and hardness of measurement bundles are configured thresholds that shape estimation difficulty (App. A.6).
  • CausalDSScore pool weights and interval cap c=10
    Composite leaderboard depends on task-count weights and the capped-mean interval aggregator (Eqs. 3–4; App. A.9).
  • SCM profile mixture α=0.3 empirical vs synthetic coverage prior
    Mechanism-family mix is a convex combination chosen after demoting biology-heavy empirical corpora (App. A.7).
axioms (5)
  • domain assumption Pearl SCM / DAG semantics and the three-rung hierarchy organize tasks and estimands
    Background causal formalism assumed throughout Sec. 2–3; standard in the field.
  • domain assumption Do-calculus / ID and ID*/IDC* decide population-ATE and R3 identifiability labels used as ground truth
    Scoring treats algorithmic graph identifiability as the correct agent target (Sec. 3.4, 3.7).
  • ad hoc to paper Observation bundles are pure measurement of one conceptual parent and never change conceptual confounding/identification
    Core design claim of the observation layer (Sec. 3.2); agents are graded under this policy even if they invent proximal arguments.
  • ad hoc to paper LLM mapper/auditor/verbalizer loops produce stories faithful enough that the story-implied graph is the intended identification graph
    Alg. 1–4 and audits are engineering assumptions; residual story-lure failures are documented in App. A.12.
  • domain assumption Empirical composition crosswalks from heterogeneous real benchmarks approximate real causal-analysis workloads
    App. A.7 maps design-family labels and accessibility fields into motifs/question types; partly hand-authored.
invented entities (3)
  • CausalDS scene (hidden SCM + public story/data/schema + private GT) no independent evidence
    purpose: Unit of evaluation that couples language, files, and causal ground truth
    Benchmark construct; independent evidence is the released generator and datasets, not an external natural object.
  • Observation layer / measurement bundles with calibration split no independent evidence
    purpose: Vary data-science difficulty without flipping conceptual identifiability
    Design object distinct from proximal causal inference; validated mainly by internal diagnostics and matched ablations.
  • CausalDSScore / SNR / Pass Rate composites with abstention routing no independent evidence
    purpose: Single-number and multi-axis leaderboard summaries
    Scoring inventions; useful but editorial, especially answer-dependent abstention pools.

pith-pipeline@v1.1.0-grok45 · 58100 in / 3600 out tokens · 44302 ms · 2026-07-10T13:11:38.159454+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use. Yet the relevant benchmark landscape largely divides into symbolic causal reasoning benchmarks without realistic data analysis or data analysis benchmarks without a principled causal data-generating structure. Furthermore, existing causal evaluation datasets are often restricted to curated examples from existing sources, with diversity coming from limited templatized variations rather than from systematic generation of novel synthetic causal structures. We introduce CausalDS, a benchmark for evaluating causal reasoning in agentic data-science workflows. Each benchmark instance is a scene consisting of a sampled structural causal model (SCM) with generated observational data and an accompanying synthetic natural-language story grounded in a realistic domain. We optionally ground the composition of the benchmark components in empirical distributions obtained from real-world datasets, thus retaining empirical structure while reducing the "causal parrot" risk through completely synthetic generation. From each scene, we then derive tasks spanning all three of Pearl's rungs, with typical data-science prediction tasks appearing as Rung 1. Most tasks include a data science coding component, where the model typically needs to use several tools to arrive at the final answer due to the frequent presence of imperfect observations, which are generated by an observation model. Additionally, recognizing when a question admits no warranted answer and abstaining is treated as a first-class scored outcome. The benchmark thus jointly evaluates symbolic causal reasoning, data science, uncertainty quantification, abstention, and tool use/coding.

Figures

Figures reproduced from arXiv: 2607.08093 by Andrej Leban, Yuekai Sun.

Figure 1
Figure 1. Figure 1: CausalDS pipeline: from hidden causal structure to benchmark execution. Scene example (R2 bias diagnostic, forbidden-controls output variant; iv motif; hard observation variant). Story. Farmers in a semi-arid region receive an Irrigation Activation Subsidy (dollars per hectare) that economically incentivizes whether an Irrigation Event is Executed on their cropland. This decision also depends on the Subsur… view at source ↗
Figure 2
Figure 2. Figure 2: A CausalDS scene: story, public dataset with observed measurements, hidden DAG, and the posed question. is active, we restrict both the main graph and the auxiliary graph motif pools toward motifs that remain stable under grafting. Anchor grafting thus allows increasingly complex structures while preserving narrative coherence by the use of the shared (already verbalized) anchor, and limiting the variable … view at source ↗
Figure 3
Figure 3. Figure 3: (a) Empirical coverage of the nominal 95% ATE intervals per model over the identifiable ate_uq_95 tasks (n=7 per model; 5 for Qwen 3.6 35B). (b) Content pass rate, abstention pass rate, and SNR (Eq. 3; lower is better) per model. Models are ordered by CausalDSScore (best at top); blue circles — frontier (closed) models, green triangles — open-weight models. Tool-use strategy splits the field ( [PITH_FULL_… view at source ↗
Figure 4
Figure 4. Figure 4: Tokens per task (log scale) versus CausalDSScore (lower is better) on the realistic exam. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Base motif templates used by the sampler. Latent variables are shown as gray dashed nodes. [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Per-scene recoverability diagnostic for the [PITH_FULL_IMAGE:figures/full_fig_p023_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Exam-composition factorization. Conditional on the structural label [PITH_FULL_IMAGE:figures/full_fig_p023_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Pass Rate per rung and model on the realistic exam; pool sizes [PITH_FULL_IMAGE:figures/full_fig_p030_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Matched observation-layer ablation: capped diagnostic loss for all [PITH_FULL_IMAGE:figures/full_fig_p039_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Exact decomposition of the mean paired capped-loss delta over the ten matched pairs, by [PITH_FULL_IMAGE:figures/full_fig_p040_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 41 canonical work pages · 5 internal anchors

  1. [1]

    MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

    doi: 10.48550/arXiv.2410.07095. URL https://openreview.net/forum?id= 6s5uXNWGIh. arXiv:2410.07095 [cs]. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishk...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2410.07095 2021

  2. [2]

    CausalGame: Benchmarking Causal Thinking of LLM Agents in Games

    Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-emnlp.363. URLhttps://aclanthology.org/2024.findings-emnlp.363. Zhenhao Chen, Yongqiang Chen, Chenxi Liu, Junchi Yu, Xiangchen Song, Zijian Li, Jialin Li, Philip Torr, Bo Han, and Kun Zhang. CausalGame: Benchmarking Causal Thinking of LLM Agents in Games. InProceedings of the 43rd I...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2024.findings-emnlp.363 2024

  3. [3]

    Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inference

    doi: 10.48550/arXiv.2505.13770. URL https://openreview.net/forum?id= MGMG7yQ18v. arXiv:2505.13770 [cs]. 15 Preprint Elliot L. Epstein, John Winnicki, Thanawat Sornwanee, and Rajat Dwaraknath. LLMs are Over- confident: Evaluating Confidence Interval Calibration with FermiEval, October 2025. URL http://arxiv.org/abs/2510.26995. arXiv:2510.26995 [stat.ME] ve...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2505.13770 2025

  4. [4]

    RoadmapBench: Evaluating Long-Horizon Agentic Software Development Across Version Upgrades

    doi: 10.1093/nar/gkx1065. URLhttps://doi.org/10.1093/nar/gkx1065. Xinbo Xu, Ruihan Yang, Haiyang Shen, Wendong Xu, Bofei Gao, Ruoyu Wu, Kean Shi, Weichu Xie, Xuanzhong Chen, Ming Wu, Jason Zeng, Michael Heinrich, Elvis Zhang, Liang Chen, Kuan Li, and Baobao Chang. RoadmapBench: Evaluating Long-Horizon Agentic Software Development Across Version Upgrades, ...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.1093/nar/gkx1065 2026

  5. [5]

    CausalBench: A Comprehensive Benchmark for Causal Learning Capability of LLMs

    doi: 10.48550/arXiv.2404.06349. URL http://arxiv.org/abs/2404.06349. arXiv:2404.06349 [cs]. 19 Preprint A Appendix A.1 Code and data availability The CausalDS source code — the scene-generation pipeline, the evaluation harness, and the grader — is available at github.com/andleb/causalds. The same repository also provides the entire datasets used for the m...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2404.06349 2026

  6. [6]

    Analyze the fixed nodes (if any) to infer the domain context

    work page

  7. [7]

    Factor A

    Rename the target nodes to specific, measurable, and realistic variables that fit this domain. Avoid generic names like "Factor A" or "Variable X"

    work page

  8. [8]

    mmHg", "years

    Ensure the ’unit’ field contains realistic measurement units (e.g., "mmHg", "years", "kg", "counts") and ’type’ is appropriate (e.g., "continuous", "binary"). This is not as crucial for the UNOBSERVED variables

    work page

  9. [9]

    The chosen variables must make sense together in a single scenario

    work page

  10. [10]

    observed

    CRITICAL for "observed" field: Check the "observed_nodes" list in the graph. Set observed=true ONLY for nodes in that list. Any node NOT in observed_nodes is LATENT/unobserved - you MUST set observed=false for these. {output_format_block} Variable mapping user prompt template.Optional placeholders are filled by the grafting and anchor logic. The auxiliary...

    work page

  11. [11]

    A causal graph structure with FIXED NODE CONCEPTS (already named from a knowledge base)

    work page

  12. [12]

    Can a skilled mapper typically find a coherent interpretation that makes the constraints work?

    The structural constraints imposed by the graph: - DIRECT EDGES: Causal relationships that MUST be plausible - NON-EDGES: Pairs where NO direct causal link must exist YOUR TASK: Decide whether there is at least one MAINSTREAM, non-contrived interpretation under which the FIXED NODE concepts satisfy the structural constraints. Ask: "Can a skilled mapper ty...

    work page

  13. [13]

    DIRECT edges (u -> v) that must be plausible as direct causal effects

    work page

  14. [14]

    NON-EDGE pairs (u, v) where NO plausible direct causal link may exist

    work page

  15. [15]

    Optionally, conditional independence (CI) relations from the graph

    work page

  16. [16]

    A proposed variable mapping from node ids to real-world meanings

    work page

  17. [17]

    Your job:

    Optionally, expected node type hints from generation. Your job:

    work page

  18. [18]

    non_edge_attestations

    For each NON-EDGE pair (U, V), decide if ANY DIRECT CAUSAL LINK exists between the mapped variables. Working definition: ’There exists an intervention on U that changes V while holding fixed the other variables in the graph (especially the other nodes that could mediate the effect).’ You MUST include an entry in "non_edge_attestations" for EVERY non-edge ...

    work page

  19. [19]

    Be GENEROUS: any reasonable mechanism (weak, partial, or context-dependent) is sufficient

    For each DIRECT edge (U -> V), decide if a plausible direct causal link exists. Be GENEROUS: any reasonable mechanism (weak, partial, or context-dependent) is sufficient. Only flag a VIOLATION if clearly implausible

    work page

  20. [20]

    For each CI statement, judge if plausible given variable meanings. (Soft constraint.) When flagging a CI violation, name the variables, explain why the chosen meanings make the independence implausible (e.g., uncovered alternative pathways), and HINT at a reframing that would restore it

    work page

  21. [21]

    explanation

    Type consistency: be lenient (treat close families as compatible, skip when missing); only flag clear contradictions. (Soft.) INTERPRETATION NOTES: We mostly care about the NON-EDGE violations. Use the definition of the DIRECT CAUSAL LINK above very strictly. For example, a verbal aggregation of a detailed effect X -> M -> Y is NOT a separate direct effec...

    work page

  22. [22]

    node_attestations

    Every required variable must be explicitly mentioned in the STORY 47 Preprint using its provided story name. Latent/unobserved variables may be narrated as hidden/background factors, but they still must be mentioned. Emit one entry in "node_attestations" for EVERY required variable

    work page

  23. [23]

    edge_attestations

    Every DIRECT edge (U -> V) must be clearly supported by the STORY itself. A path-level claim does NOT automatically cover every edge on the path. Emit one entry in "edge_attestations" for EVERY direct edge (supported / contradicted, with a justification)

    work page

  24. [24]

    SOFT CHECKS:

    If the STORY contradicts the stated direction of an edge, mark that as a hard violation. SOFT CHECKS:

    work page

  25. [25]

    Warn if the STORY introduces extra direct causal claims between NON-EDGE pairs

    work page

  26. [26]

    Warn if the STORY drifts from the proposed domain

    work page

  27. [27]

    Warn if the STORY uses graph jargon

    work page

  28. [28]

    pass" to true only if all hard requirements are satisfied. - Put ONLY hard failures in

    Warn if the STORY is implausible, globally incoherent, or mixes incompatible domains in a way that makes the scenario hard to believe as a real setting. DECISION RULE: - Set "pass" to true only if all hard requirements are satisfied. - Put ONLY hard failures in "violations"; put soft issues in "warnings". - Each issue must be actionable, ending in a "HINT...

    work page

  29. [29]

    No exceptions

    EVERY response must include EXACTLY ONE bash tool call. No exceptions

    work page

  30. [30]

    Include your reasoning as text BEFORE making the tool call

    work page

  31. [31]

    scenes/<scene_id>/data.parquet

    For multi-line Python, use a heredoc: python3 << ’PYEOF’ import pandas as pd df = pd.read_parquet("scenes/<scene_id>/data.parquet") print(df.describe()) PYEOF

    work page

  32. [32]

    key": "value

    To write files, use cat with heredoc: cat > /workspace/answers/my_answer.json << ’EOF’ {"key": "value"} EOF 48 Preprint

    work page

  33. [33]

    Do not write answers under ,→/home

    The grader reads only /workspace/answers. Do not write answers under ,→/home

    work page

  34. [34]

    Treatment

    When done, verify your answer file exists at the requested path, then ,→submit: echo DONE </rules> Shared user-message skeleton.The data-location bullets list only the files released for the task: the data.parquet/calibration.parquet/schema.json lines appear only for data-backed tasks, and a test_features.parquet line is added for prediction. The story-on...

    work page

  35. [35]

    What is the sign of the marginal (unconditional) association?

    work page

  36. [36]

    sign_before

    What is the sign of the association after conditioning on the ,→conceptual variable **Covariate1**? Conditioning here means statistical conditioning, not intervention. ### Output Format Provide your answer as a JSON object: ‘‘‘json { "sign_before": "+", // marginal association sign: "+", "-", or ,→"unknown" "sign_after": "+", // association after conditio...

    work page

  37. [37]

    trivial_zero

    "trivial_zero": Treatment has no directed causal path to Outcome, so ,→the population ATE is identifiable as zero. 52 Preprint

    work page

  38. [38]

    backdoor

    "backdoor": Otherwise, a valid backdoor adjustment set among the ,→observed conceptual variables identifies the population ATE

    work page

  39. [39]

    frontdoor

    "frontdoor": Otherwise, a valid front-door formula using observed ,→conceptual variables identifies the population ATE

    work page

  40. [40]

    other_id

    "other_id": Otherwise, the population ATE is identifiable by another ,→valid do-calculus / ID argument

    work page

  41. [41]

    none": The population ATE is not identifiable. ### Output Format Provide your answer as a JSON object: ‘‘‘json {

    "none": The population ATE is not identifiable. ### Output Format Provide your answer as a JSON object: ‘‘‘json {"method": "backdoor"} ‘‘‘ R2: Effect estimate — population ATE (point).Data-backed (Measurement Note shown). Variants share this stem and swap the output contract: ate_sign_only ({"sign": "+"/"-"/"0"/"unknown"}), ate_vs_assoc_sign_match ({"matc...

    work page