Pith. sign in

REVIEW 5 major objections 5 minor 88 references

Scientific Feasibility Control claims a finite-sample guarantee: the reasoning chain a language model selects will be scientifically valid — every step correct against physical law and deducible from the verified steps before it — with prob

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 20:10 UTC pith:RW5WVFDJ

load-bearing objection The conformal guarantee does not apply to the implemented algorithm, and the validity numbers are circular; not ready for review. the 5 major comments →

arxiv 2607.16704 v1 pith:RW5WVFDJ submitted 2026-07-18 cs.CL

Though Language Models Err While They Strive: Conformal Prediction for Self-Correcting Scientific Generation

Mingqiao Mo , Yunlong Tan , Hao Zhang This is my paper
classification cs.CL
keywords conformal predictionscientific reasoninglarge language modelsfactualitydeducibility graphself-correctionuncertainty quantificationphysics reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Scientific Feasibility Control (SFC) claims that a language model's reasoning can be certified step by step: decompose the output into atomic scientific claims, check each claim against physical law and against the verified context preceding it, and apply graph-structured conformal prediction so that, with finite-sample probability at least 1−α, the selected reasoning chain is absolute-coherent-factual — individually correct and logically deducible from prior steps. The paper's innovation is treating scientific reasoning as a dependency graph rather than a bag of independent claims, which lets the system branch to a new generation path the moment a violation appears, using verified context as a fixed foundation. On a multimodal physics benchmark, SFC with graph-aware risk weighting reaches 50.1% accuracy and 91.7% scientific validity, cutting physics-law violations by 73% relative to unmodified generation. A sympathetic reader would care because it offers a model-agnostic, distribution-free way to attach statistical guarantees to structured scientific text, not just a performance trick.

Core claim

On the paper's own terms, the central claim is Theorem 1: for any significance level α, after calibrating on exchangeable examples, the subgraph selected by SFC is absolute-coherent-factual with probability at least 1−α, regardless of how the dependency graph was constructed dynamically. Absolute-coherent-factuality requires each atomic fact-option to both satisfy the scientific ground truth and be deducible from the input, the ground truth, and the verified fact-options that precede it. The paper argues that this definition captures the difference between scientific reasoning and recall-style factuality, and that progressive validation with dynamic branching prevents the cascade effect wher

What carries the argument

The load-bearing objects are: (1) absolute-coherent-factuality, the property that each atomic claim is true against physical law and deducible from the verified prefix; (2) approximate deducibility graphs, DAGs in which edges mark sufficient substantiation and which need only satisfy ancestor connectivity and consistency; and (3) dynamic graph-structured conformal prediction, which computes nonconformity scores from a composite validator — hard-constraint checks, soft semantic assessment, and cross-modal consistency — calibrates a quantile, and selects ancestor-connected subgraphs. The graph turns 'the next step follows from what was already verified' into a checkable structural condition, a

Load-bearing premise

The coverage guarantee rests on exchangeability of the risk scores and on the acceptance threshold being the calibrated conformal quantile; the deployed system fixes the threshold at 0.1, learns the validation weights from data, and replaces cyclic deducibility graphs with linear ones, so if any of those departures break exchangeability or graph fidelity, the ≥1−α guarantee on real scientific validity no longer follows.

What would settle it

Take a held-out set of, say, 500 physics problems with independent expert labels for absolute-coherent-factuality, run the complete SFC pipeline exactly as deployed (fixed τ = 0.1, learned β and γ, linear-graph fallback), and count the fraction of selected reasoning chains that are in fact absolute-coherent-factual. If the realized fraction is below 1−α = 0.90 by more than finite-sample calibration slack, the theorem as stated for the implemented system is false. A sharper probe targets the graph: feed the pipeline a subtle conservation-law violation that the hard-constraint patterns do not fl

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If Theorem 1 is correct, SFC delivers a finite-sample, distribution-free statement — at least 1−α of selected reasoning chains are absolute-coherent-factual — without retraining the generative model or assuming a data distribution.
  • Because every accepted prefix is itself absolute-coherent-factual, an early violation cannot silently contaminate later steps; the first detected violation triggers regeneration from verified context rather than propagation.
  • The guarantee transfers across model architectures and domains by recalibration: per-domain calibration with a conservative minimum-quantile fallback preserves coverage even when domain classification is uncertain.
  • SFC retains more valid claims than post-hoc filtering at the same target factuality — about 80% of generated claims at a 90% target — so the guarantee does not come from throwing out most of the output.
  • The empirical claims imply that a less capable base generator can be upgraded to compete with larger reasoning models by adding the validation wrapper, without changing the underlying model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: the certified object is the selected reasoning chain's internal validity, not its relevance to the query; the paper itself concedes that a logically consistent non-sequitur can be absolute-coherent-factual, so users who need end-to-end correctness should pair SFC with a separate relevance or answer-check layer.
  • Inference: the practical guarantee is only as strong as the validators and graph constructor. The paper reports in Appendix K that the LLM graph builder sometimes returns cyclic graphs and is then replaced by a trivial linear chain; that substitution is not the approximate deducibility graph the theorem assumes, marking the point where the implementation can part company with the bound.
  • Inference: a direct test of the mechanism would be to vary the fidelity of the deducibility graph — or to use a symbolic dependency extractor instead of an LLM — and measure realized coverage; if realized coverage tracks graph fidelity rather than staying flat at 1−α, the graph assumption, not conformal calibration, is the binding constraint.
  • Inference: the same decomposition should transfer to other structured generation tasks with clear laws — mathematical proof steps, chemical reaction conditions, dosage calculations — since each step must be entailed by axioms or prior steps; the paper names such extensions as future work but leaves their formal conditions unexplored.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Scientific Feasibility Control (SFC), a framework for scientific text generation that decomposes outputs into atomic fact-options, builds approximate deducibility graphs, and performs progressive validation with dynamic branching. It claims a finite-sample conformal coverage guarantee on 'absolute-coherent-factuality' (Theorem 1) and reports large empirical gains: 50.1% accuracy on PhyX physics, 91.7% scientific validity, and a 73% reduction in scientific-law violations, across PhyX, MATH, ScienceQA, and ARC Challenge. The key claimed novelty is extending conformal prediction to dynamically evolving dependency graphs while preserving coverage guarantees.

Significance. If the conformal guarantee were established and the empirical evaluation were sound, the paper would address an important problem: providing statistical validity guarantees for structured, dependency-aware scientific reasoning rather than independent claim filtering. The conceptual move from independence-based factuality to coherence- and deducibility-aware validity is sensible, and the dynamic-branching mechanism is practically motivated. However, the central theoretical guarantee is not connected to the implemented algorithm, the proof is deferred to a missing appendix, and the validity metric appears circular. These are load-bearing issues, not presentation concerns. The paper also ships no code or machine-checked proofs, so the empirical claims cannot be independently verified from the manuscript.

major comments (5)
  1. [§5.3, Theorem 1; §5.1, Algorithm 3; Figure 1] Theorem 1 states P[Selected subgraph is absolute-coherent-factual] ≥ 1−α for a 'dynamic graph-structured conformal predictor', but the deployed algorithm accepts a sentence when validation.risk_score ≤ τ (Algorithm 3, line 12), and Figure 1 fixes τ = 0.1. No nonconformity score r(X,Y,G) is defined for SFC, no calibration quantile q̂_α is computed from a calibration set, and no argument shows that the fixed τ equals the ⌈(n+1)(1−α)⌉-th order statistic of calibration scores. The proof sketch only observes that deterministic functions preserve exchangeability; that is necessary but not sufficient for split-conformal coverage, which also requires the threshold to be the calibrated quantile. Thus Eq. (1) does not cover the implemented system.
  2. [§5.3, 'Proof Sketch'; Appendix D; Appendix G] The proof of Theorem 1 is not in the manuscript. The text says 'Complete formal proof provided in Appendix 7', but no such appendix exists. Appendix D is titled 'Proof of Coherent Factuality Theorem' and proves a statement about U_filtered and r(X,Y,U_T) in the setting of Rubin-Toles et al., not Theorem 1. Appendix G proves exchangeability of (X_i,Y_i,G(X_i,Y_i)) under process invariance, but again does not derive coverage for the fixed-threshold progressive rule used in Algorithm 3. The central theoretical claim is therefore unproven as written.
  3. [§5.2, Algorithm 4; Table 1] The 'scientific validity' numbers in the main tables appear to be computed with the same composite validator SFC uses to accept or reject sentences. Algorithm 4 sets is_valid ← (σ_composite ≤ threshold), so rejecting high-risk sentences will mechanically raise the validator's own pass rate. The 91.7% validity figure is therefore circular as a measure of scientific correctness: it reflects the validator's internal risk score, not an independent ground-truth assessment. The paper mentions physics-PhD annotations with κ = 0.82, but no human-validated validity rates are reported for the rows of Table 1, so the circularity is unresolved.
  4. [§6, Table 1; §5.5] The comparison in Table 1 is not controlled. The SFC results are generated with Doubao as the base model (§6: 'We achieve results using Doubao as the base model'), while GPT-4, DeepSeek-R1, o3-mini, and Grok-3 are raw model baselines. No Doubao baseline is reported, so the observed improvements on PhyX and the other benchmarks may be due to the base model rather than to SFC. The claim that SFC 'substantially outperforms' GPT-4 and DeepSeek-R1 is not supported by an apples-to-apples comparison.
  5. [Appendix F, Eq. (4); Appendix K; §4.1] Several implementation details break or are not covered by the exchangeability and graph-quality assumptions. The deducibility weights w1,w2,w3 are learned from validation data (Appendix F, Eq. (4)); validation weights β and γ are 'domain-adaptive'; and dependency graphs are produced by few-shot LLMs, with the manuscript stating that when Llama 'occasionally output cyclic graphs, we simply considered the trivial linear graph' (Appendix K). These choices introduce data-dependent and non-exchangeable components that are not accounted for in Theorem 1 or Appendix G. The theorem's claim to hold 'regardless of the specific dynamic construction process' is therefore contradicted by the need for process invariance and ancestor-connectivity conditions.
minor comments (5)
  1. [Definition 3] The notation 'f_i |= T' in Definition 3 appears reversed: as written it says each fact-option entails the entire scientific ground truth. The intended condition is presumably T |= f_i, i.e., the ground truth entails the fact-option.
  2. [Throughout (e.g., §3.2, §5.2, §5.5, Appendix J)] Multiple appendix pointers are placeholders: 'Appendix 7' appears in §3.2, §5.2, and Appendix J, and §5.5 cites 'Appendix 7 and Appendix 7'. These references need to be resolved before the manuscript can be assessed.
  3. [Table 2] The 'SFC (All Components)' row reports 50.1% accuracy and 91.7% validity, while the '+ Progressive + Scientific' row reports 53.1% and 92.1%. The text claims 'optimal balance' for the full system, but the table does not support that claim; the full system is dominated by a proper subset of components on both reported metrics.
  4. [§3.2, Eq. (2)] The displayed formula for r(X,Y,T) in §3.2 is truncated mid-expression, and Eq. (2) in §5.5 is not well-formed as written: the left-hand side appears to take a probability of an event involving a sample average, and the variables n and I_d are not fully specified.
  5. [§6, Figure 2] The text mentions Galactica as a comparison model in §6, but Galactica does not appear in Table 1. Figure 2 also lacks a precise description of how the displayed curves are computed, which makes the calibration claims hard to verify.

Circularity Check

3 steps flagged

The reported 91.7% 'scientific validity' is the pass rate of the same validator that accepts/rejects claims, and the conformal 'guarantee' restates split conformal for a deterministic graph function without connecting to the fixed-threshold deployed system.

specific steps
  1. self definitional [Figure 1; §5.1 Algorithm 3 line 12; §5.2 Algorithm 4; Table 1]
    "Threshold τ = 0.1 Accept if σ(fi)≤τ, else reject & regenerate ... Coverage Analysis α = 0.1 (90% coverage) Theoretical guarantee maintained (Figure 1); is_valid←σ composite≤threshold (Algorithm 4); Table 1 reports 'Validity' 91.7% for SFC."

    The only operational definition of 'valid' in the system is σ_composite≤threshold, which is exactly the acceptance rule. The same σ_composite is used to admit a sentence (Algorithm 3 line 12) and to define its validity. Thus the headline 91.7% 'scientific validity' and the 73% violation reduction are pass rates of the filter on its own scoring: rejecting everything the validator flags mechanically raises this number. No independent validity label is introduced; the 'prediction' of high validity is the acceptance criterion restated.

  2. self definitional [Definition 3 (§4); §4 'Computational Implementation of Deducibility'; §5.2 Eq. (4)]
    "∀i∈[n], f i |= T ∧ f i is deducible from (f1,...,f i−1), X, T (Definition 3); 'We address this through a multi-tiered framework combining symbolic logical deduction, neural semantic entailment assessment, and domain-specific inference engines'; σ(fi|C) = max{σhard(fi), β·σsoft(fi), γ·σcross(fi)}."

    The target property 'absolute-coherent-factuality' is implemented with the same multi-tiered machinery (symbolic deduction, neural entailment, domain-specific inference) that composes the acceptance score σ(fi|C). Therefore the event 'Selected subgraph is absolute-coherent-factual' in Theorem 1 is, by construction, the event that the subgraph passes the very validator that selected it. The coverage guarantee is not certifying an externally defined scientific property; it is restating the acceptance rule as the target of the guarantee.

  3. other [Theorem 1 proof sketch (§5.3); Algorithm 3; Figure 1]
    "'The key insight is that G(X,Y ) is a deterministic function of (X,Y)... deterministic functions preserve exchangeability... Complete formal proof provided in Appendix 7.' Yet Algorithm 3 accepts only when 'validation.risk_score ≤ τ', with τ fixed at 0.1 in Figure 1."

    The theorem's proof reduces to the standard split-conformal fact that deterministic functions preserve exchangeability; it never defines the nonconformity score r for SFC, never computes the calibrated quantile q̂_α, and never shows that the deployed fixed τ=0.1 equals that quantile. The claimed coverage 'regardless of the specific dynamic construction process' is asserted by stripping away the calibration step, so the formal guarantee for the implemented system is not derived from the theorem. What remains is the tautology that a fixed-threshold filter accepts what it accepts.

full rationale

The paper's architecture—graph construction, dynamic branching, and domain classification—is not inherently circular; Appendix I even attempts to break a classification/validation dependency. The circularity is concentrated in the evaluation and the guarantee. (1) The only operationalized validity measure is the composite validator score that also decides acceptance, so the headline validity and violation-reduction numbers are forced by construction rather than measured against an independent external label. (2) Definition 3's 'absolute-coherent-factuality' is implemented by the same multi-tiered deducibility machinery that forms the acceptance score, making Theorem 1's coverage event a restatement of the acceptance rule. (3) The theorem proof is a bare exchangeability-preservation argument and never connects to the implemented fixed-threshold acceptance rule; no nonconformity score or calibrated quantile is supplied. Accuracy results against external benchmarks (e.g., 50.1% on PhyX) are genuinely external and lower the circularity burden, but the paper's central validity/coverage claims reduce to the filter's own scoring as written.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 2 invented entities

The framework introduces several hand-set thresholds and learned weights, plus strong assumptions about graph constructibility and deducibility. The evaluation metrics share the same validators used for filtering, creating circularity. The conformal guarantee itself rests on exchangeability that is not verified for the actual dynamic, self-correcting pipeline.

free parameters (5)
  • Acceptance threshold τ = 0.1
    Set by hand in §5.1 and Figure 1; used to accept/reject fact-options. Not derived from the conformal quantile, so the coverage guarantee is not linked to the implemented procedure.
  • Domain-adaptive weights β, γ = learned from validation data
    Appear in §5.2 composite score σ = max{σ_hard, β·σ_soft, γ·σ_cross}; Appendix F says weights are learned from validation data, making the score function data-dependent.
  • Deducibility weights w1, w2, w3 = learned from validation data
    Equation (4) in Appendix F weights symbolic, semantic, and domain-specific deducibility; learned from validation data.
  • Dependency threshold θ_dep = not specified
    Algorithm 1 line 8 uses max(sem_dep, math_dep, causal_dep) > θ_dep to add edges; the value is never given.
  • SemanticDependency weights 0.6/0.4 = 0.6, 0.4
    Appendix H Algorithm 7: similarity and entailment are combined as 0.6·similarity + 0.4·entailment, chosen by hand.
axioms (6)
  • ad hoc to paper Superstring Deducibility (Assumption 1): if a claim is deducible from some ordering, it remains deducible after adding more 'good' information.
    Stated in §4.1 as a monotonicity property; load-bearing for graph consistency but not generally true for non-monotonic or context-sensitive scientific reasoning.
  • domain assumption Approximate deducibility graphs satisfying Definition 4 can be constructed for each (X,Y); graphs built by LLM few-shot prompting are valid proxies.
    §4.1.2 assumes constructibility; Appendix K admits Llama outputs cycles that are replaced by linear graphs, violating the ancestor-connectivity requirement.
  • standard math Exchangeability of (X_i, Y_i) in the calibration and test draws.
    Required for split conformal validity; the paper's dynamic branching and learned validator weights may break exchangeability in practice.
  • ad hoc to paper Process invariance: graph construction rules, dependency discovery algorithms, and scoring functions are identical across all examples.
    Definition 8 and Appendix G; empirically asserted on 1000 pairs, but not guaranteed when few-shot prompts or learned weights vary.
  • domain assumption Existence of a fact-option splitter S: Y → 2^F.
    The paper assumes access to a splitter and uses a physics-trained LLM in practice; no formal definition of fact-option boundaries is given.
  • domain assumption Neural and symbolic deducibility approximations faithfully model logical deducibility.
    Central to absolute-coherent-factuality; the paper reports Cohen's κ>0.7 against expert judgments, but this is an approximate surrogate, not ground truth.
invented entities (2)
  • Fact-option no independent evidence
    purpose: Atomic unit of scientific claims for independent validation
    A new decomposition unit defined in the paper; no external falsifiable handle, boundaries are left to an assumed splitter function.
  • Absolute-coherent-factuality no independent evidence
    purpose: A validity notion requiring both absolute correctness and contextual deducibility
    The central validity predicate, defined in terms of the paper's own deducibility engine; no independent benchmark measures it.

pith-pipeline@v1.3.0-alltime-deepseek · 24157 in / 15301 out tokens · 122411 ms · 2026-08-01T20:10:18.369134+00:00 · methodology

0 comments
read the original abstract

Large language models frequently violate fundamental scientific principles when generating technical content, undermining their reliability in scientific applications. We introduce Scientific Feasibility Control SFC, a graph-structured conformal prediction framework that provides statistical guarantees for scientific reasoning validity through progressive absolute-coherent-factuality validation. Our approach decomposes scientific reasoning into atomic absolute-coherent-factuality units requiring both individual correctness against physical laws and logical substantiation from preceding context, addressing the cascade effect where early scientific errors contaminate subsequent reasoning steps. Unlike independence-based methods that treat claims in isolation, SFC models logical dependencies as approximate deducibility graphs and operates through real-time validation with dynamic branching when scientific violations are detected, the system branches to alternative generation paths using verified context as foundation. We demonstrate SFC across established scientific reasoning benchmarks including PhyX multimodal physics, MATH, ScienceQA, and ARC Challenge, achieving 50.1 percent accuracy on PhyX physics reasoning, substantially outperforming recent reasoning models including DeepSeek-R1 49.8 percent and GPT-4 45.8 percent while providing 91.7 percent scientific validity with formal conformal coverage guarantees at alpha equals 0.10 confidence level and reducing scientific law violations by 73 percent across multiple model architectures.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

88 extracted references · 18 linked inside Pith

  1. [1]

    Mediaug: Exploring visual augmentation in medical imaging

    Xuyin Qi, Zeyu Zhang, Canxuan Gang, Hao Zhang, Lei Zhang, Zhiwei Zhang, and Yang Zhao. Mediaug: Exploring visual augmentation in medical imaging. InAnnual Conference on Medical Image Understand- ing and Analysis, pages 218–232. Springer, 2025

    2025

  2. [2]

    Pathohr: Breast cancer survival prediction on high-resolution pathological images.arXiv preprint arXiv:2503.17970, 2025

    Yang Luo, Shiru Wang, Jun Liu, Jiaxuan Xiao, Run- dong Xue, Zeyu Zhang, Hao Zhang, Yu Lu, Yang Zhao, and Yutong Xie. Pathohr: Breast cancer survival prediction on high-resolution pathological images.arXiv preprint arXiv:2503.17970, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Hierarchical multi-scale feature fusion network for multi-center major depres- sive disorder classification with t1-weighted mri

    Zhaoyang Cong, Ziyang Wang, Hao Zhang, Guowei Zheng, Keming Cao, Lina Zhao, Ruipeng Song, Jian- qing Li, and Chengyu Liu. Hierarchical multi-scale feature fusion network for multi-center major depres- sive disorder classification with t1-weighted mri. In Annual International Conference of the IEEE En- gineering in Medicine and Biology Society. IEEE Engine...

    2025

  4. [4]

    Med- conv: Convolutions beat transformers on long-tailed bone density prediction

    Xuyin Qi, C Zeyu Zhang, Huazhan Zheng, Mingxi Chen, Numan Kutaiba, Ruth Lim, Cherie Chiang, Zi En Tham, Xuan Ren, Wenxin Zhang, et al. Med- conv: Convolutions beat transformers on long-tailed bone density prediction. In2025 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2025

    2025

  5. [5]

    Deco-detr: De- coupled cognition detr for efficient open-vocabulary object detection.arXiv preprint arXiv:2604.02753, 2026

    Siheng Wang, Yanshu Li, Bohan Hu, Zhengdao Li, Haibo Zhan, Linshan Li, Weiming Liu, Ruizhi Qian, Guangxin Wu, Hao Zhang, et al. Deco-detr: De- coupled cognition detr for efficient open-vocabulary object detection.arXiv preprint arXiv:2604.02753, 2026

    Pith/arXiv arXiv 2026

  6. [6]

    Tiny scales, great challenges: The limits of multimodal llms in scale recognition

    Jihang Jin, Ronghao Chen, Hao Zhang, Ziyan Liu, Huacan Wang, Qi Ye, and Jingping Liu. Tiny scales, great challenges: The limits of multimodal llms in scale recognition. InProceedings of the 64th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 40619– 40632, 2026

    2026

  7. [7]

    End- to-end story visualization framework with penalty- based evaluation using vision-language models

    Lizheng Zu, Yaoqing Jin, Siyi Cao, Shiwei Suo, Haoxin Lyu, Song Fu, Hui Sun, and Hao Zhang. End- to-end story visualization framework with penalty- based evaluation using vision-language models. In ICASSP 2026-2026 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP), pages 10492–10496. IEEE, 2026

    2026

  8. [8]

    Computing full conformalpredictionsetwithapproximatehomotopy

    Eugene Ndiaye and Ichiro Takeuchi. Computing full conformalpredictionsetwithapproximatehomotopy. Advances in Neural Information Processing Systems, 32, 2019

    2019

  9. [9]

    Practical adversarial multivalid conformal prediction.Advances in neural information process- ing systems, 35:29362–29373, 2022

    Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, Ramya Ramalingam, and Aaron Roth. Practical adversarial multivalid conformal prediction.Advances in neural information process- ing systems, 35:29362–29373, 2022

    2022

  10. [10]

    A reduction from apprenticeship learning to classification.Advances in neural information processing systems, 23, 2010

    Umar Syed and Robert E Schapire. A reduction from apprenticeship learning to classification.Advances in neural information processing systems, 23, 2010

    2010

  11. [11]

    Reliable multilabel classification: Prediction with partial ab- stention

    Vu-Linh Nguyen and Eyke Hullermeier. Reliable multilabel classification: Prediction with partial ab- stention. InProceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5264–5271, 2020

    2020

  12. [12]

    On optimal interpolation in linear regression.Advances in Neu- ral Information Processing Systems, 34:29116–29128, 2021

    Eduard Oravkin and Patrick Rebeschini. On optimal interpolation in linear regression.Advances in Neu- ral Information Processing Systems, 34:29116–29128, 2021

    2021

  13. [13]

    Attribute effi- cient linear regression with distribution-dependent sampling

    Doron Kukliansky and Ohad Shamir. Attribute effi- cient linear regression with distribution-dependent sampling. InInternational Conference on Machine Learning, pages 153–161. PMLR, 2015

    2015

  14. [14]

    Horizon- independent minimax linear regression.Advances in Neural Information Processing Systems, 31, 2018

    Alan Malek and Peter L Bartlett. Horizon- independent minimax linear regression.Advances in Neural Information Processing Systems, 31, 2018

    2018

  15. [15]

    Confor- mal inference is (almost) free for neural networks trained with early stopping

    Ziyi Liang, Yanfei Zhou, and Matteo Sesia. Confor- mal inference is (almost) free for neural networks trained with early stopping. InInternational Con- ference on Machine Learning, pages 20810–20851. PMLR, 2023

    2023

  16. [16]

    Exact inference in structured prediction.Advances in Neural Informa- tion Processing Systems, 32, 2019

    Kevin Bello and Jean Honorio. Exact inference in structured prediction.Advances in Neural Informa- tion Processing Systems, 32, 2019

    2019

  17. [17]

    On structured prediction theory with cali- brated convex surrogate losses.Advances in Neural Information Processing Systems, 30, 2017

    Anton Osokin, Francis Bach, and Simon Lacoste- Julien. On structured prediction theory with cali- brated convex surrogate losses.Advances in Neural Information Processing Systems, 30, 2017

    2017

  18. [18]

    Neural struc- tured prediction for inductive node classification

    Meng Qu, Huiyu Cai, and Jian Tang. Neural struc- tured prediction for inductive node classification. arXiv preprint arXiv:2204.07524, 2022. 11

    Pith/arXiv arXiv 2022

  19. [19]

    Can representation gaps be the key to enhancing robustness in graph-text alignment?arXiv preprint arXiv:2510.12087, 2025

    Heng Zhang, Tianyi Zhang, Yuling Shi, Xiaodong Gu, Yaomin Shen, Zijian Zhang, Yilei Yuan, Hao Zhang, and Jin Huang. Can representation gaps be the key to enhancing robustness in graph-text alignment?arXiv preprint arXiv:2510.12087, 2025

    arXiv 2025

  20. [20]

    Graphgeo: Multi-agent debate framework for visual geo-localization with heterogeneous graph neural networks.arXiv preprint arXiv:2511.00908, 2025

    Heng Zheng, Yuling Shi, Xiaodong Gu, Haochen You, Zijian Zhang, Lubin Gan, Hao Zhang, Wenjun Huang, and Jin Huang. Graphgeo: Multi-agent debate framework for visual geo-localization with heterogeneous graph neural networks.arXiv preprint arXiv:2511.00908, 2025

    arXiv 2025

  21. [21]

    Mitigating generic token dominance in cross-domain foundation model for text-attributed graphs

    Heng Zhang, Haochen You, Zijian Zhang, Lubin Gan, Hao Zhang, Wenjun Huang, and Jin Huang. Mitigating generic token dominance in cross-domain foundation model for text-attributed graphs. InIn- ternational Conference on Database Systems for Ad- vanced Applications, pages 251–265. Springer, 2026

    2026

  22. [22]

    Lan- guage models with conformal factuality guaran- tees.arXiv preprint arXiv:2402.10978, 2024

    Christopher Mohri and Tatsunori Hashimoto. Lan- guage models with conformal factuality guaran- tees.arXiv preprint arXiv:2402.10978, 2024. URL https://arxiv.org/abs/2402.10978

    Pith/arXiv arXiv 2024

  23. [23]

    Cherian, Isaac Gibbs, and Emmanuel J

    John J. Cherian, Isaac Gibbs, and Emmanuel J. Candès. Large language model validity via enhanced conformal prediction methods, 2024. URL https: //arxiv.org/abs/2406.09714

    Pith/arXiv arXiv 2024

  24. [24]

    Multi-group uncertainty quantification for long-form text genera- tion, 2024

    Terrance Liu and Zhiwei Steven Wu. Multi-group uncertainty quantification for long-form text genera- tion, 2024. URL https://arxiv.org/abs/2407.21057

    Pith/arXiv arXiv 2024

  25. [25]

    Batch multivalid conformal prediction

    Christopher Jung, Georgy Noarov, Ramya Rama- lingam, and Aaron Roth. Batch multivalid conformal prediction. InThe Eleventh International Confer- ence on Learning Representations, 2023. URLhttps: //openreview.net/forum?id=Dk7QQp8jHEo

    2023

  26. [26]

    Probability-entropy calibration: An elastic indicator for adaptive fine-tuning.arXiv preprint arXiv:2602.01745, 2026

    Wenhao Yu, Shaohang Wei, Jiahong Liu, Yifan Li, Minda Hu, Aiwei Liu, Hao Zhang, and Irwin King. Probability-entropy calibration: An elastic indicator for adaptive fine-tuning.arXiv preprint arXiv:2602.01745, 2026

    Pith/arXiv arXiv 2026

  27. [27]

    Jaakkola, and Regina Barzilay

    Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi S. Jaakkola, and Regina Barzilay. Conformal language modeling. InThe Twelfth International Conference on Learning Rep- resentations, 2024. URL https://openreview.net/ forum?id=pzUhfQ74c5

    2024

  28. [28]

    Chain-of-thought prompting elicits rea- soning in large language models.Advances in neu- ral information processing systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits rea- soning in large language models.Advances in neu- ral information processing systems, 35:24824–24837, 2022

    2022

  29. [29]

    Tree of thoughts: Deliberate problem solving with large language models.Advances in neural informa- tion processing systems, 36:11809–11822, 2023

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models.Advances in neural informa- tion processing systems, 36:11809–11822, 2023

    2023

  30. [30]

    Pathsymphony: Harmoniz- ing symbolic planning and large language mod- els for curriculum-guided mathematical reasoning

    Mingqiao Mo, Hao Zhang, Yunlong Tan, Bo Huang, and Chengcheng Li. Pathsymphony: Harmoniz- ing symbolic planning and large language mod- els for curriculum-guided mathematical reasoning. Knowledge-Based Systems, page 116598, 2026

    2026

  31. [31]

    Galactica: A large language model for science.arXiv preprint arXiv:2211.09085, 2022

    Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science.arXiv preprint arXiv:2211.09085, 2022

    Pith/arXiv arXiv 2022

  32. [32]

    Scibert: A pretrained language model for scientific text.arXiv preprint arXiv:1903.10676, 2019

    Iz Beltagy, Kyle Lo, and Arman Cohan. Scibert: A pretrained language model for scientific text.arXiv preprint arXiv:1903.10676, 2019

    Pith/arXiv arXiv 1903

  33. [33]

    Phyx: Does your model have the" wits" for physical reasoning?arXiv preprint arXiv:2505.15929, 2025

    HuiShen, TaiqiangWu, QiHan, YuntaHsieh, Jizhou Wang, Yuyue Zhang, Yuxin Cheng, Zijian Hao, Yuansheng Ni, Xin Wang, et al. Phyx: Does your model have the" wits" for physical reasoning?arXiv preprint arXiv:2505.15929, 2025

    Pith/arXiv arXiv 2025

  34. [34]

    Learn to explain: Multimodal reasoning via thought chains for science question answering

    Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. InThe 36th Conference on Neural Information Processing Systems (NeurIPS), 2022

    2022

  35. [35]

    Neuro- symbolic visual reasoning: Disentangling

    Saeed Amizadeh, Hamid Palangi, Alex Polozov, Yichen Huang, and Kazuhito Koishida. Neuro- symbolic visual reasoning: Disentangling. InIn- ternational Conference on Machine Learning, pages 279–290. Pmlr, 2020

    2020

  36. [36]

    Film: Visual rea- soning with a general conditioning layer

    Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual rea- soning with a general conditioning layer. InProceed- ings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  37. [37]

    Enhancing advanced visual reasoning ability of large language models.arXiv preprint arXiv:2409.13980, 2024

    Zhiyuan Li, Dongnan Liu, Chaoyi Zhang, Heng Wang, Tengfei Xue, and Weidong Cai. Enhancing advanced visual reasoning ability of large language models.arXiv preprint arXiv:2409.13980, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073, 2022

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073, 2022

    Pith/arXiv arXiv 2022

  39. [39]

    Chateval: Towards better llm-based evalua- tors through multi-agent debate.arXiv preprint arXiv:2308.07201, 2023

    Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better llm-based evalua- tors through multi-agent debate.arXiv preprint arXiv:2308.07201, 2023. 12

    Pith/arXiv arXiv 2023

  40. [40]

    Encouraging divergent thinking in large language models through multi-agent debate

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118, 2023

    Pith/arXiv arXiv 2023

  41. [41]

    Multimodal multi-agent empowered legal judgment prediction

    Zhaolu Kang, Junhao Gong, Qingxi Chen, Hao Zhang, Jiaxin Liu, Rong Fu, Zhiyuan Feng, Yuan Wang, Simon Fong, and Kaiyue Zhou. Multimodal multi-agent empowered legal judgment prediction. arXiv preprint arXiv:2601.12815, 2026

    arXiv 2026

  42. [42]

    Sugar-coated poison: Be- nign generation unlocks llm jailbreaking.arXiv preprint arXiv:2504.05652, 2025

    Yu-Hang Wu, Yu-Jie Xiong, Hao Zhang, Jia-Chen Zhang, and Zheng Zhou. Sugar-coated poison: Be- nign generation unlocks llm jailbreaking.arXiv preprint arXiv:2504.05652, 2025

    arXiv 2025

  43. [43]

    Enhancing intent understanding for ambiguous prompt: A human-machine co-adaption strategy.arXiv preprint arXiv:2501.15167, 2025

    Yangfan He, Jianhui Wang, Yijin Wang, Yan Zhong, Xinyuan Song, Junjiang Lin, Xinhang Yuan, Jingqun Tang, Yi Xin, Hao Zhang, et al. Enhancing intent understanding for ambiguous prompt: A human-machine co-adaption strategy.arXiv preprint arXiv:2501.15167, 2025

    arXiv 2025

  44. [44]

    Missing-by-design: Certifiable modality dele- tion for revocable multimodal sentiment analysis

    Rong Fu, Ziming Wang, Chunlei Meng, Jiaxuan Lu, Jiekai Wu, Kangan Qian, Hao Zhang, and Simon Fong. Missing-by-design: Certifiable modality dele- tion for revocable multimodal sentiment analysis. arXiv preprint arXiv:2602.16144, 2026

    Pith/arXiv arXiv 2026

  45. [45]

    Efficient toxic content detection by bootstrapping and distilling large language models

    Jiang Zhang, Qiong Wu, Yiming Xu, Cheng Cao, Zheng Du, and Konstantinos Psounis. Efficient toxic content detection by bootstrapping and distilling large language models. InProceedings of the AAAI conference on artificial intelligence, volume 38, pages 21779–21787, 2024

    2024

  46. [46]

    End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks

    Richard Cheng, Gábor Orosz, Richard M Murray, and Joel W Burdick. End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks. InProceedings of the AAAI conference on artificial intelligence, volume 33, pages 3387–3395, 2019

    2019

  47. [47]

    Mirostat: A neural text decoding algorithm that directly controls perplexity.arXiv preprint arXiv:2007.14966, 2020

    Sourya Basu, Govardana Sachitanandam Ramachan- dran, Nitish Shirish Keskar, and Lav R Varsh- ney. Mirostat: A neural text decoding algorithm that directly controls perplexity.arXiv preprint arXiv:2007.14966, 2020

    Pith/arXiv arXiv 2007

  48. [48]

    A theoretical understand- ing of self-correction through in-context alignment

    Yifei Wang, Yuyang Wu, Zeming Wei, Stefanie Jegelka, and Yisen Wang. A theoretical understand- ing of self-correction through in-context alignment. Advances in Neural Information Processing Systems, 37:89869–89912, 2024

    2024

  49. [49]

    Pdtrim: Targeted pruning for prefill- decode disaggregation in inference

    Hao Zhang, Lyu Mengsi, Zhuo Chen, Yulong Ao, and Yonghua Lin. Pdtrim: Targeted pruning for prefill- decode disaggregation in inference. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 39673–39690, 2026

    2026

  50. [50]

    Adaptive prompt optimization for open-ended tasks: Uncer- tainty preference as a secondary signal

    Shuyang Zhang, Zhixuan Liu, Zhichen Dong, Hao Zhang, Chaochao Lu, and Chao Yang. Adaptive prompt optimization for open-ended tasks: Uncer- tainty preference as a secondary signal. InFindings of the Association for Computational Linguistics: ACL 2026, pages 33883–33891, 2026

    2026

  51. [51]

    Protoflow: Mitigating forgetting in class-incremental remote sensing seg- mentation via low-curvature prototype flow.arXiv preprint arXiv:2604.03212, 2026

    Jiekai Wu, Rong Fu, Chuangqi Li, Zijian Zhang, Guangxin Wu, Hao Zhang, Shiyin Lin, Jianyuan Ni, Yang Li, Dongxu Zhang, et al. Protoflow: Mitigating forgetting in class-incremental remote sensing seg- mentation via low-curvature prototype flow.arXiv preprint arXiv:2604.03212, 2026

    Pith/arXiv arXiv 2026

  52. [52]

    Shieldedcode: Learning robust representations for virtual machine protected code.arXiv preprint arXiv:2601.20679, 2026

    Mingqiao Mo, Yunlong Tan, Hao Zhang, Heng Zhang, and Yangfan He. Shieldedcode: Learning robust representations for virtual machine protected code.arXiv preprint arXiv:2601.20679, 2026

    arXiv 2026

  53. [53]

    Conformal language model reasoning with coherent factuality

    Maxon Rubin-Toles, Maya Gambhir, Keshav Ramji, Aaron Roth, and Surbhi Goel. Conformal language model reasoning with coherent factuality. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview. net/forum?id=AJpUZd8Clb

    2025

  54. [54]

    Measuring mathematical problem solving with the math dataset.NeurIPS, 2021

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset.NeurIPS, 2021

    2021

  55. [55]

    The kinetic energy of the moving object equals 1 2mv2 where m = 2.0kg and v = 3.0m/s, giving KE = 9.0J

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457v1, 2018. 13 Appendix Appendix A: Theoretical Founda- tions A.1 Why Fact-Options Beyond Graph De- pendencies The effectiveness of our fact-option deco...

    Pith/arXiv arXiv 2018

  56. [56]

    More formally, Mohri and Hashimoto

    application of conformal prediction to language model outputs. More formally, Mohri and Hashimoto

  57. [57]

    frame factuality in terms of entailment by the ground truth. 14 Algorithm 3Progressive Scientific Generation with Validation 1: Input:Problem data P, scientific context C, thresholdτ, max iterationsM 2: Output:Scientifically valid response R, valida- tion statisticsS 3: Initialize verified_context←∅ , response←∅ , attempts←1 4:whileattempts≤Mdo 5:solution...

  58. [58]

    Data Requirements: Each domain requires suffi- cient calibration examples

  59. [59]

    Domain Classification: Input queries must be correctly classified

  60. [60]

    Coverage Gaps: Novel interdisciplinary fields may lack appropriate calibration Mitigation Strategies and Theoretical Guaran- tees Graceful Degradation: Our framework ad- dresses these limitations through several mechanisms: Conservative Fallback: When domain classifica- tion is uncertain or calibration data is insufficient, the system defaults to universa...

  61. [61]

    Universal operation achieves 89.1% validity with guaranteed coverage

  62. [62]

    Domain adaptation provides 2.6% validity im- provement

  63. [63]

    Universal operation maintains strong guarantees while domain adaptation provides incremental improvements

    Classification errors result in minimal degradation (1.3%) 17 Configuration Accuracy Validity Universal Only 47.3 89.1 Domain-Adaptive 50.1 91.7 Configuration Coverage Mixed Classification 0.91 Conservative Fallback 0.93 Table 4 |Performance comparison across domain knowl- edge scenarios on PhyX dataset. Universal operation maintains strong guarantees whi...

  64. [64]

    de- ducible from( f1,...,f i−1),X,T

    Conservative fallback maintains coverage guaran- tees with slight accuracy trade-off Conclusion: Balanced Approach to Universality The domain-adaptive implementation represents an engineering optimization rather than a theoretical ne- cessity. While it introduces practical considerations regarding domain-specific calibration, these limita- tions are mitig...

  65. [65]

    Contextual encoding of accumulated factsCi−1 and inputX 18

  66. [66]

    Claim representation offi

  67. [67]

    Attention-based reasoning over context-claim re- lationships

  68. [68]

    Output probability calibration using Platt scaling Evidence Sufficiency Analysis: We assess whether the accumulated context provides sufficient evidence through information-theoretic measures: Sufficiency(fi|Ci−1) = 1− H(fi|Ci−1) H(fi) where H(·)represents entropy, measuring how much uncertainty aboutfi is resolved by the context. Tier 3: Domain-Specific ...

  69. [69]

    Implicit Knowledge Gaps: When reasoning re- quires unstated domain knowledge

  70. [70]

    Multi-step Inference: When deduction requires multiple intermediate steps

  71. [71]

    Analogical Reasoning: When arguments rely on scientific analogies We address these through conservative thresholding and robust fallback mechanisms. Validation Against Human Expert Judgments Expert Agreement Analysis: We achieve substan- tial agreement with domain experts: •Physics: Cohen’sκ= 0.78 •Chemistry: Cohen’sκ= 0.71 •Mathematics: Cohen’sκ= 0.82 Ap...

  72. [72]

    Pattern extraction:O(1)using pre-computed fea- tures

  73. [73]

    Template matching:O(logk )using binary search on sorted templates

  74. [74]

    after X, then Y

    Template application: O(T )per matched tem- plate, with at mostO(1)matches per unit Total complexity overn units: O(n(logk +T )) = O(nlogk+nT). For typical scientific reasoning withk≤ 100tem- plates and T≤ 10operations per template, this achieves practical efficiency. Empirical Validation of Process Invariance We validate that our dependency discovery fun...

  75. [75]

    Hierarchical approach achieves near-optimal accu- racy while maintaining high stability

  76. [76]

    Interdisciplinary portfolio construction signifi- cantly outperforms single-domain forcing

  77. [77]

    Content-agnostic methods provide more reliable coverage guarantees

  78. [78]

    Conservative aggregation preserves theoretical properties with minimal performance loss Addressing Complex Interdisciplinary Cases For the most challenging interdisciplinary queries (e.g., biophysics, computational chemistry, mathe- matical physics), we provide three specialized strate- gies: Strategy 1 - Hierarchical Decomposition: Break complex queries ...

  79. [79]

    Highly Abstract Queries: Pure theoretical ques- tions may not exhibit clear domain markers

  80. [80]

    Template Coverage: Template library requires periodic updates for new query types Future Research: Investigation of unsupervised do- main discovery for emerging fields and adaptive tem- plate learning for novel query patterns. Appendix J: Costs Associated with GPT Queries and Running on Llama-3.1-70B-Instruct Cost and reproducibility.We replicated our mai...

    2000

Showing first 80 references.