Pith. sign in

REVIEW 2 major objections 4 minor 44 references

Visual Grounding in Zero-Shot Vision-Language Control

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Zero-shot vision-language controllers do not reliably ground their decisions in visual input, and a constant-SLOW policy can beat a scripted geometric controller.

desk verdict A careful negative result that reshapes how to read VLM-control scores, but the headline generalization is narrower than the paper's framing because it rests on a single prompt per model. read the letter →

arxiv 2608.06154 v1 pith:N73JMEGD submitted 2026-08-06 cs.RO cs.AIcs.CV

J. de Curtò , Dayani Plasencia , Diego Sánchez , I. de Zarzà This is my paper · ORCID
classification cs.ROcs.AIcs.CV
keywords vision-languagemodelszero-shotcontrolvisualgroundinginputablationmetamorphictestinglane-axisreflectionsymmetryconsensushazardmonitoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that successful trajectories from VLM controllers do not prove visual grounding: simulator dynamics and conservative action priors can produce good scores without the model using the camera at all. It does this with an input-ablation battery that manipulates only the image—blank, noisy, shuffled, repeated, removed, or mirrored—and measures whether decisions change. Across nine direct-action models, six structured local VLMs, and a VLM–MPC hierarchy, the direct-control results are largely negative: constant policies beat a scripted geometric controller, several models are image-invariant, and models that recognize longitudinal hazards fail to swap LEFT and RIGHT under reflection. No local VLM meets the joint longitudinal and lateral grounding criteria, yet an image-only deterministic positive control recovers the lead gap to 0.090 m mean absolute error with exact mirror equivariance. The paper concludes that current VLMs should be used as bounded, selective hazard assistants, not monolithic zero-shot controllers.

What carries the argument

The central object is the input-ablation battery, a set of eight image conditions that change only the image while holding the prompt fixed: true frame, repeated true frame, blank, noise, shuffled frame, cross-scenario frame, text-only, and lane-axis reflection. Agreement with the real-frame action is measured with a condition–action chi-square test and Cramér's V as the primary statistic, with a noise floor from the repeated-frame condition to discount self-inconsistency. The battery separates whether a model uses the image at all, whether it recovers task-relevant spatial information, and whether that information supports the right control decision; the reflection condition specifically tests whether LEFT and RIGHT exchange under the mirror symmetry. A second mechanism is the symmetry-consensus guardian, which combines two models' hazard votes on original and reflected views and abstains on splits, recovering a longitudinal signal when monolithic grounding fails.

What would settle it

Running the same battery on a VLM that, on a sufficiently large set of directional frames, swaps LEFT and RIGHT under lane-axis reflection at a rate near 1.0 while also passing the image-dependence and non-visual-baseline checks would directly contradict the claim that no tested VLM meets the joint longitudinal and lateral grounding criteria.

Watch

Extended reading notes

Core claim

The central claim is that visual grounding and task success come apart in VLM control: a constant-SLOW policy outperforms a scripted geometric controller on the primary driving simulator, several edge and hosted models are image-invariant or nearly constant, and none of the six structured local VLMs passes the joint longitudinal and lateral grounding gate. The failures are not a pipeline artifact, because an image-only deterministic positive control recovers the lead gap with 0.090 m mean absolute error and exact mirror equivariance. A symmetry-consensus guardian that freezes two models and a 2-of-4 hazard vote on 16 calibration frames then reaches 0.954 balanced accuracy on a 272-frame holdout, and with abstention on ties reaches 0.973 at 0.824 coverage; when deterministic perception keeps lateral authority, modular replay agrees with the renderer policy on 0.934 of holdout frames. The paper concludes that current VLMs are credible as bounded, selective longitudinal hazard monitors, not as monolithic zero-shot controllers.

Load-bearing premise

The tested collection of two simulators, two embodiments, fifteen models, and a small set of prompt templates is assumed to represent current VLM controllers broadly enough that the conclusion about no dependable general-purpose zero-shot controller holds beyond these settings.

Editorial extensions

If this is right

  • Aggregate metrics like distance, reward, and crash rate cannot certify perception in VLM control; evaluations must include blind inputs, repeated identical inputs, mirror transformations, and non-visual baselines.
  • Prompt formulation gates measured grounding: few-shot and anti-degeneracy instructions can collapse a model's action distribution, so single-prompt null results are not evidence about a model.
  • A modular architecture that gives deterministic perception lateral authority and VLMs only longitudinal supervision can reach high agreement with an exactly mirror-equivariant reference.
  • Adding a low-level MPC downstream of a VLM improves tracking but cannot reconstruct geometry that the high-level visual intent lacks.
  • Current VLMs are most credible as selective hazard monitors with abstention, not as general-purpose zero-shot controllers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the lane-axis reflection test could become a standard metamorphic probe for lateral grounding in any VLM control benchmark, since it isolates directional priors that image-dependence tests miss.
  • Beyond the paper, the abstention behavior—treating 2-of-4 disagreement as a rejection signal—suggests a general uncertainty mechanism for VLM-based perception modules in safety-critical pipelines.
  • Beyond the paper, the observation that the cheapest hosted models were among the most grounded implies that deployment cost is not a reliable proxy for visual grounding, and future model selection should include direct grounding probes.
  • Beyond the paper, the prompt-collapse findings imply that action-vocabulary design, specifically which action is most salient in a format, is itself a confound in VLM control benchmarks separate from the model's perceptual ability.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper investigates whether vision-language models (VLMs) used as zero-shot controllers actually ground their decisions in visual input, rather than exploiting simulator dynamics and conservative action priors. The authors run an input-ablation battery (blind images, repeated identical inputs, lane-axis reflection, non-visual baselines, pipeline-integrity checks) across direct-action, structured local, hosted, and VLM-MPC setups, scoring 32,874 calls. The direct-control results are largely negative: a constant-SLOW policy beats a scripted geometric controller, several models are image-invariant or nearly constant, and models that recognize longitudinal hazards fail to exchange LEFT and RIGHT under reflection. A deterministic image-only positive control recovers lead gap with 0.090 m MAE and exact mirror equivariance, and a leakage-controlled symmetry-consensus guardian reaches 0.954 held-out balanced accuracy (0.973 at 0.824 coverage with abstention). The paper concludes that current VLMs are useful as bounded, selective hazard monitors rather than monolithic zero-shot controllers.

Significance. If the results hold, the paper makes a strong methodological contribution: it provides a reusable battery of groundedness tests for VLM control, with unusually careful statistical controls including cluster-bootstrap confidence intervals, permutation tests, Holm correction, noise-floor adjustment, and a positive control that validates the stimulus and interface. The negative results for the exact models and prompts tested are robust, and the paper explicitly separates 'uses the image', 'recovers task-relevant geometry', and 'supports the correct control decision'—a useful decomposition for the field. The paper also ships released code, data, and templates, and the guardian result is supported by a 16-frame calibration/holdout split plus nested leave-one-episode-out stability. The main limitation is that the headline generalization about 'current VLMs' is prompt-conditional, and the paper's own Section V-F warns against single-prompt null results.

major comments (2)
  1. [Section V-F, Tables III, VIII, IX] The central generalization that no local VLM meets the joint longitudinal and lateral grounding criteria, and hence that current VLMs are not dependable zero-shot controllers, is prompt-conditional, and the paper itself states in Section V-F that 'any claim that a VLM cannot ground control must be qualified by prompt formulation; a single-prompt null result is not evidence about the model.' Yet Table III (structured local arm) and Table VIII (hosted mirror tests) report a single pinned template per model, while the prompt battery in Table IX is applied only to Qwen2.5-VL-72B, MiniCPM-V-4.5, and SmolVLM. Table IX shows prompt choice can flip modal share from 0.37 to 0.96 (Qwen2.5-VL-72B), from 0.98 to 0.49 (MiniCPM-V-4.5), and from 0.89 to 0.48 (SmolVLM), so the 'no VLM meets the criteria' claim is underdetermined by the tested prompts. The authors should either apply the prompt battery, including reflection-explicit prompts, to the six structured local models whose failures drive the conclusion, or explicitly restrict the conclusion to the evaluated prompt set and model sample.
  2. [Section IV-E, V-C, Abstract] The MPC diagnostic is explicitly missing reproducibility support: Section IV-E states that 'the exact model digest and the separate ablation driver were not present in the returned bundle, so this arm is explicitly exploratory,' yet the abstract and Section V-C present the result as a finding that low-level optimization cannot recover missing geometry from collapsed visual intent. The Code Availability section promises that all code, data, and experimental artifacts are publicly available, which is in tension with the omitted driver and digest. Since this arm is not needed for the direct-control conclusion, the paper can be fixed by either releasing the missing artifacts or clearly labeling the MPC result as preliminary in the abstract and conclusion.
minor comments (4)
  1. [Section V-A, Table III] The positive-control row in Table III reports a crash rate of 0.167, which might confuse readers because the positive control is presented as validating the visual stimulus; a sentence explaining why the deterministic renderer policy still crashes (e.g., lateral authority or episode-level failures) would help.
  2. [Section IV-B] The phrase 'The original study contributes 23,416 model queries' is ambiguous about whether this paper is a follow-up to a previously published study; please clarify the relation to prior work or remove the phrase.
  3. [Table IX] The prompt-ablation table does not specify which frames or scenarios the 100 queries per cell were drawn from, making it difficult to compare the modal shares with the main battery results; adding a one-sentence description of the query set would improve reproducibility.
  4. [Section V-D, Table VI] The claim that two model–platform cells produce 'numerically identical trajectories' is interesting, but the paper does not state whether this was verified only in highway-env or also in the drone simulator; please specify the simulator for that observation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the held-out guardian evaluation, the deterministic positive control, and the baseline ladder are genuinely predictive, and the paper's prompt-sensitivity caveat is a limitation rather than a circular derivation.

full rationale

The paper's derivation chain is self-contained and does not reduce to its own inputs. The central positive claim, the symmetry-consensus guardian, is evaluated on 272 frames that are disjoint from the 16 calibration frames on which the model pair and 2-of-4 threshold were frozen; this is a standard train/holdout split, not a fitted parameter renamed as a prediction, and the paper explicitly documents the leakage controls (Section IV-D, Section V-B). The deterministic positive control estimates lead gaps from RGB using a renderer legend and ruler, then reports 0.090 m MAE on visible gaps; it is a stimulus-information sanity check, not a model output fitted to the target. The negative direct-control results are empirical comparisons against constant, random, and scripted baselines, with no equation or criterion defined in terms of the conclusion. The paper's own prompt-ablation section (V-F) explicitly warns that 'a single-prompt null result is not evidence about the model,' which is an acknowledged representativeness limitation and a correctness risk for the generality of the negative claim, but it does not make the per-prompt measurements circular. Self-citations [25], [26] are methodological background on metamorphic testing and are not load-bearing: the battery is fully specified in Section III and independently executable. No load-bearing step is equivalent to its inputs by construction, and no prediction is forced by a self-citation chain.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims are empirical; the ledger records thresholds and assumptions rather than derived constants. No new physical or conceptual entities are introduced.

free parameters (4)
  • guardian vote threshold = 2-of-4 hazard votes
    Selected on 16 calibration frames from six local VLMs; frozen before reading the 272-frame holdout.
  • lead gap hazard threshold = 18 m
    Defines the hazard class in contingency tests and mutual information binning; chosen by authors.
  • admission gate thresholds = parse >= 0.95, acc >= 0.75, balanced acc >= 0.75, scene/action equivariance >= 0.65/0.75
    Hand-set criteria controlling which models enter the structured controller evaluation.
  • prompt token budgets = per-model measured probes (e.g., 4096 for Kimi models)
    Set from measured truncation probes rather than a global budget; disclosed as configuration, but affects parse rates.
assumptions (4)
  • domain assumption Lane-axis reflection must exchange LEFT and RIGHT decisions if the controller is laterally grounded.
    The mirror probe in Section V assumes this metamorphic relation; the positive control's exact mirror equivariance supports it.
  • domain assumption The real_repeat condition measures the model's stochastic noise floor at temperature 0.
    Used to compute IDIadj and to classify Kimi-K3 as self-inconsistent; Section III.
  • domain assumption Simulator reward can be obtained without perception (shortcut policies exist).
    The constant-SLOW baseline in Section V demonstrates this in highway-env.
  • ad hoc to paper The tested frame bank, episodes, and prompts are representative of the target claim about current VLMs.
    The conclusion generalizes beyond the two simulators and 15 models; Section VI.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Grounding in Zero-Shot Vision-Language Control." pith.science (2026). https://pith.science/paper/N73JMEGD

@misc{pith2026260806154,
  author       = {Pith},
  title        = {Pith review of: Visual Grounding in Zero-Shot Vision-Language Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N73JMEGD}},
  note         = {Machine review of arXiv:2608.06154}
}
read the original abstract

Vision-language models (VLMs) are increasingly used as zero-shot controllers, but successful trajectories do not necessarily show that decisions are grounded in visual input: simulator dynamics and conservative action priors can produce favourable scores without meaningful perception. We investigate this with an input-ablation battery: blind-image controls, repeated identical inputs, lane-axis reflection, non-visual baselines, and pipeline-integrity checks. Across nine direct-action models, six structured local VLMs, and an exploratory VLM-MPC hierarchy, we analyse 32,874 scored calls over two embodiments and three simulators. The direct-control results are largely negative: a constant-SLOW policy outperforms a scripted geometric controller, several models are image-invariant or nearly constant, and models that recognize longitudinal hazards still fail to transform LEFT and RIGHT under reflection. No local VLM meets the joint longitudinal and lateral grounding criteria. However, an image-only deterministic positive control estimates the lead gap with 0.090 m MAE and exact mirror equivariance, confirming the stimuli carry sufficient visual information; the failures are modular, not universal. A post-hoc, leakage-controlled symmetry-consensus guardian selects two models from 16 calibration frames and freezes a 2-of-4 hazard vote across original and reflected views. On 272 held-out frames it reaches 0.954 balanced accuracy (episode-cluster bootstrap 95% CI [0.895,0.990]); nested leave-one-episode-out recovers the same pair and threshold in all 12 folds. Abstaining on ties raises committed balanced accuracy to 0.973 at 0.824 coverage. With deterministic perception retaining lateral authority, offline modular replay achieves 0.934 action agreement and exact mirror equivariance. These results support current VLMs as bounded, selective hazard assistants, not monolithic zero-shot controllers.

Figures

Figures reproduced from arXiv: 2608.06154 by the authors.

Figure 1
Figure 1. Visual interface used by the structured controller. From left to right: the released [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Action entropy against mutual information with scene geometry. High [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Agreement with the real-frame action across blind-input conditions, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 25 canonical work pages

  1. [1]

    Learning transferable visual models from natural language supervision,

    A. Radfordet al., “Learning transferable visual models from natural language supervision,” inProc. 38th Int. Conf. Machine Learning (ICML), vol. 139, 2021, pp. 8748–8763

    work page 2021

  2. [2]

    Improved baselines with visual instruction tuning,

    H. Liu, C. Li, Y . Li, and Y . J. Lee, “Improved baselines with visual instruction tuning,”arXiv:2310.03744, 2023

    arXiv 2023

  3. [3]

    Qwen2-VL: Enhancing vision-language model’s per- ception of the world at any resolution,

    P. Wanget al., “Qwen2-VL: Enhancing vision-language model’s per- ception of the world at any resolution,”arXiv:2409.12191, 2024

    arXiv 2024

  4. [4]

    Qwen2.5-VL technical report,

    S. Baiet al., “Qwen2.5-VL technical report,”arXiv:2502.13923, 2025

    arXiv 2025

  5. [5]

    MiniCPM-V: A GPT-4V level MLLM on your phone,

    Y . Yaoet al., “MiniCPM-V: A GPT-4V level MLLM on your phone,” arXiv:2408.01800, 2024

    arXiv 2024

  6. [6]

    Qwen3-VL technical report,

    S. Baiet al., “Qwen3-VL technical report,”arXiv:2511.21631, 2025

    arXiv 2025

  7. [7]

    Gemma 4 technical report,

    Gemma Team, “Gemma 4 technical report,”arXiv:2607.02770, 2026

    arXiv 2026

  8. [8]

    Qwen3.5: Towards native multimodal agents,

    Qwen Team, “Qwen3.5: Towards native multimodal agents,” technical release, 2026

    work page 2026

Show all 44 references
  1. [9]

    Introducing Mistral 3,

    Mistral AI, “Introducing Mistral 3,” 2025

    2025

  2. [10]

    MiniCPM-V 4.5: Cooking efficient MLLMs via architec- ture, data, and training recipe,

    T. Yuet al., “MiniCPM-V 4.5: Cooking efficient MLLMs via architec- ture, data, and training recipe,”arXiv:2509.18154, 2025

    2025 arXiv

  3. [11]

    SmolVLM: Redefining small and efficient multi- modal models,

    A. Marafiotiet al., “SmolVLM: Redefining small and efficient multi- modal models,”arXiv:2504.05299, 2025

    2025 arXiv

  4. [12]

    NavGPT: Explicit reasoning in vision- and-language navigation with large language models,

    G. Zhou, Y . Hong, and Q. Wu, “NavGPT: Explicit reasoning in vision- and-language navigation with large language models,” inProc. AAAI Conf. Artificial Intelligence, vol. 38, no. 7, 2024, pp. 7641–7649

    2024

  5. [13]

    VLM-Social-Nav: Socially aware robot navigation through scoring using vision-language models,

    D. Song, J. Liang, A. Payandeh, A. H. Raj, X. Xiao, and D. Manocha, “VLM-Social-Nav: Socially aware robot navigation through scoring using vision-language models,”IEEE Robotics and Automation Letters, vol. 10, no. 1, pp. 508–515, 2024

    2024

  6. [14]

    VLM-GroNav: Robot navigation using physically grounded vision-language models in outdoor environments,

    M. Elnooret al., “VLM-GroNav: Robot navigation using physically grounded vision-language models in outdoor environments,” inProc. IEEE Int. Conf. Robotics and Automation (ICRA), 2025, pp. 2391–2398

    2025

  7. [15]

    MapNav: A novel memory representation via annotated semantic maps for VLM-based vision-and-language navigation,

    L. Zhanget al., “MapNav: A novel memory representation via annotated semantic maps for VLM-based vision-and-language navigation,” in Proc. 63rd Annual Meeting Assoc. Computational Linguistics, 2025, pp. 13032–13056

    2025

  8. [16]

    HazardVLM: A video language model for real-time hazard description in automated driving systems,

    D. Xiao, M. Dianati, P. Jennings, and R. Woodman, “HazardVLM: A video language model for real-time hazard description in automated driving systems,”IEEE Trans. Intelligent Vehicles, 2024

    2024

  9. [17]

    LLM-powered cooperative perception framework for mixed UA V-vehicle platoons,

    J. de Curt `o and I. de Zarz `a, “LLM-powered cooperative perception framework for mixed UA V-vehicle platoons,” inProc. 10th Int. Conf. Fog and Mobile Edge Computing (FMEC), 2025, pp. 282–289

    2025

  10. [18]

    Semantic scene understand- ing with large language models on unmanned aerial vehicles,

    J. de Curt `o, I. de Zarz`a, and C. T. Calafate, “Semantic scene understand- ing with large language models on unmanned aerial vehicles,”Drones, vol. 7, no. 2, art. 114, 2023

    2023

  11. [19]

    Shortcut learning in deep neural networks,

    R. Geirhoset al., “Shortcut learning in deep neural networks,”Nature Machine Intelligence, vol. 2, no. 11, pp. 665–673, 2020

    2020

  12. [20]

    Beyond accuracy: Behavioral testing of NLP models with CheckList,

    M. T. Ribeiro, T. Wu, C. Guestrin, and S. Singh, “Beyond accuracy: Behavioral testing of NLP models with CheckList,” inProc. 58th Annual Meeting Assoc. Computational Linguistics, 2020, pp. 4902–4912

    2020

  13. [21]

    Holistic evaluation of language models,

    P. Lianget al., “Holistic evaluation of language models,” arXiv:2211.09110, 2022

    2022 arXiv

  14. [22]

    Metamorphic testing: A review of challenges and opportunities,

    T. Y . Chen, F.-C. Kuo, H. Liu, P.-L. Poon, D. Towey, T. H. Tse, and Z. Q. Zhou, “Metamorphic testing: A review of challenges and opportunities,” ACM Computing Surveys, vol. 51, no. 1, pp. 1–27, 2018

    2018

  15. [23]

    DeepTest: Automated testing of DNN-driven autonomous cars,

    Y . Tian, K. Pei, S. Jana, and B. Ray, “DeepTest: Automated testing of DNN-driven autonomous cars,” inProc. 40th Int. Conf. Software Engineering, 2018, pp. 303–314

    2018

  16. [24]

    DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems,

    M. Zhang, Y . Zhang, L. Zhang, C. Liu, and S. Khurshid, “DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems,” inProc. 33rd IEEE/ACM Int. Conf. Automated Software Engineering, 2018, pp. 132–142

    2018

  17. [25]

    Metamorphic testing for semantic invariance in large language models,

    J. de Curt `o and I. de Zarz`a, “Metamorphic testing for semantic invariance in large language models,”IEEE Access, vol. 13, pp. 214772–214791, 2025

    2025

  18. [26]

    Semantic invariance in agentic AI,

    I. de Zarz `a, J. de Curt `o, J. Cabot, P. Manzoni, and C. T. Calafate, “Semantic invariance in agentic AI,” inProc. 20th Int. Conf. Agents and Multi-Agent Systems (AMSTA), Springer Nature, 2026

    2026

  19. [27]

    Energy-aware multilingual evaluation of large language models,

    I. de Zarz `a, M. Liz, J. de Curt `o, and C. T. Calafate, “Energy-aware multilingual evaluation of large language models,”Electronics, vol. 15, no. 7, art. 1395, 2026

    2026

  20. [28]

    EdgeShard: Efficient LLM inference via collaborative edge computing,

    M. Zhang, X. Shen, J. Cao, Z. Cui, and S. Jiang, “EdgeShard: Efficient LLM inference via collaborative edge computing,”IEEE Internet of Things Journal, vol. 12, no. 10, pp. 13119–13131, 2024

    2024

  21. [29]

    Power hungry processing: Watts driving the cost of AI deployment?

    A. S. Luccioni, Y . Jernite, and E. Strubell, “Power hungry processing: Watts driving the cost of AI deployment?” inProc. ACM Conf. Fairness, Accountability, and Transparency, 2024

    2024

  22. [30]

    An environment for autonomous driving decision- making,

    E. Leurent, “An environment for autonomous driving decision- making,” GitHub repository, 2018. [Online]. Available: https://github. com/eleurent/highway-env

    2018

  23. [31]

    Learning to fly—a Gym environment with PyBullet physics for reinforcement learning of multi-agent quadcopter control,

    J. Panerati, H. Zheng, S. Zhou, J. Xu, A. Prorok, and A. P. Schoel- lig, “Learning to fly—a Gym environment with PyBullet physics for reinforcement learning of multi-agent quadcopter control,” inProc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), 2021, pp. 7512–7519

    2021

  24. [32]

    Gymnasium: A standard interface for reinforcement learning environments,

    M. Towerset al., “Gymnasium: A standard interface for reinforcement learning environments,”arXiv:2407.17032, 2024

    2024 arXiv

  25. [33]

    Chain-of-thought prompting elicits reasoning in large lan- guage models,

    J. Weiet al., “Chain-of-thought prompting elicits reasoning in large lan- guage models,” inAdvances in Neural Information Processing Systems, vol. 35, 2022, pp. 24824–24837

    2022

  26. [34]

    Language models don’t always say what they think: Unfaithful explanations in chain- of-thought prompting,

    M. Turpin, J. Michael, E. Perez, and S. Bowman, “Language models don’t always say what they think: Unfaithful explanations in chain- of-thought prompting,” inAdvances in Neural Information Processing Systems, vol. 36, 2024

    2024

  27. [35]

    Measuring faithfulness in chain-of-thought reason- ing,

    T. Lanhamet al., “Measuring faithfulness in chain-of-thought reason- ing,”arXiv:2307.13702, 2023

    2023 arXiv

  28. [36]

    Do prompt-based models really understand the meaning of their prompts?

    A. Webson and E. Pavlick, “Do prompt-based models really understand the meaning of their prompts?”arXiv:2109.01247, 2022

    2022 arXiv

  29. [37]

    PromptBench: Towards evaluating the robustness of large language models on adversarial prompts,

    K. Zhuet al., “PromptBench: Towards evaluating the robustness of large language models on adversarial prompts,”arXiv:2306.04528, 2023

    2023 arXiv

  30. [38]

    Measuring and improving consistency in pretrained language models,

    Y . Elazar, N. Kassner, S. Ravfogel, A. Ravichander, E. Hovy, H. Sch¨utze, and Y . Goldberg, “Measuring and improving consistency in pretrained language models,”Trans. Assoc. Computational Linguistics, vol. 9, pp. 1012–1031, 2021

    2021

  31. [39]

    Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity,

    Y . Lu, M. Bartolo, A. Moore, S. Riedel, and P. Stenetorp, “Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity,”arXiv:2104.08786, 2022

    2022 arXiv

  32. [40]

    Probing classifiers: Promises, shortcomings, and ad- vances,

    Y . Belinkov, “Probing classifiers: Promises, shortcomings, and ad- vances,”Computational Linguistics, vol. 48, no. 1, pp. 207–219, 2022

    2022

  33. [41]

    Con- strained model predictive control: Stability and optimality,

    D. Q. Mayne, J. B. Rawlings, C. V . Rao, and P. O. M. Scokaert, “Con- strained model predictive control: Stability and optimality,”Automatica, vol. 36, no. 6, pp. 789–814, 2000

    2000

  34. [42]

    A simple sequentially rejective multiple test procedure,

    S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian Journal of Statistics, vol. 6, no. 2, pp. 65–70, 1979

    1979

  35. [43]

    SciPy 1.0: Fundamental algorithms for scientific computing in Python,

    P. Virtanenet al., “SciPy 1.0: Fundamental algorithms for scientific computing in Python,”Nature Methods, vol. 17, no. 3, pp. 261–272, 2020

    2020

  36. [44]

    Transformers: State-of-the-art natural language pro- cessing,

    T. Wolfet al., “Transformers: State-of-the-art natural language pro- cessing,” inProc. 2020 Conf. Empirical Methods in Natural Language Processing: System Demonstrations, 2020, pp. 38–45

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.