Pith. sign in

REVIEW 2 cited by

I Need Help! Evaluating LLM's Ability to Ask for Users' Support: A Case Study on Text-to-SQL Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.14767 v2 pith:UMYRJUQW submitted 2024-07-20 cs.CL cs.AI

Cheng-Kuang Wu , Zhi Rui Tam , Chao-Chung Wu , Chieh-Yen Lin , Hung-yi Lee , Yun-Nung Chen This is my paper · ORCID
classification cs.CLcs.AI
keywords llmssupportuserabilityexternalhelpneedappier-research
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This study explores the proactive ability of LLMs to seek user support. We propose metrics to evaluate the trade-off between performance improvements and user burden, and investigate whether LLMs can determine when to request help under varying information availability. Our experiments show that without external feedback, many LLMs struggle to recognize their need for user support. The findings highlight the importance of external signals and provide insights for future research on improving support-seeking strategies. Source code: https://github.com/appier-research/i-need-help

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ELABORATION: A Comprehensive Benchmark on Human-LLM Competitive Programming

    cs.AI 2025-05 conditional novelty 7.0 of 10

    ELABORATION provides a four-stage human-feedback taxonomy and an 8,320-problem dataset, with experiments showing human-LLM collaboration improves pass@1 by about 7 percent.

  2. Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities

    cs.CL 2025-05 conditional novelty 4.0 of 10

    A systematic review organizing LLM-based text-to-SQL methods into pre-processing, in-context learning, fine-tuning, and post-processing paradigms, with a catalog of datasets, metrics, challenges, and future directions.

Pith tools