REVIEW 12 cited by
Binding Language Models in Symbolic Languages
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Though end-to-end neural approaches have recently been dominating NLP tasks in both performance and ease-of-use, they lack interpretability and robustness. We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e.g., SQL, Python) to extend its grammar coverage and thus tackle more diverse questions, (2) adopts an LM as both the program parser and the underlying model called by the API during execution, and (3) requires only a few in-context exemplar annotations. Specifically, we employ GPT-3 Codex as the LM. In the parsing stage, with only a few in-context exemplars, Codex is able to identify the part of the task input that cannot be answerable by the original programming language, correctly generate API calls to prompt Codex to solve the unanswerable part, and identify where to place the API calls while being compatible with the original grammar. In the execution stage, Codex can perform versatile functionalities (e.g., commonsense QA, information extraction) given proper prompts in the API calls. Binder achieves state-of-the-art results on WikiTableQuestions and TabFact datasets, with explicit output programs that benefit human debugging. Note that previous best systems are all finetuned on tens of thousands of task-specific samples, while Binder only uses dozens of annotations as in-context exemplars without any training. Our code is available at https://github.com/HKUNLP/Binder .
Forward citations
Cited by 12 Pith papers
-
Fact-Check Your Information (FYI): A Design Probe to Understand How People Actually Fact-Check Data-Driven Articles
In an exploratory study, fact-checkers using the FYI browser extension adopted AI-first, manual-first, and parallel workflows, using visualizations to audit AI verdicts.
-
SemOpt: LLM-Driven Code Optimization via Rule-Based Analysis
SemOpt generates Semgrep static-analysis rules from LLM-summarized optimization commits and uses them to locate and apply optimization strategies, outperforming retrieval-based baselines on C/C++ code.
-
Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models
Table-r1 combines a layout-transformation self-supervised task and a mix-paradigm GRPO stage so 7B/8B models outperform other small-model table reasoners and approach GPT-4o-level accuracy.
-
GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data?
GeoGrid-Bench evaluates 11 foundation models on 3,200 expert-curated questions about gridded climate data across 16 variables, finding vision-language models strongest and code generation weakest.
-
Efficient Multi-Agent Collaboration with Tool Use for Online Planning in Complex Table Question Answering
MACT, a two-agent framework with tool use, reaches GPT-4-level exact-match scores on two of four table question answering benchmarks using open-weight LLMs without fine-tuning.
-
NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalization
NeSyCoCo improves compositional generalization in vision-language reasoning by combining LLM-generated programs, normalized differentiable predicate scores, and shared word-embedding-conditioned predicate functions.
-
AutoDCWorkflow: LLM-based Data Cleaning Workflow Auto-Generation and Benchmark
AutoDCWorkflow uses an iterative LLM agent loop to generate purpose-driven OpenRefine cleaning workflows, and a new 142-purpose benchmark shows it outperforms direct prompting on the authors' synthetic dirty tables.
-
SVA-ICL: Improving LLM-based Software Vulnerability Assessment via In-Context Learning and Information Fusion
An in-context learning approach that retrieves similar vulnerability examples by fusing code and description similarities improves LLM-based severity assessment over prior baselines.
-
LLM-Symbolic Integration for Robust Temporal Tabular Reasoning
Schema-based SQL generation makes LLM temporal table QA more robust to counterfactual data and large tables than direct prompting, on a new synthetic benchmark.
-
An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems
A survey and position paper that reviews LLM prompting, RAG, and RL techniques and argues they could support open-ended implementation generation, without presenting new results.
-
Building Task Bots with Self-learning for Enhanced Adaptability, Extensibility, and Factuality
A thesis that combines self-learning from dialog logs, schema-guided prompting, and self-aligned factuality to build task bots with minimal human intervention.
-
LLM Inference Enhanced by External Knowledge: A Survey
A survey of methods that enhance LLM inference by integrating external structured knowledge from tables and knowledge graphs.
Discussion (0). Continue with ORCID to comment.