REVIEW 11 cited by
Large Language Models as General Pattern Machines
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art. Surprisingly, pattern completion proficiency can be partially retained even when the sequences are expressed using tokens randomly sampled from the vocabulary. These results suggest that without any additional training, LLMs can serve as general sequence modelers, driven by in-context learning. In this work, we investigate how these zero-shot capabilities may be applied to problems in robotics -- from extrapolating sequences of numbers that represent states over time to complete simple motions, to least-to-most prompting of reward-conditioned trajectories that can discover and represent closed-loop policies (e.g., a stabilizing controller for CartPole). While difficult to deploy today for real systems due to latency, context size limitations, and compute costs, the approach of using LLMs to drive low-level control may provide an exciting glimpse into how the patterns among words could be transferred to actions.
Forward citations
Cited by 11 Pith papers
-
Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention
Gazette is the first generative gaze-to-text model: it decodes a single gaze scanpath into free-form natural-language descriptions of the viewer's goal, using GPT-4-generated 'think-aloud' transcripts as auxiliary tra...
-
Systematic Evaluation of Knowledge Graph Repair with Large Language Models
A systematic VIO-based framework generates SHACL-violating graph test cases and shows that LLM repair systems perform best with concise, violation-focused prompts.
-
Can Large Language Models Generalize Procedures Across Representations?
Post-training on graph or code versions of a planning task does not transfer to natural-language versions, but a symbolic-then-natural-language RL curriculum achieves strong transfer.
-
Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction
On 100 list-function tasks, LLM hypothesis search approaches human-level acquisition (0.487 vs 0.521 mean test accuracy) and clearly beats direct program generation (0.359), with hypothesis generation as the main erro...
-
EXP-Bench: Can AI Conduct AI Research Experiments?
EXP-Bench is a new benchmark of 461 end-to-end AI research experiments, and leading AI agents complete fewer than 1 percent of them successfully.
-
SV-TrustEval-C: Evaluating Structure and Semantic Reasoning in Large Language Models for Source Code Vulnerability Analysis
Large language models perform poorly on a new C-code vulnerability benchmark, indicating they rely on pattern matching rather than genuine reasoning.
-
HuggingGraph: Understanding the Supply Chain of LLM Ecosystem
A directed heterogeneous graph of 402,654 Hugging Face models and datasets is constructed and analyzed to reveal supply-chain dependencies and structural patterns such as a connected core and heavy-tailed reuse.
-
The Stochastic Parrot on LLM's Shoulder: A Summative Assessment of Physical Concept Understanding
A new grid-based benchmark, PhysiCo, shows LLMs can recall and describe physical concepts in text yet lag humans by about 40% when the same concepts are presented as abstract grid transformations.
-
LLM-Based Instance-Driven Heuristic Bias In the Context of a Biased Random Key Genetic Algorithm
An LLM-generated, per-instance bias vector improves a BRKGA on the NP-hard Longest Run Subsequence problem, with statistically significant gains on 15 of 35 instance groups, concentrated on complex instances.
-
On the Surprising Efficacy of LLMs for Penetration-Testing
A critical review arguing that LLMs are surprisingly effective for penetration testing because the task is largely pattern-matching, while noting serious reliability, safety, and cost barriers to autonomous use.
-
Reducing Latency in LLM-Based Natural Language Commands Processing for Robot Navigation
A direct ChatGPT-to-ROS2 voice command pipeline for robot navigation is reported to cut latency by 7.01%, but the paper's own data do not reproduce that number.
Discussion (0). Sign in to comment.