REVIEW 4 cited by
LMPriors: Pre-Trained Language Models as Task-Specific Priors
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Particularly in low-data regimes, an outstanding challenge in machine learning is developing principled techniques for augmenting our models with suitable priors. This is to encourage them to learn in ways that are compatible with our understanding of the world. But in contrast to generic priors such as shrinkage or sparsity, we draw inspiration from the recent successes of large-scale language models (LMs) to construct task-specific priors distilled from the rich knowledge of LMs. Our method, Language Model Priors (LMPriors), incorporates auxiliary natural language metadata about the task -- such as variable names and descriptions -- to encourage downstream model outputs to be consistent with the LM's common-sense reasoning based on the metadata. Empirically, we demonstrate that LMPriors improve model performance in settings where such natural language descriptions are available, and perform well on several tasks that benefit from such prior knowledge, such as feature selection, causal inference, and safe reinforcement learning.
Forward citations
Cited by 4 Pith papers
-
Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Prior
A logit-based method converts an LLM's numeric guesses into a calibrated prior over Pearson correlations and ranks expert-flagged hypotheses better than ranking by magnitude or by a fine-tuned RoBERTa classifier.
-
HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving
The HCRMP planner feeds LLM semantic hints into state representation and critic weighting instead of letting the LLM decide actions, reporting better CARLA driving metrics.
-
Context-Adaptive Inference: A Unified Statistical and Foundation-Model View
Under linear, squared-loss assumptions, explicit context adaptation and in-context learning both reduce to kernel ridge regression on joint input-context features.
-
Causal MAS: A Survey of Large Language Model Architectures for Discovery and Effect Estimation
A structured survey that defines and catalogs multi-agent LLM systems for causal reasoning, discovery, and effect estimation, including their architectures, benchmarks, and applications.
Discussion (0). Sign in to comment.