REVIEW 1 cited by
In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this note, we explore inference-time alignment through in-context learning. We consider a vanilla pretrained language model Llama-2 before any fine-tuning and retrieve an average of 9 demonstration alignment examples when the model is prompted to follow chat-style instructions. Compared to direct prompting, the in-context alignment without changing model weights leads to a 7x increase in win-rate w.r.t. the text-davinci-003 model from OpenAI, making the vanilla language model comparable to strong baselines with alignment fine-tuning.
Forward citations
Cited by 1 Pith paper
-
TSPORec: Token Selection via Preference Optimization for LLM-Based Sequential Recommendation
TSPORec learns to select informative tokens from item text for LLM-based sequential recommendation, improving accuracy slightly and reducing input length.
Discussion (0). Continue with ORCID to comment.