Pith. sign in

REVIEW 41 cited by

Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2311.10702 v2 pith:E6M3GHPR submitted 2023-11-17 cs.CL

Hamish Ivison , Yizhong Wang , Valentina Pyatkin , Nathan Lambert , Matthew Peters , Pradeep Dasigi , Joel Jang , David Wadden
show 3 more authors
Noah A. Smith Iz Beltagy Hannaneh Hajishirzi
This is my paper · ORCID
classification cs.CL
keywords modelscodeopenreleaseadaptingevaluationfinetunedimproved
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Since the release of T\"ULU [Wang et al., 2023b], open resources for instruction tuning have developed quickly, from better base models to new finetuning techniques. We test and incorporate a number of these advances into T\"ULU, resulting in T\"ULU 2, a suite of improved T\"ULU models for advancing the understanding and best practices of adapting pretrained language models to downstream tasks and user preferences. Concretely, we release: (1) T\"ULU-V2-mix, an improved collection of high-quality instruction datasets; (2) T\"ULU 2, LLAMA-2 models finetuned on the V2 mixture; (3) T\"ULU 2+DPO, T\"ULU 2 models trained with direct preference optimization (DPO), including the largest DPO-trained model to date (T\"ULU 2+DPO 70B); (4) CODE T\"ULU 2, CODE LLAMA models finetuned on our V2 mix that outperform CODE LLAMA and its instruction-tuned variant, CODE LLAMA-Instruct. Our evaluation from multiple perspectives shows that the T\"ULU 2 suite achieves state-of-the-art performance among open models and matches or exceeds the performance of GPT-3.5-turbo-0301 on several benchmarks. We release all the checkpoints, data, training and evaluation code to facilitate future open efforts on adapting large language models.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 41 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs

    cs.CL 2025-07 conditional novelty 7.0 of 10

    Cognitive biases in LLMs are largely set during pretraining, while finetuning data and seed randomness only modulate them.

  2. ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Per-step retrieval of solved exemplars injected into the reasoning trace improves test-time scaling accuracy, with up to 13.4 absolute points gained on AIME 2025.

  3. Beyond Post-Hoc Temperature Scaling: Bilevel Optimization for LLM Calibration

    cs.LG 2026-08 conditional novelty 6.0 of 10

    CALM uses bilevel optimization to tune per-vocabulary temperature-like logit adjustments during LLM fine-tuning, and reports improved out-of-domain calibration for aligned language models.

  4. A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)

    cs.LG 2026-02 conditional novelty 6.0 of 10

    Only gradient-based (LESS) representations make subset-to-query distance a reliable predictor of instruction-tuning performance; greedy round-robin helps most at small budgets, and random selection is surprisingly com...

  5. Less is Enough: Synthesizing Diverse Data in LLM Feature Space with Sparse Autoencoders

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Coverage of sparse-autoencoder-identified task features predicts post-training performance and can guide synthesis of small, high-impact datasets (2,000 vs. 300,000 samples).

  6. Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations

    cs.CL 2026-01 reject novelty 6.0 of 10

    A greedy gradient-component selection appears to beat full gradients and random projection for training-data retrieval, but the comparison is confounded because components are selected on the evaluation set itself.

  7. Icon$^{2}$: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation

    cs.CL 2025-09 conditional novelty 6.0 of 10

    ICON2 uses representation-space steering to generate preference data from the model itself, improving alignment benchmarks and cutting cost.

  8. DPO Unchained: Your Training Algorithm is Secretly Disentangled in Human Choice Theory (and its Loss' Convexity is Dispensable)

    cs.LG 2025-07 conditional novelty 6.0 of 10

    DPO's connection to human choice theory generalizes: any monotone loss can pair with any choice model in a new KLST* class, so loss convexity is not normatively required.

  9. On the Effect of Instruction Tuning Loss on Generalization

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Weighted Instruction Tuning, with low-to-moderate prompt weight and moderate-to-high response weight, beats the standard response-only instruction tuning loss in most of the 75 (model, dataset, benchmark) settings tested.

  10. Disentangling the Roles of Representation and Selection in Data Pruning

    cs.CL 2025-07 conditional novelty 6.0 of 10

    In NLP data pruning, the representation used to score examples (especially gradients) influences selected data and downstream performance more than the selection algorithm, and difficulty-oriented algorithms often do ...

  11. Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    The monotonicity of token probabilities during initial decoding predicts chain-of-thought gains, enabling dynamic selection between CoT and direct answers.

  12. Data Swarms: Optimizable Generation of Synthetic Evaluation Data

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Data Swarms uses particle swarm optimization over data-generator LLM weights to produce synthetic evaluation data that scores higher on five quantitative evaluation objectives than eight baselines.

  13. Systematic Evaluation of Machine-Generated Reasoning and PHQ-9 Labeling for Depression Detection Using Large Language Models

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A subtask decomposition of depression detection shows LLMs are biased by explicit depression keywords, and DPO fine-tuning on quality-filtered machine-generated rationales improves joint PHQ-9 labeling on the hardest samples.

  14. IHEval: Evaluating Language Models on Following the Instruction Hierarchy

    cs.CL 2025-02 conditional novelty 6.0 of 10

    IHEval shows that current language models often follow lower-priority instructions over system messages, and simple prompting does not fix the problem.

  15. SHARP: Accelerating Language Model Inference by SHaring Adjacent layers with Recovery Parameters

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Sharing one MLP layer's weights across several layers plus low-rank adapters recovers most of a pretrained LLM's quality with a fraction of the storage and faster phone inference.

  16. Automatic Evaluation of Healthcare LLMs Beyond Question-Answering

    cs.CL 2025-02 reject novelty 6.0 of 10

    In healthcare LLM evaluation, multiple-choice accuracy and open-ended task scores correlate only weakly, and the paper's proposed Relaxed Perplexity metric aims to improve open-ended factuality scoring but rests on an...

  17. Data-adaptive Safety Rules for Training Reward Models

    cs.CL 2025-01 conditional novelty 6.0 of 10

    Selecting the five safety rules with largest response discrepancy maximizes mutual information under stated assumptions, and a reward model trained with this adaptive labeling achieves top RewardBench safety.

  18. NExtLong: Toward Effective Long-Context Training without Long Documents

    cs.CL 2025-01 conditional novelty 6.0 of 10

    Interleaving hard negative distractors between chunks of short documents improves long-context language model performance on HELMET and RULER.

  19. T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation

    cs.CL 2025-01 conditional novelty 6.0 of 10

    T2ISafety is a large annotated benchmark plus a fine-tuned MLLM evaluator (ImageGuard) for measuring toxicity, privacy, and fairness in text-to-image models.

  20. Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities

    cs.CL 2025-01 conditional novelty 6.0 of 10

    BIDS, a balanced influence-based data selection algorithm using per-task normalization and iterative greedy selection, improves balanced multi-capability instruction tuning and can outperform full-dataset training on ...

  21. Aligning Instruction Tuning with Pre-training

    cs.AI 2025-01 conditional novelty 6.0 of 10

    AITP selects pre-training texts that are underrepresented relative to instruction-tuning data, rewrites them into instruction-response pairs, and improves average SFT performance on three open LLMs.

  22. CDS: Knowledge Component-Driven Data Synthesis Guided by Cognitive Diagnosis Theory

    cs.AI 2025-01 conditional novelty 6.0 of 10

    A knowledge-component diagnostic pipeline, inspired by cognitive diagnosis theory, generates weakness-targeted synthetic data that improves small LLMs on math, code, and exam benchmarks by up to 13.1 percentage points.

  23. Bootstrap Your Own Context Length

    cs.CL 2024-12 conditional novelty 6.0 of 10

    The authors synthesize long-context training data from short chunks and fine-tune Llama-3 models to reach 1M-token contexts with top open-source RULER scores.

  24. Understanding the Logic of Direct Preference Alignment through Logic

    cs.CL 2024-12 conditional novelty 6.0 of 10

    Direct preference alignment losses can be expressed as logical programs over model predictions, yielding an organized landscape of billions of definable losses and a route to new variants.

  25. Evaluating the Consistency of LLM Evaluators

    cs.CL 2024-11 conditional novelty 6.0 of 10

    LLM evaluator consistency is not tied to model quality: GPT-3.5 scores high on MT-Bench but is less self-consistent and inter-scale consistent than Mistral in several settings.

  26. Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models

    cs.AI 2025-08 unverdicted novelty 5.0 of 10

    The authors introduce and test an optimization method for fine-tuning data mixtures that matches costly grid-search results within 0.66% validation loss.

  27. From Black Box to Transparency: Enhancing Automated Interpreting Assessment with Explainable AI in College Classrooms

    cs.CL 2025-08 unverdicted novelty 5.0 of 10

    An explainable model using BLEURT, CometKiwi, pause features, and Chinese phraseological diversity predicts human-rated quality dimensions in English-Chinese consecutive interpreting, with SHAP identifying the stronge...

  28. Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Nile-Chat models for dual-script Egyptian Arabic beat strong baselines on newly translated benchmarks, but the evaluation may be inflated by training/eval data overlap and Claude-generated script data.

  29. AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs

    cs.LG 2025-05 conditional novelty 5.0 of 10

    AutoMixAlign adaptively reweights or resamples task data during DPO training to match specialist-model losses, improving average performance on helpfulness, coding, and safety benchmarks compared to standard DPO and m...

  30. Efficient Data Selection at Scale via Influence Distillation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Influence Distillation selects LLM fine-tuning data by approximating each sample's gradient influence on a target task via landmarks and JVP embeddings, matching or beating RDS+ accuracy at roughly one third the selec...

  31. Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs

    cs.LG 2024-12 conditional novelty 5.0 of 10

    For 3B-7B LLMs, larger batch sizes with lower learning rates improve instruction-tuning benchmarks, early gradient and loss signals predict final quality, and stacked training matches phased training with fewer samples.

  32. CoinMath: Harnessing the Power of Coding Instruction for Math LLMs

    cs.CL 2024-12 conditional novelty 5.0 of 10

    CoinMath improves math LLM accuracy by training on GPT-4o-generated code rationales with concise comments, descriptive naming, and hardcoded solutions.

  33. Benchmarking Linguistic Diversity of Large Language Models

    cs.CL 2024-12 conditional novelty 5.0 of 10

    State-of-the-art LLMs generate less linguistically diverse text than humans on creative tasks, and several training and deployment choices systematically shift lexical, syntactic, and semantic diversity.

  34. Large Language Models Can Self-Improve in Long-context Reasoning

    cs.CL 2024-11 conditional novelty 5.0 of 10

    LLMs can improve their own long-context reasoning by fine-tuning on outputs selected by Minimum Bayes Risk consensus, without external annotations.

  35. Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation

    cs.CV 2025-08 conditional novelty 4.0 of 10

    A hierarchical direct preference optimization with four alignment levels plus automated data selection improves physical plausibility of text-to-video models.

  36. DiffLoRA: Differential Low-Rank Adapters for Large Language Models

    cs.CL 2025-07 conditional novelty 4.0 of 10

    DiffLoRA, a low-rank adapter variant of differential attention, generally underperforms LoRA but shows gains on HumanEval and multi-value needle retrieval.

  37. Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A self-generated and committee-filtered synthetic rehearsal dataset lets a fine-tuned LLM nearly preserve its general benchmark scores while adding domain data.

  38. Why Do More Experts Fail? A Theoretical Analysis of Model Merging

    cs.LG 2025-05 reject novelty 4.0 of 10

    The paper claims to prove an upper bound and diminishing returns in model merging, but the proofs are not sound and the heavy-tailed claim is contradicted by its own equations.

  39. Privacy-Preserving Personalized Federated Prompt Learning for Multimodal Large Language Models

    cs.LG 2025-01 conditional novelty 4.0 of 10

    DP-FPL applies local DP to low-rank prompt factors and global DP to the shared prompt, reporting stronger accuracy under privacy than baselines.

  40. SmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better Reasoning in SLMs

    cs.CL 2024-12 conditional novelty 4.0 of 10

    Higher learning-rate-to-batch-size ratios improve reasoning-task performance in small language models, and the resulting SmolTulu-1.7B reports top sub-2B scores on IFEval and GSM8K.

  41. An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems

    cs.CL 2024-12 unverdicted novelty 3.0 of 10

    A survey and position paper that reviews LLM prompting, RAG, and RL techniques and argues they could support open-ended implementation generation, without presenting new results.

Pith tools