REVIEW 4 cited by
CITB: A Benchmark for Continual Instruction Tuning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Continual learning (CL) is a paradigm that aims to replicate the human ability to learn and accumulate knowledge continually without forgetting previous knowledge and transferring it to new tasks. Recent instruction tuning (IT) involves fine-tuning models to make them more adaptable to solving NLP tasks in general. However, it is still uncertain how instruction tuning works in the context of CL tasks. This challenging yet practical problem is formulated as Continual Instruction Tuning (CIT). In this work, we establish a CIT benchmark consisting of learning and evaluation protocols. We curate two long dialogue task streams of different types, InstrDialog and InstrDialog++, to study various CL methods systematically. Our experiments show that existing CL methods do not effectively leverage the rich natural language instructions, and fine-tuning an instruction-tuned model sequentially can yield similar or better results. We further explore different aspects that might affect the learning of CIT. We hope this benchmark will facilitate more research in this direction.
Forward citations
Cited by 4 Pith papers
-
TiEBe: Tracking Language Model Recall of Notable Worldwide Events Through Time
A new benchmark, TiEBe, measures LLM recall of notable events across time, regions, and languages, and finds large geographic disparities correlated with GDP, HDI, and schooling.
-
Data-driven atomistic modelling of hybrid halide perovskite passivation
A continual fine-tuning protocol for machine-learned interatomic potentials enables large-scale simulation of amino-silane passivation at hybrid perovskite surfaces, revealing coverage-dependent lattice disruption.
-
SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling
A LoRA update split into several fixed, differently-scaled low-rank experts with orthogonal input directions improves fine-tuning accuracy at the same parameter count.
-
Enhancing Memory Recall in LLMs with Gauss-Tin: A Hybrid Instructional and Gaussian Replay Approach
Gauss-Tin, a replay method using a Gaussian mixture model with prompt-guided exemplar selection, reports positive backward transfer on the Natural Instructions benchmark versus sequential fine-tuning.
Discussion (0). Continue with ORCID to comment.