RECORD
Task-similarity Aware Meta-learning through Nonparametric Kernel Regression
Machine Learning · Submitted 12 Jun 2020
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim This supports our hypothesis that task-similarity helps improve the metalearning performance in task-limited and adverse settings.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
This paper investigates the use of nonparametric kernel-regression to obtain a tasksimilarity aware meta-learning algorithm. Our hypothesis is that the use of tasksimilarity helps meta-learning when the available tasks are limited and may contain outlier/ dissimilar tasks. While existing meta-learning approaches implicitly assume the tasks as being similar, it is generally unclear how this task-similarity could be quantified and used in the learning. As a result, most popular metalearning approaches do not actively use the similarity/dissimilarity between the tasks, but rely on availability of huge number of tasks for their working. Our contribution is a novel framework for meta-learning that explicitly uses task-similarity in the form of kernels and an associated meta-learning algorithm. We model the task-specific parameters to belong to a reproducing kernel Hilbert space where the kernel function captures the similarity across tasks. The proposed algorithm iteratively learns a meta-parameter which is used to assign a task-specific descriptor for every task. The task descriptors are then used to quantify the task-similarity through the kernel function. We show how our approach conceptually generalizes the popular meta-learning approaches of model-agnostic meta-learning (MAML) and Meta-stochastic gradient descent (Meta-SGD) approaches. Numerical experiments with regression tasks show that our algorithm outperforms these approaches when the number of tasks is limited, even in the presence of outlier or dissimilar tasks. This supports our hypothesis that task-similarity helps improve the metalearning performance in task-limited and adverse settings.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | This paper investigates the use of nonparametric kernel-regression to obtain a tasksimilarity aware meta-learning algorithm. | contribution 0.93 | 0.77 | 0.05 | asserted 0.82 |
| s1 | Our hypothesis is that the use of tasksimilarity helps meta-learning when the available tasks are limited and may contain outlier/ dissimilar tasks. | background 0.69 | 0.13 | 0.12 | hedged possibility 0.79 |
| s2 | While existing meta-learning approaches implicitly assume the tasks as being similar, it is generally unclear how this task-similarity could be quantified and used in the learning. | gap 0.97 | 0.05 | 0.03 | asserted 0.69 |
| s3 | As a result, most popular metalearning approaches do not actively use the similarity/dissimilarity between the tasks, but rely on availability of huge number of tasks for their working. | background 0.50 | 0.06 | 0.03 | asserted 0.98 |
| s4 | Our contribution is a novel framework for meta-learning that explicitly uses task-similarity in the form of kernels and an associated meta-learning algorithm. | contribution 1.00 | 0.92 | 0.10 | asserted 1.00 |
| s5 | We model the task-specific parameters to belong to a reproducing kernel Hilbert space where the kernel function captures the similarity across tasks. | method 0.97 | 0.83 | 0.04 | asserted 1.00 |
| s6 | The proposed algorithm iteratively learns a meta-parameter which is used to assign a task-specific descriptor for every task. | method 0.99 | 0.87 | 0.04 | asserted 1.00 |
| s7 | The task descriptors are then used to quantify the task-similarity through the kernel function. | method 1.00 | 0.82 | 0.05 | asserted 1.00 |
| s8 | We show how our approach conceptually generalizes the popular meta-learning approaches of model-agnostic meta-learning (MAML) and Meta-stochastic gradient descent (Meta-SGD) approaches. | contribution 0.86 | 0.84 | 0.04 | asserted 1.00 |
| s9 | Numerical experiments with regression tasks show that our algorithm outperforms these approaches when the number of tasks is limited, even in the presence of outlier or dissimilar tasks. | result 1.00 | 0.96 | 0.65 | asserted 1.00 |
| s10 | This supports our hypothesis that task-similarity helps improve the metalearning performance in task-limited and adverse settings. | interpretation 1.00 | 0.79 | 0.64 | asserted 0.69 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
This paper investigates the use of nonparametric kernel-regression to obtain a tasksimilarity aware meta-learning algorithm. Our hypothesis is that the use of tasksimilarity helps meta-learning when the available tasks are limited and may contain outlier/ dissimilar tasks. While existing meta-learning approaches implicitly assume the tasks as being similar, it is generally unclear how this task-similarity could be quantified and used in the learning. As a result, most popular metalearning approaches do not actively use the similarity/dissimilarity between the tasks, but rely on availability of huge number of tasks for their working. Our contribution is a novel framework for meta-learning that explicitly uses task-similarity in the form of kernels and an associated meta-learning algorithm. We model the task-specific parameters to belong to a reproducing kernel Hilbert space where the kernel function captures the similarity across tasks. The proposed algorithm iteratively learns a meta-parameter which is used to assign a task-specific descriptor for every task. The task descriptors are then used to quantify the task-similarity through the kernel function. We show how our approach conceptually generalizes the popular meta-learning approaches of model-agnostic meta-learning (MAML) and Meta-stochastic gradient descent (Meta-SGD) approaches. Numerical experiments with regression tasks show that our algorithm outperforms these approaches when the number of tasks is limited, even in the presence of outlier or dissimilar tasks. This supports our hypothesis that task-similarity helps improve the metalearning performance in task-limited and adverse settings.
Human review
How the literature uses it
3 arXiv papers cite this in 4 sentences
Papers citing it · 3 sentences without a confident label
Cites it as background 1 paper
-
Topics related to kernel tricks and meta-learning have been explored in [ref] .
Non-Gaussian Gaussian Processes for Few-Shot Regression · Related Work · 0.96
What it cites · 1 sentence without a confident label
Cites as background 2 papers
-
While the use of kernels or similarity metrics is not new in meta-learning, they are typically seen in the context of defining relations between the classes or samples within a given task [ref] .
Meta-Learning Mean Functions for Gaussian Processes · Related work · 0.88
-
Information-theoretic ideas have also been used in the study of the topology and the geometry of task spaces [ref] .
Toward Understanding Catastrophic Forgetting in Continual Learning · Related work · 0.96
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.