Pith. sign in

REVIEW 4 major objections 5 minor 154 references

Improving Data and Parameter Efficiency of Neural Language Models Using Representation Analysis

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This thesis argues that representation smoothness is the lever: regularizing the Jacobian and Hessian norms of transformer layers improves generalization, calibration, and data and parameter efficiency of neural language models.

desk verdict JACHESS is a genuine and promising regularization method backed by unusually careful GLUE experiments, but the Hutchinson estimator details and dev-set tuning leave the smoothness mechanism underdetermined. read the letter →

arxiv 2507.12004 v1 pith:S2IAQFJN submitted 2025-07-16 cs.CL

Josip Jukić This is my paper
classification cs.CL
keywords neurallanguagemodelsrepresentationanalysisdataefficiencyparameteractivelearningweaksupervisionin-contextJacobian-Hessianregularization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The thesis argues that the internal representations of neural language models carry information about generalization, robustness, and calibration, and that deliberately making those representations smoother improves all three. Its headline method, JACHESS, adds a penalty on the Jacobian and Hessian norms of transformer layers during fine-tuning; on the GLUE benchmark it reports average gains of 2 to 4.5 percentage points over standard fine-tuning and over other regularizers, together with better robustness to token corruption and embedding perturbations and better-calibrated confidence scores. The same smoothness signal is then reused as a stopping rule that needs no labeled validation set, as a guide for combining active learning with parameter-efficient fine-tuning, and as the basis of a weak-supervision framework for in-context learning. A reader should care because the paper points to one measurable property that can be optimized cheaply, potentially reducing both annotation effort and compute in low-resource settings.

What carries the argument

The load-bearing object is the layer-wise Jacobian-Hessian regularization term of JACHESS, Eq. (4.1): $\sum_{k=1}^{K}\big(\lambda^{(k)}_1 \|J^{(k)}\|_F + \lambda^{(k)}_2 \sum_{d\in D^{(k)}} \|H^{(k)}_d\|_F\big)$. Here the Jacobian $J^{(k)}$ is the derivative of a transformer layer's output with respect to its input token embeddings and $H^{(k)}_d$ is the Hessian of one output dimension; bounding their Frobenius norms acts as a surrogate for Lipschitz smoothness of the representation map. Hutchinson's estimator replaces the intractable full matrices with Monte Carlo estimates from random normal vectors, and a dual-mode procedure alternates the task loss on labeled data with smoothness-only minimization on a separate unlabeled pool, which the experiments report as the strongest configuration.

What would settle it

Take a small transformer where the full Jacobian and Hessian matrices are computable, fine-tune it with JACHESS, and check whether the true Frobenius norms actually decrease and whether average GLUE gains track that decrease; if the gains persist while the true norms stay flat or rise, the smoothness mechanism is not the operative cause. Similarly, BEAST's stopping decisions can be compared against oracle stopping on a held-out set: if the Besov-index stopping points consistently diverge from the validation-optimal points, the smoothness proxy is not carrying the argument.

Watch

Extended reading notes

Core claim

The paper's central claim is that enforcing smoothness in the hidden representations of transformer language models improves generalization and calibration without sacrificing in-distribution accuracy. JACHESS implements this by adding, for each layer, a penalty on the Frobenius norms of the layer's input-output Jacobian and Hessian matrices, estimated with Hutchinson's random projections and with a random subset of output dimensions for the Hessian; the regularization factors are set in proportion to the base model's own pre-fine-tuning layer smoothness. Trained in a dual mode that alternates the supervised loss with regularization on a separate unlabeled pool, JACHESS is reported to beat standard fine-tuning, L2, TAPT, SAM, Jacobian regularization, and Cross-Hölder regularization on average GLUE scores by 2 to 4.5 percentage points, with the largest gains on OPT-6.7B and Llama-2-7B, and to lower Brier score and expected calibration error across binary GLUE tasks. The thesis then carries the same smoothness concept into efficiency: Besov-measured smoothness drives BEAST early stopping without labeled validation data, and the integration of active learning with parameter-efficient fine-tuning shows that PEFT methods outperform full fine-tuning in low-resource active-learning settings while preferring moderately hard examples and staying closer to the pretrained model.

Load-bearing premise

The load-bearing premise is that Hutchinson's estimator, with a small number of random projections and sampled output dimensions, estimates the Jacobian and Hessian Frobenius norms accurately enough that minimizing those estimates genuinely enforces representation smoothness, so the reported gains in generalization and calibration are due to smoothness and not to artifacts of the stochastic approximation.

Editorial extensions

If this is right

  • If JACHESS works as reported, fine-tuning a transformer language model with smoothness penalties becomes a cheap add-on: with 1000 unlabeled instances the reported training-cost increase is about 1.22x, while the average GLUE score rises.
  • Validation-free Besov early stopping would remove the labeled held-out set from fine-tuning and active-learning pipelines, which matters for privacy-sensitive and low-resource tasks.
  • Active learning with parameter-efficient fine-tuning should reach a given accuracy with fewer annotated examples and fewer updated parameters than full fine-tuning, because PEFT keeps representations closer to the pretrained model.
  • WILDA-style weak supervision converts in-context demonstrations into knowledge stored in adapter modules, so demonstrations do not have to be repeated inside every prompt.
  • Better calibration from smoothness would make confidence scores of fine-tuned models usable for selective prediction and deferral in high-stakes NLP applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the thesis leaves open is applying the same Jacobian-Hessian penalty during pre-training rather than fine-tuning; the reported experiments only cover fine-tuning on GLUE-sized data, so whether smoothness regularization helps at pre-training scale is unknown.
  • The comparison between JACHESS train and JACHESS unlab hints that part of the gain may come from the extra unlabeled-data passes rather than from the smoothness penalty itself; a clean ablation would keep the dual-mode training while replacing the norm penalty with a dummy loss.
  • The paper's smoothness-guided active-learning stopping could be transferred to batch active learning for larger decoder-only models, where the Besov index might also serve as a cheap monitor of distribution shift during iterative querying.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The thesis proposes representation-analysis-based techniques to improve data and parameter efficiency of neural language models. The main contribution is JACHESS, a regularization method that penalizes Jacobian and Hessian Frobenius norms of intermediate transformer layers, estimated via Hutchinson's estimator, with the aim of enforcing representation smoothness and thereby improving generalization, robustness, and calibration. Secondary contributions include BEAST, an early-stopping method based on Besov smoothness that avoids labeled validation sets; a study combining active learning with parameter-efficient fine-tuning; and WILDA, a weak-supervision framework for in-context learning. The empirical evaluation centers on GLUE benchmark results for BERT, OPT, and Llama models, reporting average-score gains of 2% to 4.5% for JACHESS over standard fine-tuning.

Significance. If the results hold, the thesis provides a conceptually clean mechanism—enforcing representation smoothness—for improving both generalization and calibration of transformer language models, with a dual-mode training recipe that leverages unlabeled data. The manuscript has several strengths: it reports five-seed experiments, uses Mann-Whitney U tests with Holm-Bonferroni correction for the main comparisons, spans multiple model families and scales, and combines three orthogonal efficiency levers (regularization, active learning, and weak supervision). However, the central smoothness mechanism rests on the accuracy of the Hutchinson estimator, which is not adequately characterized, and the design choices for JACHESS are tuned on the same GLUE development sets used for the headline results. These issues must be addressed before the causal claim that smoothness drives the observed gains can be accepted.

major comments (4)
  1. [Section 3.4 / Eq. (3.7)-(3.8) and Section 4.1 / Eq. (4.1)] The manuscript never reports the number of Hutchinson projections p used for the Jacobian and Hessian estimates, and Eq. (4.1) is an unnormalized sum over a random subset D(k) of output dimensions. Because the expected value of the Hessian term scales linearly with |D(k)|, the results in Table 4.7 (which vary the sampled dimensions from 0 to 50) are confounded with the effective regularization strength λ_2; the observed optimum at 10 dimensions therefore does not demonstrate estimation fidelity, only that a particular effective penalty is helpful. To support the claim that minimizing the estimated norms enforces smoothness, the authors should report p, normalize the dimension sum (e.g., divide by |D(k)|), and provide a sensitivity analysis over p or variance diagnostics for the estimator.
  2. [Section 4.5 / Tables 4.6, 4.7 and Figure 4.3] The regularization-factor strategy, the number of sampled Hessian dimensions, and the size of the unlabeled set are all selected on the GLUE development sets, which are also the evaluation sets for the headline results in Tables 4.2 and 4.3. This constitutes tuning on the test data for the central empirical claim, and the Holm-Bonferroni-corrected p-values in Tables 4.2 and 4.3 do not account for this selection. The authors should either evaluate on held-out test sets (e.g., via the official GLUE benchmark or an internal train/validation/test split) or clearly separate model selection from evaluation and report the selection procedure.
  3. [Section 4.3.3 / Tables 4.2 and 4.3] The text claims that JACHESS 'significantly' outperforms standard fine-tuning and other regularization methods, but the significance indicators (†) only mark comparisons between JACHESS variants and the BASE model. No pairwise significance tests are reported against L2, TAPT, SAM, Jacobian regularization, or Cross-Hölder regularization. Given that the average-score improvements over Cross-Hölder unlab are often small (e.g., BERT .783 vs. .762, OPT-1.3B .846 vs. .837), the claim of superiority over other regularization methods is unsupported by the reported statistics. Pairwise tests against all baselines, or a more cautious wording, are needed.
  4. [Section 6.2 / Section 3.5] BEAST, the proposed early-stopping method, relies entirely on the Besov smoothness index estimated through the wavelet random-forest approximation of Elisha and Dekel [68], but the manuscript does not specify the number of wavelets M, the least-squares fitting procedure, or any validation of the estimated index against actual generalization performance. Since the stopping decision is driven by this proxy, a systematic mis-estimate would invalidate the method. The authors should provide the algorithmic details and an evaluation of the estimator's reliability (e.g., correlation with validation accuracy on held-out data).
minor comments (5)
  1. [Section 4.1.1] The notation for regularization factors is confusing: λ_i is introduced as a vector across layers, but the text later states 'λ 1 = λ 2 = λ' without clarifying whether this is a scalar shared across layers or a per-layer vector. Please disambiguate.
  2. [Table 4.1] The column header says 'Token corruption [%]' but the first column lists values 5, 10, 15, 20, while the main text states corruption is 'varied between 10%, 15%, and 20%.' Either include 5% in the text or remove it from the table.
  3. [Section 3.4] In Eqs (3.7) and (3.8), the dimensionality of the random vector v is not stated; specify that v∈R^d where d is the output dimension of the relevant layer, and clarify whether the same vector is reused across the sampled dimensions.
  4. [Section 4.5 / Figure 4.3] The sentence 'Without any unlabeled data, the setup corresponds to the BASE model' appears to be imprecise, because JACHESS train also exists and uses labeled data; clarify which variant is being compared.
  5. [Section 4.3.3] The captions of Tables 4.2 and 4.3 mention that the second-highest scores are underlined, but the body text never interprets the underlined values; either remove the underlining or add a sentence discussing the second-best methods.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: JACHESS, the design-choice analyses, and the efficiency claims are derived from explicit definitions and evaluated on external GLUE/IMDb benchmarks, not from fitted inputs or self-citations.

full rationale

The derivation chain is self-contained. JACHESS is defined by Eq. (4.1) as a weighted sum of estimated Jacobian and Hessian Frobenius norms, with the estimator given in Eqs. (3.7)-(3.8); the regularization target is therefore stated explicitly and does not presuppose the reported GLUE improvements. The claim of 2-4.5% average GLUE gains is an empirical comparison to BASE, L2, TAPT, SAM, Jacobian, and Cross-Hölder baselines on held-out development sets, so it is not a fitted quantity renamed as a prediction. The only self-reference is citation [71] for the JACHESS method, but Section 4.1 re-derives the method in full, so the citation is provenance rather than load-bearing. Hyperparameters such as lambda, number of sampled Hessian dimensions, and unlabeled-data pool size are selected on the development sets, which is standard model selection and not definitional circularity. The Hutchinson-estimator noise concern raised by the skeptical reader is a correctness and approximation risk, not a case where the conclusion is equivalent to the input by construction. No step in the available text reduces a predicted quantity to the quantity it was fitted from.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on computational proxies for smoothness (Hutchinson estimates and Besov indices) and on the causal link between smoothness and generalization. No new physical entities are introduced; the free parameters are regularization strengths, sampled dimensions, unlabeled set sizes, and Besov estimation settings, most of which are tuned on the evaluation sets.

free parameters (5)
  • JACHESS regularization factors lambda_1 and lambda_2 per layer = softmax-normalized proportional to pre-fine-tuning smoothness, exact values not reported
    Chosen in Section 4.5 on GLUE development sets as the best of four strategies; the final GLUE numbers are reported on the same development sets.
  • Number of sampled Hessian output dimensions per layer D(k) = 10
    Selected in Table 4.7 as the best value on GLUE development sets; this selection affects the main JACHESS results.
  • Number of Hutchinson random projections p = not reported
    Used in Eqs. 3.7 and 3.8 to estimate Jacobian and Hessian norms; no value or variance analysis is given in the visible text.
  • Number of unlabeled instances for JACHESS unlab = 1000
    Set as the default based on Figure 4.3, which shows diminishing returns beyond 1000 on GLUE development sets.
  • Besov smoothness estimation parameters (number of wavelets M and least-squares fit) = not reported
    The Besov index estimate in Section 3.5 depends on M and the regression; the visible text gives the log-linear relation but not the exact settings used for BEAST.
assumptions (5)
  • standard math Hutchinson's estimator provides an unbiased estimate of the trace via E[v^T B v] = Tr(B) for v ~ N(0,I).
    Invoked in Section 3.4, Eq. 3.5, as the basis for estimating Jacobian and Hessian Frobenius norms without materializing the matrices.
  • domain assumption Minimizing the Frobenius norm of Jacobian and Hessian matrices reduces the Lipschitz constant and enforces representation smoothness.
    Used to justify the JACHESS regularization objective in Section 4.1; the paper does not prove this holds for transformer intermediate layers.
  • ad hoc to paper The wavelet-based random forest approximation from Elisha and Dekel [68] yields a valid Besov smoothness index for PLM hidden states.
    Adopted in Section 3.5 and used for BEAST; the thesis extends this approximation from DNNs to transformer language models without independent verification.
  • domain assumption Smoothness of representations is causally linked to generalization, calibration, and data efficiency in PLMs.
    The central premise of the thesis, stated in Chapters 3 and 4; the thesis provides correlations and benchmark results but no causal proof.
  • domain assumption Active learning query strategies and pseudo-label correction mechanisms used in WILDA improve performance under weak supervision.
    Relied upon for the third part of the thesis; the details in Chapter 10 are only partially visible in the provided text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Data and Parameter Efficiency of Neural Language Models Using Representation Analysis." pith.science (2026). https://pith.science/paper/S2IAQFJN

@misc{pith2026250712004,
  author       = {Pith},
  title        = {Pith review of: Improving Data and Parameter Efficiency of Neural Language Models Using Representation Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S2IAQFJN}},
  note         = {Machine review of arXiv:2507.12004}
}
read the original abstract

This thesis addresses challenges related to data and parameter efficiency in neural language models, with a focus on representation analysis and the introduction of new optimization techniques. The first part examines the properties and dynamics of language representations within neural models, emphasizing their significance in enhancing robustness and generalization. It proposes innovative approaches based on representation smoothness, including regularization strategies that utilize Jacobian and Hessian matrices to stabilize training and mitigate sensitivity to input perturbations. The second part focuses on methods to significantly enhance data and parameter efficiency by integrating active learning strategies with parameter-efficient fine-tuning, guided by insights from representation smoothness analysis. It presents smoothness-informed early-stopping techniques designed to eliminate the need for labeled validation sets and proposes innovative combinations of active learning and parameter-efficient fine-tuning to reduce labeling efforts and computational resources. Extensive experimental evaluations across various NLP tasks demonstrate that these combined approaches substantially outperform traditional methods in terms of performance, stability, and efficiency. The third part explores weak supervision techniques enhanced by in-context learning to effectively utilize unlabeled data, further reducing dependence on extensive labeling. It shows that using in-context learning as a mechanism for weak supervision enables models to better generalize from limited labeled data by leveraging unlabeled examples more effectively during training. Comprehensive empirical evaluations confirm significant gains in model accuracy, adaptability, and robustness, especially in low-resource settings and dynamic data environments.

Figures

Figures reproduced from arXiv: 2507.12004 by the authors.

Figure 2.1
Figure 2.1. Illustration of one-hot encoding for a vocabulary of five words. Each word is [PITH_FULL_IMAGE:figures/full_fig_p028_2_1.png] view at source ↗
Figure 2.2
Figure 2.2. Illustration of the Word2Vec training objectives. [PITH_FULL_IMAGE:figures/full_fig_p034_2_2.png] view at source ↗
Figure 2.3
Figure 2.3. The neural network architecture for CBOW. Context words (one-hot encoded) are [PITH_FULL_IMAGE:figures/full_fig_p035_2_3.png] view at source ↗
Figures from the paper (43 more)
Figure 2.4
Figure 2.4. Figure 2.4: Illustration of the recurrent structure of an RNN. At each time step [PITH_FULL_IMAGE:figures/full_fig_p040_2_4.png]
Figure 2
Figure 2. Figure 2: illustrates the [PITH_FULL_IMAGE:figures/full_fig_p040_2.png]
Figure 2.5
Figure 2.5. Figure 2.5: Unrolled RNN throughout the sequence. This parameter-sharing mechanism reduces the overall complexity of the model, making it more efficient to train while improving generalization. Additionally, these networks enable contextual learning by leveraging hidden states t…
Figure 2
Figure 2. Figure 2: illustrates the internal structure of an LSTM cell. The gating mechanism allows [PITH_FULL_IMAGE:figures/full_fig_p042_2.png]
Figure 2.6
Figure 2.6. Figure 2.6: Structure of an LSTM cell The input, forget, and output gates enable the model to regulate which information should be stored, updated, or discarded at each time step. This selective filtering allows LSTMs to focus on relevant parts of the sequence while ignoring les…
Figure 2.7
Figure 2.7. Figure 2.7: Schematics of the scaled dot-product attention. This diagram illustrates the steps of [PITH_FULL_IMAGE:figures/full_fig_p045_2_7.png]
Figure 2.8
Figure 2.8. Figure 2.8: Transformer architecture • Autoregressive decoding: During inference, the decoder generates tokens sequentially, predicting one token at a time based on previously generated tokens and encoder outputs (if applicable). This step-by-step process enables the generation …
Figure 2.9
Figure 2.9. Figure 2.9: Multi-head attention on the surrounding context. This enables the model to learn deep contextual dependen￾cies; • Causal Language Modeling (CLM): An autoregressive framework in which the model learns to predict the next token in a sequence, given only the preceding t…
Figure 2.10
Figure 2.10. Figure 2.10: LLM landscape. The development of neural language models began with static [PITH_FULL_IMAGE:figures/full_fig_p055_2_10.png]
Figure 3.1
Figure 3.1. Figure 3.1: The effects of representation smoothness enforced through regularization. While [PITH_FULL_IMAGE:figures/full_fig_p060_3_1.png]
Figure 3.2
Figure 3.2. Figure 3.2: Calibration plot illustrating the relationship between predicted probabilities and [PITH_FULL_IMAGE:figures/full_fig_p062_3_2.png]
Figure 4
Figure 4. Figure 4: illustrates the predictive accuracy of different models under varying perturbation [PITH_FULL_IMAGE:figures/full_fig_p074_4.png]
Figure 4.1
Figure 4.1. Figure 4.1: Predictive accuracy under varying perturbation magnitudes [PITH_FULL_IMAGE:figures/full_fig_p075_4_1.png]
Figure 4
Figure 4. Figure 4: provides calibration plots for binary classification tasks from the GLUE benchmark, [PITH_FULL_IMAGE:figures/full_fig_p081_4.png]
Figure 4.2
Figure 4.2. Figure 4.2: Calibration plots for binary classification datasets (GLUE) with Llama 2, averaged [PITH_FULL_IMAGE:figures/full_fig_p082_4_2.png]
Figure 4.3
Figure 4.3. Figure 4.3: Average GLUE scores for Llama 2 and OPT-6.7B with increasing numbers of unla [PITH_FULL_IMAGE:figures/full_fig_p084_4_3.png]
Figure 5.1
Figure 5.1. Figure 5.1: Illustration of a hypothesis in a version space within AL. The purple circles repre [PITH_FULL_IMAGE:figures/full_fig_p090_5_1.png]
Figure 5.2
Figure 5.2. Figure 5.2: Active learning loop in pool-based sampling [PITH_FULL_IMAGE:figures/full_fig_p093_5_2.png]
Figure 6
Figure 6. Figure 6: visualizes the layer-wise Besov smoothness distributions across training regimes, [PITH_FULL_IMAGE:figures/full_fig_p105_6.png]
Figure 6.1
Figure 6.1. Figure 6.1: Normalized Besov smoothness of PLM layers across training regimes. Black error [PITH_FULL_IMAGE:figures/full_fig_p106_6_1.png]
Figure 6
Figure 6. Figure 6: presents AL performance curves for BERT under [PITH_FULL_IMAGE:figures/full_fig_p106_6.png]
Figure 6.2
Figure 6.2. Figure 6.2: Active learning performance curves for BERT under [PITH_FULL_IMAGE:figures/full_fig_p107_6_2.png]
Figure 6.3
Figure 6.3. Figure 6.3: AL performance curves for different training regimes across datasets for BERT. [PITH_FULL_IMAGE:figures/full_fig_p110_6_3.png]
Figure 6.4
Figure 6.4. Figure 6.4: AL performance curves for different training regimes across datasets for ELECTRA. [PITH_FULL_IMAGE:figures/full_fig_p111_6_4.png]
Figure 6.5
Figure 6.5. Figure 6.5: Relationship between active and random sample smoothness (Figure 6.5a) and cor [PITH_FULL_IMAGE:figures/full_fig_p113_6_5.png]
Figure 6.6
Figure 6.6. Figure 6.6: Besov smoothness of actively acquired samples with [PITH_FULL_IMAGE:figures/full_fig_p114_6_6.png]
Figure 7.1
Figure 7.1. Figure 7.1: Bottleneck adapter: A lightweight module inserted between pre-trained layers in a [PITH_FULL_IMAGE:figures/full_fig_p120_7_1.png]
Figure 7.2
Figure 7.2. Figure 7.2: Parallel adapter: A task-specific module operating alongside the pre-trained layer [PITH_FULL_IMAGE:figures/full_fig_p121_7_2.png]
Figure 7.3
Figure 7.3. Figure 7.3: LoRA: Task-specific adaptations represented as low-rank updates to the pre-trained [PITH_FULL_IMAGE:figures/full_fig_p124_7_3.png]
Figure 7.4
Figure 7.4. Figure 7.4: Hard Routing In hard routing, each module is either activated or not based on a binary decision for a given input. The gating function bi(x) ∈ {0,1} indicates whether module i is active: bi(x) =    1 if module i is activated, 0 otherwise. The model’s output is the…
Figure 7.4
Figure 7.4. Figure 7.4: Illustration of routing mechanisms in parameter-efficient learning. Left: Fixed [PITH_FULL_IMAGE:figures/full_fig_p129_7_4.png]
Figure 8.1
Figure 8.1. Figure 8.1: Learning curves under passive learning for PEFT methods and FFT. Results are [PITH_FULL_IMAGE:figures/full_fig_p134_8_1.png]
Figure 8.2
Figure 8.2. Figure 8.2: Comparison of best-performing adapters and FFT from Table 8.2 with and without [PITH_FULL_IMAGE:figures/full_fig_p137_8_2.png]
Figure 8.3
Figure 8.3. Figure 8.3: AL learning curves compared with random sampling on the [PITH_FULL_IMAGE:figures/full_fig_p138_8_3.png]
Figure 8
Figure 8. Figure 8: illustrates the distribution of forgetting events for [PITH_FULL_IMAGE:figures/full_fig_p139_8.png]
Figure 8.4
Figure 8.4. Figure 8.4: Forgetting dynamics for random sampling (passive learning) and AL with [PITH_FULL_IMAGE:figures/full_fig_p140_8_4.png]
Figure 8
Figure 8. Figure 8: visualizes these differences across layers for the [PITH_FULL_IMAGE:figures/full_fig_p140_8.png]
Figure 8.5
Figure 8.5. Figure 8.5: Layerwise difference in representation similarity for the UniPELT adapter and the [PITH_FULL_IMAGE:figures/full_fig_p141_8_5.png]
Figure 8.6
Figure 8.6. Figure 8.6: Layerwise difference in representation similarity for the UniPELT adapter [PITH_FULL_IMAGE:figures/full_fig_p142_8_6.png]
Figure 10.1
Figure 10.1. Figure 10.1: The figure shows how incorrect pseudo-labels are refined using high-confidence [PITH_FULL_IMAGE:figures/full_fig_p157_10_1.png]
Figure 10.2
Figure 10.2. Figure 10.2: Illustration of WILDA. The teacher processes a concatenation (denoted by ○; ) of demonstrations Xd, consisting of n demonstrations [x1,x2,...,xn], and the query xq. The student, using only the query, fine-tunes its adapter weights to produce outputs ys aligned with …
Figure 10.3
Figure 10.3. Figure 10.3: Empirical analysis of WILDA-S on the aggregated GLUE datasets for Llama 3: (a) Histogram of approximated Lipschitz constants across datasets, computed as the Frobenius norm of the input-output Jacobian matrix; (b) Rate of pseudo-label correction over training epochs…
Figure 10
Figure 10. Figure 10: c illustrates the rate of prediction flips within these bins, where a flip refers to [PITH_FULL_IMAGE:figures/full_fig_p174_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

154 extracted references · 54 canonical work pages

  1. [68]

    W., (ur.), Vol

    Suzuki, T., Nitanda, A., ``Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic B esov space'', in Advances in Neural Information Processing Systems, Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J. W., (ur.), Vol. 34. Curran Associates, Inc., 2021, str. 3609--3621, https://proceedings.neurips.cc/paper/...

    2021

  2. [1]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write newline " =0pt " write newline " " ALTinterwordstretchfactor * " " * write newli...

  3. [2]

    S., ``Distributional structure'', Word, Vol

    Harris, Z. S., ``Distributional structure'', Word, Vol. 10, No. 2-3, 1954, str. 146--162

    1954

  4. [3]

    R., ``A synopsis of linguistic theory, 1930-1955'', Studies in linguistic analysis, 1957

    Firth, J. R., ``A synopsis of linguistic theory, 1930-1955'', Studies in linguistic analysis, 1957

    1930

  5. [4]

    D., Pantel, P., ``From frequency to meaning: Vector space models of semantics'', Journal of artificial intelligence research, Vol

    Turney, P. D., Pantel, P., ``From frequency to meaning: Vector space models of semantics'', Journal of artificial intelligence research, Vol. 37, 2010, str. 141--188

    2010

  6. [5]

    160--167

    Collobert, R., Weston, J., ``A unified architecture for natural language processing: Deep neural networks with multitask learning'', in Proceedings of the 25th international conference on Machine learning, 2008, str. 160--167

    2008

  7. [6]

    Frege, G., ``The thought: A logical inquiry'', Mind, Vol. 65, No. 259, 1956, str. 289--311

    1956

  8. [7]

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A.,...

    2020

Show all 154 references
  1. [8]

    B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., Amodei, D., ``Scaling laws for neural language models'', arXiv preprint arXiv:2001.08361, 2020

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., Amodei, D., ``Scaling laws for neural language models'', arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  2. [9]

    A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Vinyals, O., Rae, J

    Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de las Casas, D., Hendricks, L. A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Vinyals, O., Ra...

    2022

  3. [10]

    Settles, B., ``Active learning literature survey'', Computer Sciences Technical Report, 2009, http://axon.cs.byu.edu/ martinez/classes/778/Papers/settles.activelearning.pdf

    2009

  4. [11]

    Zhang, J., Hsieh, C.-Y., Yu, Y., Zhang, C., Ratner, A., ``A survey on programmatic weak supervision'', arXiv preprint arXiv:2202.05433, 2022

    2022 arXiv

  5. [12]

    U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., Mian, A., ``A comprehensive overview of large language models'', arXiv preprint arXiv:2307.06435, 2023

    Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., Mian, A., ``A comprehensive overview of large language models'', arXiv preprint arXiv:2307.06435, 2023

    2023 arXiv

  6. [13]

    A., Introduction to formal language theory

    Harrison, M. A., Introduction to formal language theory. Addison-Wesley Longman Publishing Co., Inc., 1978

    1978

  7. [14]

    Pad \'o , S., Lapata, M., ``Dependency-based construction of semantic space models'', Computational Linguistics, Vol. 33, No. 2, 2007, str. 161--199, https://aclanthology.org/J07-2002/

    2007

  8. [15]

    T., Furnas, G

    Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., Harshman, R., ``Indexing by latent semantic analysis'', Journal of the American society for information science, Vol. 41, No. 6, 1990, str. 391--407

    1990

  9. [16]

    K., Foltz, P

    Landauer, T. K., Foltz, P. W., Laham, D., ``An introduction to latent semantic analysis'', Discourse processes, Vol. 25, No. 2-3, 1998, str. 259--284

    1998

  10. [17]

    S., Dean, J., ``Distributed representations of words and phrases and their compositionality'', in Advances in neural information processing systems, 2013, str

    Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., Dean, J., ``Distributed representations of words and phrases and their compositionality'', in Advances in neural information processing systems, 2013, str. 3111--3119

    2013

  11. [18]

    D., `` GloVe : Global vectors for word representation'', in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, str

    Pennington, J., Socher, R., Manning, C. D., `` GloVe : Global vectors for word representation'', in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, str. 1532--1543

    2014

  12. [19]

    Mikolov, T., Chen, K., Corrado, G., Dean, J., ``Efficient estimation of word representations in vector space'', arXiv preprint arXiv:1301.3781, 2013

    2013 arXiv

  13. [20]

    F., Della Pietra, V

    Brown, P. F., Della Pietra, V. J., Desouza, P. V., Lai, J. C., Mercer, R. L., ``Class-based n-gram models of natural language'', Computational linguistics, Vol. 18, No. 4, 1992, str. 467--480

    1992

  14. [21]

    E., Hinton, G

    Rumelhart, D. E., Hinton, G. E., Williams, R. J., ``Learning internal representations by error propagation, parallel distributed processing, explorations in the microstructure of cognition, ed. de rumelhart and j. mcclelland. vol. 1. 1986'', Biometrika, Vol. 71, No. 599-607, 1...

    1986

  15. [22]

    technical report, june 1985-march 1986'', California Univ., San Diego, La Jolla (USA)

    Jordan, M., ``Serial order: a parallel distributed processing approach. technical report, june 1985-march 1986'', California Univ., San Diego, La Jolla (USA). Inst. for Cognitive Science, Tech. Rep., 1986

    1985

  16. [23]

    Hochreiter, S., Schmidhuber, J., ``Long short-term memory'', Neural computation, Vol. 9, No. 8, 1997, str. 1735--1780

    1997

  17. [24]

    Cho, K., Van Merri \"e nboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y., ``Learning phrase representations using rnn encoder-decoder for statistical machine translation'', arXiv preprint arXiv:1406.1078, 2014

    2014 arXiv

  18. [25]

    N., Kaiser, ., Polosukhin, I., ``Attention is all you need'', Advances in neural information processing systems, Vol

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, ., Polosukhin, I., ``Attention is all you need'', Advances in neural information processing systems, Vol. 30, 2017

    2017

  19. [26]

    Bahdanau, D., Cho, K., Bengio, Y., ``Neural machine translation by jointly learning to align and translate'', arXiv preprint arXiv:1409.0473, 2014

    2014 arXiv

  20. [27]

    D., ``Effective approaches to attention-based neural machine translation'', in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

    Luong, T., Pham, H., Manning, C. D., ``Effective approaches to attention-based neural machine translation'', in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: Association for Computational Linguistics, Sep. 2015, str. ...

    2015

  21. [28]

    Robbins, H., Monro, S., `` A Stochastic Approximation Method '', The Annals of Mathematical Statistics, Vol. 22, No. 3, 1951, str. 400 -- 407, https://doi.org/10.1214/aoms/1177729586

    1951

  22. [29]

    Kingma, D. P., Ba, J., ``Adam: A method for stochastic optimization'', in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Bengio, Y., LeCun, Y., (ur.), 2015, http://arxiv.org/abs/1412.6980

    2015 arXiv

  23. [30]

    Loshchilov, I., Hutter, F., ``Decoupled weight decay regularization'', in International Conference on Learning Representations, 2018

    2018

  24. [31]

    Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., Huang, X., ``Pre-trained models for natural language processing: A survey'', Science China technological sciences, Vol. 63, No. 10, 2020, str. 1872--1897

    2020

  25. [32]

    A., ``Don ' t stop pretraining: Adapt language models to domains and tasks'', in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

    Gururangan, S., Marasovi \'c , A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., Smith, N. A., ``Don ' t stop pretraining: Adapt language models to domains and tasks'', in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Asso...

    2020

  26. [33]

    Minneapolis, Minnesota: Association for Computational Linguistics, Jun

    Devlin, J., Chang, M.-W., Lee, K., Toutanova, K., `` BERT : Pre-training of deep bidirectional transformers for language understanding'', in Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Techn...

    2019

  27. [34]

    V., Manning, C

    Clark, K., Luong, M.-T., Le, Q. V., Manning, C. D., `` ELECTRA : Pre-training text encoders as discriminators rather than generators'', in ICLR, 2020, https://openreview.net/pdf?id=r1xMH1BtvB

    2020

  28. [35]

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., ``Language models are unsupervised multitask learners'', 2019, https://api.semanticscholar.org/CorpusID:160025533

    2019

  29. [36]

    J., ``Exploring the limits of transfer learning with a unified text-to-text transformer'', Journal of machine learning research, Vol

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P. J., ``Exploring the limits of transfer learning with a unified text-to-text transformer'', Journal of machine learning research, Vol. 21, No. 140, 2020, str. 1--67

    2020

  30. [37]

    Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., Hu, X., ``Harnessing the power of llms in practice: A survey on chatgpt and beyond'', ACM Transactions on Knowledge Discovery from Data, Vol. 18, No. 6, 2024, str. 1--32

    2024

  31. [38]

    Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V. et al., `` OPT : Open pre-trained transformer language models'', arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  32. [40]

    F., ``The L lama 3 herd of models'', https://arxiv.org/abs/2407.21783 2024

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., et al., A. F., ``The L lama 3 herd of models'', https://arxiv.org/abs/2407.21783 2024

    2024 arXiv

  33. [41]

    A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., et al., H

    Abdin, M., Aneja, J., Awadalla, H., Awadallah, A., Awan, A. A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., et al., H. B., ``Phi-3 technical report: A highly capable language model locally on your phone'', https://arxiv.org/abs/2404.14219 2024

    2024 arXiv

  34. [42]

    Jawahar, G., Sagot, B., Seddah, D., ``What does bert learn about the structure of language?'', in ACL 2019-57th Annual Meeting of the Association for Computational Linguistics, 2019

    2019

  35. [43]

    Florence, Italy: Association for Computational Linguistics, Jul

    Tenney, I., Das, D., Pavlick, E., `` BERT rediscovers the classical NLP pipeline'', in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, Jul. 2019, str. 4593--4601, https://aclan...

    2019

  36. [44]

    Singapore: Association for Computational Linguistics, Dec

    Liu, G., Xue, Z., Zhang, X., Johnson, K., Wang, R., `` PAC -tuning: Fine-tuning pre-trained language models with PAC -driven perturbed gradient descent'', in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Bouamor, H., Pino, J., Bali, K....

    2023

  37. [45]

    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., ``Dropout: A simple way to prevent neural networks from overfitting'', Journal of Machine Learning Research, Vol. 15, No. 56, 2014, str. 1929--1958, http://jmlr.org/papers/v15/srivastava14a.html

    2014

  38. [46]

    Online: Association for Computational Linguistics, Jun

    Yu, Y., Zuo, S., Jiang, H., Ren, W., Zhao, T., Zhang, C., ``Fine-tuning pre-trained language model with weak supervision: A contrastive-regularized self-training approach'', in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computationa...

    2021

  39. [47]

    Dublin, Ireland: Association for Computational Linguistics, May 2022, str

    Okimura, I., Reid, M., Kawano, M., Matsuo, Y., ``On the impact of data augmentation on downstream performance in natural language processing'', in Proceedings of the Third Workshop on Insights from Negative Results in NLP. Dublin, Ireland: Association for Computational Linguis...

    2022

  40. [48]

    Zhou, J., Zheng, Y., Tang, J., Li, J., Yang, Z., ``Flipda: Effective and robust data augmentation for few-shot learning'', arXiv preprint arXiv:2108.06332, 2021

    2021 arXiv

  41. [49]

    A., Yaida, S., ``Robust learning with J acobian regularization'', arXiv preprint arXiv:1908.02729, 2019

    Hoffman, J., Roberts, D. A., Yaida, S., ``Robust learning with J acobian regularization'', arXiv preprint arXiv:1908.02729, 2019

    1908 arXiv

  42. [50]

    A., Kloft, M., ``Input H essian regularization of neural networks'', arXiv preprint arXiv:2009.06571, 2020

    Mustafa, W., Vandermeulen, R. A., Kloft, M., ``Input H essian regularization of neural networks'', arXiv preprint arXiv:2009.06571, 2020

    2009 arXiv

  43. [51]

    M., Osindero, S., Jaderberg, M., Swirszcz, G., Pascanu, R., ``Sobolev training for neural networks'', Advances in neural information processing systems, Vol

    Czarnecki, W. M., Osindero, S., Jaderberg, M., Swirszcz, G., Pascanu, R., ``Sobolev training for neural networks'', Advances in neural information processing systems, Vol. 30, 2017

    2017

  44. [52]

    R., ``Robust large margin deep neural networks'', IEEE Transactions on Signal Processing, Vol

    Sokoli \'c , J., Giryes, R., Sapiro, G., Rodrigues, M. R., ``Robust large margin deep neural networks'', IEEE Transactions on Signal Processing, Vol. 65, No. 16, 2017, str. 4265--4280

    2017

  45. [53]

    Drucker, H., Le Cun, Y., ``Improving generalization performance using double backpropagation'', IEEE transactions on neural networks, Vol. 3, No. 6, 1992, str. 991--997

    1992

  46. [54]

    L., Foster, D

    Bartlett, P. L., Foster, D. J., Telgarsky, M. J., ``Spectrally-normalized margin bounds for neural networks'', in Advances in Neural Information Processing Systems, Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., (ur.), Vol. 30. C...

    2017

  47. [55]

    Q., ``On calibration of modern neural networks'', in International conference on machine learning

    Guo, C., Pleiss, G., Sun, Y., Weinberger, K. Q., ``On calibration of modern neural networks'', in International conference on machine learning. PMLR, 2017, str. 1321--1330

    2017

  48. [56]

    W., ``Verification of forecasts expressed in terms of probability'', Monthly weather review, Vol

    Brier, G. W., ``Verification of forecasts expressed in terms of probability'', Monthly weather review, Vol. 78, No. 1, 1950, str. 1--3

    1950

  49. [57]

    H., Fienberg, S

    DeGroot, M. H., Fienberg, S. E., ``The comparison and evaluation of forecasters'', Journal of the Royal Statistical Society: Series D (The Statistician), Vol. 32, No. 1-2, 1983, str. 12--22

    1983

  50. [58]

    P., Cooper, G., Hauskrecht, M., ``Obtaining well calibrated probabilities using bayesian binning'', in Proceedings of the AAAI conference on artificial intelligence, Vol

    Naeini, M. P., Cooper, G., Hauskrecht, M., ``Obtaining well calibrated probabilities using bayesian binning'', in Proceedings of the AAAI conference on artificial intelligence, Vol. 29, No. 1, 2015

    2015

  51. [59]

    et al., Principles of mathematical analysis

    Rudin, W. et al., Principles of mathematical analysis. McGraw-hill New York, 1964, Vol. 3

    1964

  52. [60]

    Khromov, G., Singh, S. P., ``Some intriguing aspects about L ipschitz continuity of neural networks'', in The Twelfth International Conference on Learning Representations, 2024, https://openreview.net/forum?id=5jWsW08zUh

    2024

  53. [61]

    Springer Publishing Company, Incorporated, 2014

    Nesterov, Y., Introductory Lectures on Convex Optimization: A Basic Course, 1st ed. Springer Publishing Company, Incorporated, 2014

    2014

  54. [62]

    35, 2022, str

    Dherin, B., Munn, M., Rosca, M., Barrett, D., ``Why neural networks find simple solutions: The many regularizers of geometric complexity'', Advances in Neural Information Processing Systems, Vol. 35, 2022, str. 2333--2349

    2022

  55. [63]

    Latorre, F., Rolland, P., Cevher, V., ``Lipschitz constant estimation of neural networks via sparse polynomial optimization'', in International Conference on Learning Representations, 2020, https://openreview.net/forum?id=rJe4_xSFDB

    2020

  56. [64]

    F., ``A stochastic estimator of the trace of the influence matrix for L aplacian smoothing splines'', Communications in Statistics-Simulation and Computation, Vol

    Hutchinson, M. F., ``A stochastic estimator of the trace of the influence matrix for L aplacian smoothing splines'', Communications in Statistics-Simulation and Computation, Vol. 18, No. 3, 1989, str. 1059--1076

    1989

  57. [65]

    27, 2018, str

    Varga, D., Csisz \'a rik, A., Zombori, Z., ``Gradient regularization improves accuracy of discriminative models'', Schedae Informaticae, Vol. 27, 2018, str. 31--45

    2018

  58. [66]

    Monographs in Mathematics

    Triebel, H., Theory of Function Spaces, ser. Monographs in Mathematics. Birkhäuser, 1983, Vol. 78

    1983

  59. [67]

    Suzuki, T., ``Adaptivity of deep re LU network for learning in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality'', in International Conference on Learning Representations, 2019, https://openreview.net/forum?id=H1ebTsActm

    2019

  60. [69]

    Elisha, O., Dekel, S., ``Function space analysis of deep learning representation layers'', arXiv preprint arXiv:1710.03263, 2017

    2017 arXiv

  61. [70]

    Stephenson, C., Padhy, S., Ganesh, A., Hui, Y., Tang, H., Chung, S., ``On the geometry of generalization and memorization in deep neural networks'', in International Conference on Learning Representations, 2021, https://openreview.net/forum?id=V8jrrnwGbuc

    2021

  62. [71]

    W., (ur.), Vol

    Baldock, R., Maennel, H., Neyshabur, B., ``Deep learning through the lens of example difficulty'', in Advances in Neural Information Processing Systems, Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J. W., (ur.), Vol. 34. Curran Associates, Inc., 2021, str. 10...

    2021

  63. [72]

    13, 2025, str

    Juki \'c , J., S najder, J., ``From robustness to improved generalization and calibration in pre-trained language models'', Transactions of the Association for Computational Linguistics, Vol. 13, 2025, str. 264--280, https://aclanthology.org/2025.tacl-1.13/

    2025

  64. [73]

    et al., ``Pytorch: An imperative style, high-performance deep learning library'', Advances in neural information processing systems, Vol

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L. et al., ``Pytorch: An imperative style, high-performance deep learning library'', Advances in neural information processing systems, Vol. 32, 2019

    2019

  65. [74]

    Online: Association for Computational Linguistics, Oct

    Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., Drame, M., Lhoest, Q., Rush, A., ``Transformers: State-...

    2020

  66. [75]

    Foret, P., Kleiner, A., Mobahi, H., Neyshabur, B., ``Sharpness-aware minimization for efficiently improving generalization'', in International Conference on Learning Representations, 2021, https://openreview.net/forum?id=6Tm1mposlrM

    2021

  67. [76]

    Brussels, Belgium: Association for Computational Linguistics, Nov

    Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., Bowman, S., `` GLUE : A multi-task benchmark and analysis platform for natural language understanding'', in Proceedings of the 2018 EMNLP Workshop B lackbox NLP : Analyzing and Interpreting Neural Networks for NLP . Brussel...

    2018

  68. [77]

    PMLR, 2019, str

    Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., Gelly, S., ``Parameter-efficient transfer learning for NLP '', in International Conference on Machine Learning. PMLR, 2019, str. 2790--2799

    2019

  69. [78]

    L., Daly, R

    Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., Potts, C., ``Learning word vectors for sentiment analysis'', in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Portland, Oregon, USA: Association...

    2011

  70. [79]

    30, 2017

    Lakshminarayanan, B., Pritzel, A., Blundell, C., ``Simple and scalable predictive uncertainty estimation using deep ensembles'', Advances in neural information processing systems, Vol. 30, 2017

    2017

  71. [80]

    W., Gal, Y., ``Uncertainty estimation using a single deep deterministic neural network'', in International conference on machine learning

    Van Amersfoort, J., Smith, L., Teh, Y. W., Gal, Y., ``Uncertainty estimation using a single deep deterministic neural network'', in International conference on machine learning. PMLR, 2020, str. 9690--9700

    2020

  72. [81]

    Dasgupta, S., ``Two faces of active learning'', Theoretical Computer Science, Vol. 412, No. 19, 2011, str. 1767-1781, algorithmic Learning Theory (ALT 2009), https://www.sciencedirect.com/science/article/pii/S0304397510007620

    2011

  73. [82]

    M., ``Generalization as search'', Artificial intelligence, Vol

    Mitchell, T. M., ``Generalization as search'', Artificial intelligence, Vol. 18, No. 2, 1982, str. 203--226

    1982

  74. [83]

    S., Shamir, E., Tishby, N., ``Selective sampling using the query by committee algorithm'', Machine learning, Vol

    Freund, Y., Seung, H. S., Shamir, E., Tishby, N., ``Selective sampling using the query by committee algorithm'', Machine learning, Vol. 28, No. 2, 1997, str. 133--168

    1997

  75. [84]

    353--360

    Hanneke, S., ``A bound on the label complexity of agnostic active learning'', in Proceedings of the 24th international conference on Machine learning, 2007, str. 353--360

    2007

  76. [85]

    B., ``Optimal aggregation of classifiers in statistical learning'', The Annals of Statistics, Vol

    Tsybakov, A. B., ``Optimal aggregation of classifiers in statistical learning'', The Annals of Statistics, Vol. 32, No. 1, 2004, str. 135--166

    2004

  77. [86]

    Balcan, M.-F., Beygelzimer, A., Langford, J., ``Agnostic active learning'', Journal of Computer and System Sciences, Vol. 75, No. 1, 2009, str. 78--89

    2009

  78. [87]

    147, 2015, str

    Wang, L., Hu, X., Yuan, B., Lu, J., ``Active learning via query synthesis and nearest neighbour search'', Neurocomputing, Vol. 147, 2015, str. 426--434

    2015

  79. [88]

    Tong, S., Koller, D., ``Support vector machine active learning with applications to text classification'', Journal of machine learning research, Vol. 2, No. Nov, 2001, str. 45--66

    2001

  80. [89]

    Proceedings of Machine Learning Research, Balcan, M

    Gal, Y., Ghahramani, Z., ``Dropout as a bayesian approximation: Representing model uncertainty in deep learning'', in Proceedings of The 33rd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, Balcan, M. F., Weinberger, K. Q., (ur.), V...

    2016

  81. [90]

    Gissin, D., Shalev-Shwartz, S., ``Discriminative active learning'', arXiv preprint arXiv:1907.06347, 2019

    1907 arXiv

  82. [91]

    Attenberg, J., Provost, F., ``Inactive learning? D ifficulties employing active learning in practice'', ACM SIGKDD Explorations Newsletter, Vol. 12, No. 2, 2011, str. 36--41

    2011

  83. [92]

    C., Wallace, B

    Lowell, D., Lipton, Z. C., Wallace, B. C., ``Practical obstacles to deploying active learning'', in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ...

    2019

  84. [93]

    Vlachos, A., ``A stopping criterion for active learning'', Computer Speech & Language, Vol. 22, No. 3, 2008, str. 295--312

    2008

  85. [94]

    Boulder, Colorado: Association for Computational Linguistics, Jun

    Bloodgood, M., Vijay-Shanker, K., ``A method for stopping active learning based on stabilizing predictions and the need for user-adjustable stopping'', in Proceedings of the Thirteenth Conference on Computational Natural Language Learning ( C o NLL -2009). Boulder, Colorado: A...

    2009

  86. [95]

    Mosbach, M., Andriushchenko, M., Klakow, D., ``On the stability of fine-tuning BERT : Misconceptions, explanations, and strong baselines'', in International Conference on Learning Representations, 2021, https://openreview.net/forum?id=nzpLWnVAyah

    2021

  87. [96]

    Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., Smith, N., ``Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping'', arXiv preprint arXiv:2002.06305, 2020

    2002 arXiv

  88. [97]

    Florence, Italy: Association for Computational Linguistics, Jul

    Kasai, J., Qian, K., Gurajada, S., Li, Y., Popa, L., ``Low-resource deep entity resolution with transfer and active learning'', in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistic...

    2019

  89. [98]

    Hong Kong, China: Association for Computational Linguistics, Nov

    Prabhu, A., Dognin, C., Singh, M., ``Sampling bias in deep active classification: An empirical study'', in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJ...

    2019

  90. [99]

    Online: Association for Computational Linguistics, Nov

    Ein-Dor, L., Halfon, A., Gera, A., Shnarch, E., Dankin, L., Choshen, L., Danilevsky, M., Aharonov, R., Katz, Y., Slonim, N., `` A ctive L earning for BERT : A n E mpirical S tudy'', in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNL...

    2020

  91. [100]

    Dublin, Ireland: Association for Computational Linguistics, May 2022, str

    Schr \"o der, C., Niekler, A., Potthast, M., ``Revisiting uncertainty-based query strategies for active learning with transformers'', in Findings of the Association for Computational Linguistics: ACL 2022. Dublin, Ireland: Association for Computational Linguistics, May 2022, s...

    2022

  92. [101]

    Online: Association for Computational Linguistics, Nov

    Yuan, M., Lin, H.-T., Boyd-Graber, J., ``Cold-start active learning through self-supervised language modeling'', in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, Nov. 2020, str...

    2020

  93. [102]

    Gothenburg, Sweden: Association for Computational Linguistics, Sep

    Juki \'c , J., S najder, J., ``Smooth sailing: Improving active learning for pre-trained language models with representation smoothness analysis'', in Proceedings of the 2023 CLASP Conference on Learning with Small Data (LSD). Gothenburg, Sweden: Association for Computational ...

    2023

  94. [103]

    Li, X., Roth, D., ``Learning question classifiers'', in COLING 2002: The 19th International Conference on Computational Linguistics, 2002, https://aclanthology.org/C02-1150

    2002

  95. [104]

    2004, str

    Pang, B., Lee, L., ``A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts'', in Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics ( ACL -04), Barcelona, Spain, Jul. 2004, str. 271--278, https:/...

    2004

  96. [105]

    28, 2015

    Zhang, X., Zhao, J., LeCun, Y., ``Character-level convolutional networks for text classification'', Advances in neural information processing systems, Vol. 28, 2015

    2015

  97. [106]

    D., Gale, W

    Lewis, D. D., Gale, W. A., ``A sequential algorithm for training text classifiers'', in SIGIR’94. Springer, 1994, str. 3--12

    1994

  98. [107]

    Sener, O., Savarese, S., ``Active learning for convolutional neural networks: A core-set approach'', in International Conference on Learning Representations, 2018, https://openreview.net/forum?id=H1aIuk-RW

    2018

  99. [108]

    Elisha, O., Dekel, S., ``Wavelet decompositions of random forests - smoothness analysis, sparse approximation and applications'', Journal of Machine Learning Research, Vol. 17, No. 198, 2016, str. 1--38, http://jmlr.org/papers/v17/15-203.html

    2016

  100. [109]

    H., Agarwal, A., Belgrave, D., Cho, K., (ur.), 2022, https://openreview.net/forum?id=CCahlgHoQG

    Bansal, R., Pruthi, D., Belinkov, Y., ``Measures of information reflect memorization patterns'', in Advances in Neural Information Processing Systems, Oh, A. H., Agarwal, A., Belgrave, D., Cho, K., (ur.), 2022, https://openreview.net/forum?id=CCahlgHoQG

    2022

  101. [110]

    M., ``Modular deep learning'', 2024, https://arxiv.org/abs/2302.11529

    Pfeiffer, J., Ruder, S., Vulić, I., Ponti, E. M., ``Modular deep learning'', 2024, https://arxiv.org/abs/2302.11529

    2024 arXiv

  102. [111]

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., ``Lo RA : Low-rank adaptation of large language models'', in International Conference on Learning Representations, 2022, https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  103. [112]

    Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov

    Lester, B., Al-Rfou, R., Constant, N., ``The power of scale for parameter-efficient prompt tuning'', in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics,...

    2021

  104. [113]

    Li, X. L., Liang, P., ``Prefix-tuning: Optimizing continuous prompts for generation'', in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)....

    2021

  105. [114]

    Dublin, Ireland: Association for Computational Linguistics, May 2022, str

    Mao, Y., Mathias, L., Hou, R., Almahairi, A., Ma, H., Han, J., Yih, S., Khabsa, M., `` U ni PELT : A unified framework for parameter-efficient language model tuning'', in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa...

    2022

  106. [115]

    Cai, W., Jiang, J., Wang, F., Tang, J., Kim, S., Huang, J., ``A survey on mixture of experts'', arXiv preprint arXiv:2407.06204, 2024

    2024 arXiv

  107. [116]

    Singapore: Association for Computational Linguistics, Dec

    Juki \'c , J., S najder, J., ``Parameter-efficient language model tuning with active learning in low-resource settings'', in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Bouamor, H., Pino, J., Bali, K., (ur.). Singapore: Association f...

    2023

  108. [117]

    D., Ng, A

    Socher, R., Bauer, J., Manning, C. D., Ng, A. Y., ``Parsing with compositional vector grammars'', in Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Sofia, Bulgaria: Association for Computational Linguistics, Aug...

    2013

  109. [118]

    Online: Association for Computational Linguistics, Aug

    He, R., Liu, L., Ye, H., Tan, Q., Ding, B., Cheng, L., Low, J., Bing, L., Si, L., ``On the effectiveness of adapter-based tuning for pretrained language model adaptation'', in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th ...

    2021

  110. [119]

    T., Trischler, A., Bengio, Y., Gordon, G

    Toneva, M., Sordoni, A., des Combes, R. T., Trischler, A., Bengio, Y., Gordon, G. J., ``An empirical study of example forgetting during deep neural network learning'', in International Conference on Learning Representations, 2019, https://openreview.net/forum?id=BJlxm30cKm

    2019

  111. [120]

    Online: Association for Computational Linguistics, Aug

    Karamcheti, S., Krishna, R., Fei-Fei, L., Manning, C., ``Mind your outliers! investigating the negative impact of outliers on active learning for visual question answering'', in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11t...

    2021

  112. [121]

    PMLR, 2019, str

    Kornblith, S., Norouzi, M., Lee, H., Hinton, G., ``Similarity of neural network representations revisited'', in International Conference on Machine Learning. PMLR, 2019, str. 3519--3529

    2019

  113. [122]

    Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., Sun, X., Li, L., Sui, Z., ``A survey on in-context learning'', https://arxiv.org/abs/2301.00234 2024

    2024 arXiv

  114. [123]

    Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec

    Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., Zettlemoyer, L., ``Rethinking the role of demonstrations: What makes in-context learning work?'', in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Goldberg, Y., Ko...

    2022

  115. [124]

    Wies, N., Levine, Y., Shashua, A., ``The learnability of in-context learning'', in Thirty-seventh Conference on Neural Information Processing Systems, 2023, https://openreview.net/forum?id=f3JNQd7CHM

    2023

  116. [125]

    Hoogland, J., Wang, G., Farrugia-Roberts, M., Carroll, L., Wei, S., Murfet, D., ``The developmental landscape of in-context learning'', arXiv preprint arXiv:2402.02364, 2024

    2024 arXiv

  117. [126]

    E., Papailiopoulos, D., Oymak, S., ``Transformers as algorithms: G eneralization and stability in in-context learning'', in International Conference on Machine Learning

    Li, Y., Ildiz, M. E., Papailiopoulos, D., Oymak, S., ``Transformers as algorithms: G eneralization and stability in in-context learning'', in International Conference on Machine Learning. PMLR, 2023, str. 19\,565--19\,594

    2023

  118. [127]

    et al., ``Debiasing in-context learning by instructing LLM s how to follow demonstrations'', in Findings of the Association for Computational Linguistics ACL 2024, 2024, str

    Li, L., Chen, J., Lu, X., Lu, Y., Lin, H., Zhou, S., Zhu, H., Wang, W., Liu, Z., Han, X. et al., ``Debiasing in-context learning by instructing LLM s how to follow demonstrations'', in Findings of the Association for Computational Linguistics ACL 2024, 2024, str. 7203--7215

    2024

  119. [128]

    Lu, Y., Bartolo, M., Moore, A., Riedel, S., Stenetorp, P., ``Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity'', arXiv preprint arXiv:2104.08786, 2021

    2021 arXiv

  120. [129]

    Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., Liang, P., `` Lost in the Middle: How Language Models Use Long Contexts '', Transactions of the Association for Computational Linguistics, Vol. 12, 02 2024, str. 157-173, https://doi.org/10.1162/tacl\...

    2024 doi

  121. [130]

    X., Wang, B., Tian, Z., Chen, W., Wen, J.-R., ``Exploring context window of large language models via decomposed positional vectors'', arXiv preprint arXiv:2405.18009, 2024

    Dong, Z., Li, J., Men, X., Zhao, W. X., Wang, B., Tian, Z., Chen, W., Wen, J.-R., ``Exploring context window of large language models via decomposed positional vectors'', arXiv preprint arXiv:2405.18009, 2024

    2024 arXiv

  122. [131]

    PMLR, 2021, str

    Zhao, Z., Wallace, E., Feng, S., Klein, D., Singh, S., ``Calibrate before use: Improving few-shot performance of language models'', in International conference on machine learning. PMLR, 2021, str. 12\,697--12\,706

    2021

  123. [132]

    Kossen, J., Gal, Y., Rainforth, T., ``In-context learning learns label relationships but is not conventional learning'', in The Twelfth International Conference on Learning Representations, 2024, https://openreview.net/forum?id=YPIA7bgd5y

    2024

  124. [133]

    Online: Association for Computational Linguistics, Apr

    Schick, T., Sch \"u tze, H., ``Exploiting cloze-questions for few-shot text classification and natural language inference'', in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Online: Association for Com...

    2021

  125. [134]

    Seattle, United States: Association for Computational Linguistics, Jul

    Rubin, O., Herzig, J., Berant, J., ``Learning to retrieve prompts for in-context learning'', in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Seattle, United States: Association f...

    2022

  126. [135]

    Singapore: Association for Computational Linguistics, Dec

    Hendel, R., Geva, M., Globerson, A., ``In-context learning creates task vectors'', in Findings of the Association for Computational Linguistics: EMNLP 2023, Bouamor, H., Pino, J., Bali, K., (ur.). Singapore: Association for Computational Linguistics, Dec. 2023, str. 9318--9333...

    2023

  127. [136]

    Liu, S., Xing, L., Zou, J., ``In-context vectors: Making in context learning more effective and controllable through latent space steering'', arXiv preprint arXiv:2311.06668, 2023

    2023 arXiv

  128. [137]

    Toronto, Canada: Association for Computational Linguistics, Jul

    Dai, D., Sun, Y., Dong, L., Hao, Y., Ma, S., Sui, Z., Wei, F., ``Why can GPT learn in-context? language models secretly perform gradient descent as meta-optimizers'', in Findings of the Association for Computational Linguistics: ACL 2023, Rogers, A., Boyd-Graber, J., Okazaki, ...

    2023

  129. [138]

    S., Mueller, A., Wallace, B

    Todd, E., Li, M., Sharma, A. S., Mueller, A., Wallace, B. C., Bau, D., ``Function vectors in large language models'', in The Twelfth International Conference on Learning Representations, 2024, https://openreview.net/forum?id=AwyxtyMwaG

    2024

  130. [139]

    Zhang, K., Lv, A., Chen, Y., Ha, H., Xu, T., Yan, R., ``Batch- ICL : E ffective, efficient, and order-agnostic in-context learning'', arXiv preprint arXiv:2401.06469, 2024

    2024 arXiv

  131. [140]

    25, 1964, str

    Aizerman, A., ``Theoretical foundations of the potential function method in pattern recognition learning'', Automation and remote control, Vol. 25, 1964, str. 821--837

    1964

  132. [141]

    PMLR, 2022, str

    Irie, K., Csord \'a s, R., Schmidhuber, J., ``The dual form of neural networks revisited: Connecting test time predictions to training patterns via spotlights of attention'', in International Conference on Machine Learning. PMLR, 2022, str. 9639--9659

    2022

  133. [142]

    Lang, H., Sontag, D., Vijayaraghavan, A., ``Theoretical analysis of weak-to-strong generalization'', arXiv preprint arXiv:2405.16043, 2024

    2024 arXiv

  134. [143]

    Singapore: Association for Computational Linguistics, Dec

    Huang, J., Gu, S., Hou, L., Wu, Y., Wang, X., Yu, H., Han, J., ``Large language models can self-improve'', in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Bouamor, H., Pino, J., Bali, K., (ur.). Singapore: Association for Computationa...

    2023

  135. [144]

    Qu, Y., Zhang, T., Garg, N., Kumar, A., ``Recursive introspection: Teaching foundation model agents how to self-improve'', in Automated Reinforcement Learning: E xploring Meta-Learning, AutoML, and LLM s, 2024, https://openreview.net/forum?id=qDXdmdBLhR

    2024

  136. [145]

    Y., Fazel-Zarandi, M., Weston, J., Li, X., ``Self-taught evaluators'', arXiv preprint arXiv:2408.02666, 2024

    Wang, T., Kulikov, I., Golovneva, O., Yu, P., Yuan, W., Dwivedi-Yu, J., Pang, R. Y., Fazel-Zarandi, M., Weston, J., Li, X., ``Self-taught evaluators'', arXiv preprint arXiv:2408.02666, 2024

    2024 arXiv

  137. [146]

    Amini, M.-R., Feofanov, V., Pauletto, L., Hadjadj, L., Devijver, E., Maximov, Y., ``Self-training: A survey'', arXiv preprint arXiv:2202.12040, 2022

    2022 arXiv

  138. [147]

    Wei, C., Shen, K., Chen, Y., Ma, T., ``Theoretical analysis of self-training with deep networks on unlabeled data'', in International Conference on Learning Representations, 2021, https://openreview.net/forum?id=rC8sJ4i6kaH

    2021

  139. [148]

    B., ``Llama 2: O pen foundation and fine-tuned chat models'', https://arxiv.org/abs/2307.09288 2023

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., et al., S. B., ``Llama 2: O pen foundation and fine-tuned chat models'', https://arxiv.org/abs/2307.09288 2023

    2023 arXiv

  140. [149]

    Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., Steinhardt, J., ``Measuring massive multitask language understanding'', in International Conference on Learning Representations, 2021, https://openreview.net/forum?id=d7KBjmI3GmQ

    2021

  141. [150]

    Loshchilov, I., Hutter, F., ``Decoupled weight decay regularization'', in International Conference on Learning Representations, 2019

    2019

  142. [151]

    Chitale, R., Vaidya, A., Kane, A., Ghotkar, A. S., ``Task arithmetic with L o RA for continual learning'', in Workshop on Advancing Neural Network Training: Computational Efficiency, Scalability, and Resource Optimization (WANT@NeurIPS 2023), 2023, https://openreview.net/forum...

    2023

  143. [152]

    Wan, F., Huang, X., Cai, D., Quan, X., Bi, W., Shi, S., ``Knowledge fusion of large language models'', in The Twelfth International Conference on Learning Representations, 2024, https://openreview.net/forum?id=jiDsk12qcz

    2024

  144. [153]

    Ilharco, G., Ribeiro, M. T., Wortsman, M., Schmidt, L., Hajishirzi, H., Farhadi, A., ``Editing models with task arithmetic'', in The Eleventh International Conference on Learning Representations, 2023, https://openreview.net/forum?id=6t0Kwf8-jrj

    2023

  145. [154]

    Ortiz-Jimenez, G., Favero, A., Frossard, P., ``Task arithmetic in the tangent space: Improved editing of pre-trained models'', in Thirty-seventh Conference on Neural Information Processing Systems, 2023, https://openreview.net/forum?id=0A9f2jZDGW

    2023

  146. [155]

    Q., Artzi, Y., `` BERTS core: Evaluating text generation with BERT '', in International Conference on Learning Representations, 2020, https://openreview.net/forum?id=SkeHuCVFDr

    Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., Artzi, Y., `` BERTS core: Evaluating text generation with BERT '', in International Conference on Learning Representations, 2020, https://openreview.net/forum?id=SkeHuCVFDr

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.