Pith. sign in

REVIEW 4 major objections 4 minor 26 references

Parametric Neural Amp Modeling with Active Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read PANAMA claims that the most informative amp settings are the ones where an ensemble of models disagrees most, and that finding them by gradient ascent beats random sampling at 64 recorded points.

desk verdict A clean late-breaking demo that makes a real case for active learning over amp knob settings, but the single-run evidence and unvalidated disagreement proxy keep it at 'promising' rather than 'proven'. read the letter →

arxiv 2507.02109 v1 pith:G4EHGLEN submitted 2025-07-02 cs.LG cs.SDeess.AS

classification cs.LGcs.SDeess.AS
keywords activelearningparametricguitarampmodelingneuralaudioeffectsensembledisagreementgradient-basedacquisitioncontinuouscontrolspacesdata-efficientmodeltraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that active learning can reduce how many amp knob settings you need to record in order to train a parametric virtual guitar amplifier. It proposes PANAMA, which keeps a small ensemble of dilated-convolutional audio models, measures their disagreement at candidate settings, and then climbs the disagreement gradient to find the next setting to record. On a 64-point budget, the chosen settings give a validation mean-squared error of $3.4 \times 10^{-4}$, versus $8.6 \times 10^{-4}$ for uniform random sampling and $9 \times 10^{-4}$ for an extreme-biased $\beta$ heuristic. If this holds, building a controllable amp model from scratch becomes a lighter task: fewer recordings of knob configurations, and less manual data collection.

What carries the argument

The central object is the cross-model disagreement $D_G(x,g)$, computed from $M$ independently trained copies of the model evaluated at a fixed input signal $x$ and a candidate knob vector $g$. It serves as a differentiable proxy for uncertainty: gradients are propagated back from $D_G$ to $g$, an optimizer climbs them to find local maxima, and the distinct maxima (usually 4 to 5 per round after clustering) become the next settings to record. This turns the question of which setting to label into a continuous optimization problem.

What would settle it

A direct control: gather 64 settings by maximizing disagreement and another 64 by minimizing it, using the same 10 starting points, ensemble size, and training budget; if the low-disagreement settings match or beat the high-disagreement settings on the same validation set, then disagreement is not the useful signal. A second control would run the same loop with a different fixed input signal for selection, to test whether the gains depend on that particular 3-minute clip.

Watch

Extended reading notes

Core claim

The paper proposes PANAMA, a parametric neural amp modeler whose acquisition step is driven by ensemble disagreement. The claim is that the best next amp settings to record are found by maximizing the cross-model disagreement $D_G(x,g) = \frac{1}{M} \operatorname{tr}(\operatorname{Var}_i[f^{(i)}_G(x,g)])$ over the continuous space of knob vectors, using gradient ascent, with the input signal held fixed. The reported evidence is that at 64 recorded settings the active-learning model reaches $3.4 \times 10^{-4}$ validation mean-squared error, versus $8.6 \times 10^{-4}$ for uniform random sampling and $9 \times 10^{-4}$ for $\beta$-distributed sampling, and that the chosen settings cluster near the extremes of the knob range.

Load-bearing premise

The load-bearing premise is that when the ensemble members disagree strongly at an unrecorded amp setting, recording that setting will actually reduce the model's error; the paper takes disagreement as a proxy for information gain without directly testing that connection.

Editorial extensions

If this is right

  • At a 64-point budget, active selection reaches roughly 2.5 times lower validation error than uniform random sampling, showing that the acquisition policy, not just the model architecture, determines data efficiency.
  • The selected settings concentrate near knob extremes, indicating that extreme settings carry information that interpolation from interior data cannot provide.
  • The beta-distribution heuristic that also favors extremes performs no better than random, so the gain is not simply sampling corners; the disagreement signal matters.
  • If the procedure scales to more knobs, the exponential growth of configurations becomes less of a blocker, since each active-learning round adds only a handful of settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same acquisition loop should transfer to other parametric audio effects, such as distortion pedals, compressors, or synthesizer parameters, where labels are expensive and the control space is continuous.
  • Because the chosen settings depend on the fixed 3-minute input signal, an untested extension would optimize over short alternative input excerpts or jointly over input and settings.
  • A practical extension is to stop adding datapoints when the top disagreement values stop decreasing, turning the fixed 64-point budget into an automatic stopping rule; the paper does not evaluate this stopping criterion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes PANAMA, an active-learning framework for training parametric guitar amplifier models. The model is a WaveNet-style network conditioned on a six-dimensional amp-knob setting vector, and the training data consists of input-signal/amp-setting pairs whose labels are recorded wet signals. The acquisition function is the average cross-model output variance over an ensemble of independently trained models, and candidate settings are found by gradient ascent on this disagreement measure over the continuous knob space. The authors report results on an amp-sim setup: starting from 10 random points and acquiring up to 64 total points, PANAMA achieves a validation MSE of 3.4e-04, compared with 8.6e-04 for uniform random sampling and 9e-04 for a beta-distribution heuristic. The paper also releases code and provides a histogram of the acquired setting values, which cluster near 0 and 1.

Significance. If the central claim is valid, PANAMA would address a genuine practical bottleneck in parametric amp modeling: reducing the number of physical recording configurations needed to train a controllable model. The contribution includes an open-source implementation, a differentiable acquisition procedure over a continuous setting space, and a direct comparison against random and heuristic sampling. These are real strengths. However, the empirical evidence is currently very thin: the headline result rests on a single table with one MSE value per method, no repeated seeds or error bars, and no independent validation of the disagreement proxy. The significance of the paper is therefore conditional on substantially stronger experimental support.

major comments (4)
  1. [Table 1] The entire load-bearing claim that active learning improves over random sampling rests on a single MSE value per method, with no reported variance, no number of seeds, and no statistical comparison. Because model training involves random initialization and data shuffling, the observed 8.6e-04 vs. 3.4e-04 gap could be within run-to-run variability. The authors should report mean and standard deviation over at least 5–10 independent runs for each sampling method, and ideally paired comparisons using the same initialization conditions.
  2. [Section 2, Eq. (1)] The acquisition function D_G(x,g) is introduced as a proxy for model uncertainty, but the paper never validates that maximizing D_G actually reduces future validation error. The sentence "we cannot compute the uncertainty directly and opt to use model disagreement... as a proxy" is an assertion, not a demonstrated property. A concrete test would be to compare settings chosen by D_G against settings chosen by simpler baselines (e.g., farthest-point sampling, random extremes, or D_G averaged over multiple input signals) and to measure the correlation between D_G values and the realized error reduction after labeling. Without such validation, the observed improvement could be an artifact of high-gain/high-sensitivity regions producing large output variance without providing generalizable information.
  3. [Section 3, experimental setup] All active-learning acquisitions are performed with a single fixed 3-minute input signal taken from NAM, while validation uses 30 minutes of multi-genre guitar audio. The paper does not address whether settings selected as informative for this one signal remain informative for other input signals. The authors should repeat the acquisition procedure with at least two or three different fixed input signals (e.g., clean, distorted, different genres) and verify that the improvement over random sampling is consistent. Otherwise the reported gain may be specific to the chosen excitation signal.
  4. [Section 3, active-learning hyperparameters] Several choices that are likely to affect the result are not ablated: ensemble size M=4, the number of Adam restarts (10), the initial dataset size (10), the total budget (64), and the clustering procedure used to deduplicate local optima. In particular, the ensemble size directly controls the quality of the disagreement estimate, and the number of restarts controls how well the continuous optimization explores the acquisition surface. The paper should include a sensitivity analysis over at least ensemble size and number of restarts to show that the method does not rely on a fragile configuration.
minor comments (4)
  1. [Section 2, Eq. (1)] The prose after the equation says "we take the cross-model variances of the output signals in element and then average them into one scalar"; this is missing the word "element-wise" and should clarify that the trace is taken over the output channels/features and then averaged over time samples.
  2. [Figure 3] The caption does not state whether the histogram aggregates the 64 acquired g-vectors or all intermediate acquisitions, nor does it note that the flattened component values are pooled across the six knobs. Please specify this in the caption.
  3. [Section 3] The sentence "For active learning, We use a model ensemble of size 4" has an unnecessary capital 'W'; also, the Adam optimizer hyperparameters (learning rate, number of steps) are not provided, which hinders reproducibility.
  4. [References] The code URL is mentioned in the text but not included in the reference list; for a late-breaking/demo paper this is acceptable, but the authors should at least provide the full repository URL and a version or commit hash for archival reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the acquisition function is computed from ensemble disagreement, and the central claim is evaluated on held-out random amp settings with an external dataset.

full rationale

The paper's claimed derivation chain is self-contained. The active learning target is D_G(x,g) = (1/M) tr(Var_i[f^(i)_G(x,g)]), which is computed purely from the outputs of independently trained ensemble members on an unlabeled datapoint (fixed input signal x and candidate settings g). The label for a datapoint is the wet signal recorded from an amp sim, and the final comparison in Table 1 is against a validation set of roughly 1,000 randomly sampled amp settings with 30 minutes of external guitar audio from the IDMT-SMT-GUITAR dataset. No parameter is fitted to the validation targets, and no equation in the paper identifies maximizing D with minimizing validation MSE by construction. The beta-distribution heuristic is motivated by a histogram of the gathered g-vectors, but the paper uses α=β=0.5 rather than the fitted values α=0.5396, β=0.4122, and the heuristic performs worse than random, so the comparison is not forced. The central concern raised by a skeptical reader—that ensemble disagreement may not be a validated proxy for label informativeness—is a correctness or robustness issue about the acquisition function, not a circularity: the outcome is empirically testable and the paper reports a non-trivial result. There are no load-bearing self-citations: references to NAM, WaveNet, and IDMT-SMT-GUITAR are external prior work. Overall, the derivation reduces neither by definition nor by fitted-parameter renaming, so the circularity score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method relies on standard supervised learning plus a domain assumption that ensemble output variance predicts useful data. No numbers are fitted to the validation target; the listed free parameters are experimental choices. No new physical or ontological entities are introduced.

free parameters (6)
  • Ensemble size M = 4
    Hand-chosen ensemble size for disagreement estimation; not swept or justified in the paper.
  • Initial datapoints = 10
    Active learning starts from 10 randomly chosen amp settings, a hand-set budget.
  • Adam restarts = 10
    10 independent gradient-based optimizations are used to find candidate settings; no sensitivity analysis is reported.
  • Total datapoint budget = 64
    Comparison point for sampling methods; chosen as the constrained budget for the experiment.
  • Training epochs = 50
    Fixed training length for all models; effect on the active learning comparison is not explored.
  • Beta heuristic parameters = alpha=beta=0.5
    Chosen as a heuristic baseline; the paper also fits alpha=0.5396, beta=0.4122 to the observed histogram but does not use those values in the experiment.
assumptions (4)
  • domain assumption Cross-model disagreement is a valid proxy for informativeness
    Section 2 states 'we cannot compute the uncertainty directly and opt to use model disagreement over the datapoint across an ensemble, denoted as D, as a proxy.' The paper does not verify that maximizing D reduces future error.
  • domain assumption The fixed input signal x is representative for selecting knob settings
    The optimal g is found for a single pre-chosen, roughly 3-minute signal from NAM; the paper assumes settings chosen with this signal generalize to other guitar inputs.
  • domain assumption A feed-forward WaveNet-like architecture can represent the parametric amp transform
    The model is adapted from WaveNet and NAM; the paper relies on prior work for the architecture's sufficiency rather than demonstrating it here.
  • standard math Validation settings are sampled i.i.d. and MSE is a meaningful metric
    Validation uses roughly 1000 random amp settings from IDMT-SMT-GUITAR; this is a standard evaluation assumption, though no error bars are provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Parametric Neural Amp Modeling with Active Learning." pith.science (2026). https://pith.science/paper/G4EHGLEN

@misc{pith2026250702109,
  author       = {Pith},
  title        = {Pith review of: Parametric Neural Amp Modeling with Active Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G4EHGLEN}},
  note         = {Machine review of arXiv:2507.02109}
}
read the original abstract

We introduce PANAMA, an active learning framework for the training of end-to-end parametric guitar amp models using a WaveNet-like architecture. With \model, one can create a virtual amp by recording samples that are determined by an active learning strategy to use a minimum amount of datapoints (i.e., amp knob settings). We show that gradient-based optimization algorithms can be used to determine the optimal datapoints to sample, and that the approach helps under a constrained number of samples.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 24 canonical work pages

  1. [1]

    Parametric Neural Amp Modeling with Active Learning

    INTRODUCTION In recent years, data-driven guitar amp modeling has be- come increasing popular. Such approaches treat an amp as a blackbox, and simply learn the transform which the amp applies to the raw guitar signal in an end-to-end fash- ion. The trained models can then be integrated as plugins into Digital Audio Workstations (DAWs), or deployed on mode...

  2. [2]

    METHODOLOGY Model Architecture.We adapt the WaveNet [21] architec- ture to be used in a feed-forward manner, similar to prior works. WaveNet was originally developed as an autore- gressive generative model, which processes audio using a stack of dilated convolutional layers and predicts a categor- ical distribution over the next sample to be generated. We...

  3. [3]

    This in- cludes the following parameters: Gain, Bass, Mid, Treble, Master, Presence

    EXPERIMENTAL EV ALUA TION For our experiments, we choosedim(g) = 6. This in- cludes the following parameters: Gain, Bass, Mid, Treble, Master, Presence. The Master knob is included, as it dif- fers from a standard volume knob. The fixed input signal is taken from NAM [20] and roughly 3 minutes long. We ob- tain the ground truth signals from an amp sim. Al...

  4. [4]

    Deep Learning for Tube Amplifier Emulation

    E.-P. Damskägg, L. Juvela, E. Thuillier, and V . Välimäki, “Deep learning for tube am- plifier emulation,” 2019. [Online]. Available: https://arxiv.org/abs/1811.00334

  5. [5]

    Real- time modeling of audio distortion circuits with deep learning,

    E.-P. Damskägg, L. Juvela, and V . Välimäki, “Real- time modeling of audio distortion circuits with deep learning,” 2019. [Online]. Available: https: //api.semanticscholar.org/CorpusID:196205025

  6. [6]

    Deep learning for black-box modeling of audio effects,

    M. A. Martínez Ramírez, E. Benetos, and J. D. Reiss, “Deep learning for black-box modeling of audio effects,”Applied Sciences, vol. 10, no. 2,

  7. [7]

    A vacuum- tube guitar amplifier model using a recurrent neural network,

    J. M. Covert and D. L. Livingston, “A vacuum- tube guitar amplifier model using a recurrent neural network,”2013 Proceedings of IEEE Southeastcon, pp. 1–5, 2013. [Online]. Available: https://api. semanticscholar.org/CorpusID:38784328

  8. [9]

    Efficient neural networks for real-time modeling of analog dynamic range compression,

    C. J. Steinmetz and J. D. Reiss, “Efficient neural networks for real-time modeling of analog dynamic range compression,” 2022. [Online]. Available: https: //arxiv.org/abs/2102.06200

Show all 26 references
  1. [10]

    Modelling black-box audio effects with time-varying feature modulation,

    M. Comunità, C. Steinmetz, H. Phan, and J. D. Reiss, “Modelling black-box audio effects with time-varying feature modulation,”ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, 2022. [Online]. Available: https://api.s...

  2. [11]

    A general-purpose deep learning approach to model time- varying audio effects,

    M. A. M. Ramírez, E. Benetos, and J. D. Reiss, “A general-purpose deep learning approach to model time- varying audio effects,”ArXiv, vol. abs/1905.06148,

  3. [12]

    A vacuum-tube guitar amplifier model using long/short-term memory net- works,

    Z. Zhang, E. Olbrych, J. Bruchalski, T. J. Mc- Cormick, and D. L. Livingston, “A vacuum-tube guitar amplifier model using long/short-term memory net- works,”SoutheastCon 2018, pp. 1–5, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID: 52935728

  4. [13]

    Real time emulation of parametric guitar tube amplifier with long short term memory neural network,

    T. Schmitz and J.-J. Embrechts, “Real time emulation of parametric guitar tube amplifier with long short term memory neural network,” 2018. [Online]. Available: https://arxiv.org/abs/1804.07145

  5. [14]

    Nonlinear modeling of the guitar signal chain enabling its real-time emulation,

    T. Schmitz, “Nonlinear modeling of the guitar signal chain enabling its real-time emulation,” 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID: 201147276

  6. [15]

    Pruning deep neural network models of guitar distortion effects,

    D. Sudholt, A. Wright, C. Erkut, and V . Välimäki, “Pruning deep neural network models of guitar distortion effects,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 256–264, 2023. [Online]. Available: https://api. semanticscholar.org/CorpusID:253694043

  7. [16]

    Sample rate independent recurrent neural networks for audio effects processing,

    A. Carson, A. Wright, J. Chowdhury, V . Välimäki, and S. Bilbao, “Sample rate independent recurrent neural networks for audio effects processing,”ArXiv, vol. abs/2406.06293, 2024. [Online]. Available: https: //api.semanticscholar.org/CorpusID:270371338

  8. [17]

    Real- time black-box modelling with recurrent neural net- works,

    A. Wright, E.-P. Damskägg, and V . Välimäki, “Real- time black-box modelling with recurrent neural net- works,” 09 2019

  9. [18]

    Neural modelling of periodically modulated time-varying effects,

    A. Wright and V . Välimäki, “Neural modelling of periodically modulated time-varying effects,” 2020. [Online]. Available: https://api.semanticscholar.org/ CorpusID:226306209

  10. [19]

    Neural modeling of phaser and flanging effects,

    A. Wright and V . Välimäki, “Neural modeling of phaser and flanging effects,”Journal of the Audio Engineering Society, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:236505785

  11. [20]

    Atkinson

    S. Atkinson. (2025) Neural amp modeller. Ac- cessed: 2025-06-13. [Online]. Available: https: //www.neuralampmodeler.com/

  12. [21]

    Wavenet: A generative model for raw audio,

    A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” 2016. [Online]. Available: https://arxiv.org/abs/1609.03499

  13. [22]

    Hyper recurrent neural network: Condition mechanisms for black-box audio effect modeling,

    Y .-T. Yeh, W.-Y . Hsiao, and Y .-H. Yang, “Hyper recurrent neural network: Condition mechanisms for black-box audio effect modeling,”ArXiv, vol. abs/2408.04829, 2024. [Online]. Available: https: //api.semanticscholar.org/CorpusID:271843407

  14. [23]

    Real-time guitar amplifier em- ulation with deep learning,

    A. Wright, E.-P. Damskägg, L. Juvela, and V . Välimäki, “Real-time guitar amplifier em- ulation with deep learning,”Applied Sciences, vol. 10, no. 3, 2020. [Online]. Available: https://www.mdpi.com/2076-3417/10/3/766

  15. [24]

    End-to-end amp modeling: From data to controllable guitar amplifier models,

    L. Juvela, E.-P. Damskägg, A. Peussa, J. Mäkinen, T. Sherson, S. I. Mimilakis, and A. Gotsopoulos, “End-to-end amp modeling: From data to controllable guitar amplifier models,” 2024. [Online]. Available: https://arxiv.org/abs/2403.08559

  16. [27]

    IDMT- SMT-Guitar dataset,

    C. Kehling, A. Männchen, and A. Eppler, “IDMT- SMT-Guitar dataset,” 2023. [Online]. Available: https: //doi.org/10.5281/zenodo.7544110

  17. [2019]

    Available: https://api.semanticscholar

    [Online]. Available: https://api.semanticscholar. org/CorpusID:155093236

  18. [2020]

    Available: https://www.mdpi.com/ 2076-3417/10/2/638

    [Online]. Available: https://www.mdpi.com/ 2076-3417/10/2/638

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.