Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

SwiftF0: Fast and Accurate Monophonic Pitch Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 95,842-parameter CNN that reads a band-limited STFT spectrogram and decodes pitch by a local expected value is claimed to beat CREPE on monophonic pitch estimation, with the largest gains under added noise and a 42x CPU speed-up.

desk verdict Compact, fast pitch tracker with a useful open benchmark; the SOTA claim currently leans too hard on a self-generated test set and missing per-dataset numbers. read the letter →

arxiv 2508.18440 v1 pith:VOS22KH7 submitted 2025-08-25 cs.SD cs.AIcs.LGeess.AS

Lars Nieradzik This is my paper
classification cs.SDcs.AIcs.LGeess.AS
keywords pitchdetectionF0estimationmonophonicSTFT-CNNnoiserobustnesssyntheticspeechdatasetevaluationmetricreal-timeaudio
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a compact neural network, SwiftF0, can outperform much larger pitch-estimation models like CREPE while running about 42 times faster on a CPU. The key idea is to feed the network a frequency-band-limited STFT magnitude spectrogram rather than raw audio, and to train it with a loss that directly optimizes the same local expected-value decoding used at inference. On a three-dataset benchmark, SwiftF0 is reported to reach a harmonic mean of 91.80% at 10 dB SNR, ahead of the next-best system by over 12 percentage points, with only a 2.3-point drop from clean audio. The paper also introduces SpeechSynth, a synthetic speech dataset with exact ground-truth pitch, and a unified harmonic-mean metric that combines six pitch-evaluation measures. If the claims hold, accurate and robust pitch tracking becomes practical on resource-constrained and real-time systems.

What carries the argument

The load-bearing mechanism is the local expected-value decoder applied to a softmax distribution over 200 logarithmically spaced pitch bins. At inference, the model finds the argmax bin, normalizes probabilities within a window of plus or minus nine bins, and computes a weighted average of bin frequencies as the continuous pitch estimate. The training objective includes an L1 loss in log-frequency space on exactly this expected value, so the model is directly optimized for its decoding procedure; the same probability mass near the peak also yields a voicing confidence score.

What would settle it

Run SwiftF0 and CREPE on an independent held-out natural speech corpus with reliable pitch ground truth (for example, laryngograph-based recordings not used in training) and on a standard music pitch dataset, computing the same harmonic mean at 10 dB SNR. If the gap to CREPE falls substantially below the reported 12 percentage points, or if CREPE's harmonic mean exceeds SwiftF0's on those sets, the claimed state-of-the-art advantage is benchmark-dependent.

Watch

Extended reading notes

Core claim

The paper's central claim is that a lightweight STFT-CNN with 95,842 parameters can set a new state of the art for monophonic pitch estimation, beating CREPE by more than 12 percentage points in harmonic mean at 10 dB SNR and degrading only slightly from clean conditions. The architecture selects only STFT bins between 46.875 Hz and 2093.75 Hz, reducing input size by 74%, then passes a log-compressed spectrogram through five 2D convolutional layers before projecting to 200 logarithmically spaced pitch bins. Training combines categorical cross-entropy over bins with an L1 loss on expected log-frequency, so the model is explicitly optimized for the local expected-value decoder used at inferenc

Load-bearing premise

SwiftF0's claimed lead over CREPE rests on the assumption that the averaged three-dataset benchmark, which includes a synthetic speech test set generated by the same TTS-family approach used in training, fairly represents monophonic pitch estimation across real speech and music.

Editorial extensions

If this is right

  • If the reported results hold, monophonic pitch estimation can be both highly accurate under noise and fast enough for real-time or embedded deployment on a CPU.
  • The combination of band-limited STFT input and an expected-value decoder trained with a matching loss offers a concrete design template for other frequency-estimation tasks.
  • SpeechSynth-style synthetic data with exact ground-truth pitch could support training and evaluation for speech applications where algorithmic pitch labels are unreliable.
  • The harmonic-mean metric, by combining raw pitch accuracy, cents accuracy, voicing precision and recall, octave accuracy, and gross-error accuracy, discourages models from optimizing one metric at the expense of others.
  • The reported stability under noise suggests that aggressive augmentation with mixed environmental and white noise can transfer to held-out domains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The claimed 12-point lead over CREPE may be sensitive to benchmark composition: two of the three evaluation sets are synthetic, and the SpeechSynth test set is generated with the same TTS-family approach used for training, so a comparison on natural speech and music could shrink the gap.
  • The harmonic-mean metric's exponential weightings (e.g., 500 for cents error, 10 for octave errors, 5 for gross errors) are design choices; reweighting these factors could change relative rankings among the evaluated methods.
  • The local expected-value decoder plus log-frequency L1 loss is a transferable mechanism: it could plausibly improve other pitch- or frequency-estimation networks without architectural changes.
  • A natural next test is to evaluate SwiftF0 under non-stationary noise, reverberation, and varying SNR levels beyond the single 10 dB condition used in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents SwiftF0, a lightweight STFT-CNN pitch tracker (95,842 parameters) that predicts pitch via a learned softmax over 200 log-spaced bins and decodes with a local expected value. Training combines cross-entropy with an L1 loss on expected log-frequency and uses a large, augmented multi-dataset corpus. The authors claim a new state-of-the-art for monophonic pitch estimation, reporting a harmonic mean (HM) of 91.80% at 10 dB SNR, roughly 12 percentage points above CREPE, with only a 2.3-point drop from clean conditions. They also introduce SpeechSynth, a synthetic Mandarin TTS speech dataset with exact ground-truth F0, and a new six-metric harmonic-mean evaluation score. The paper includes open-source code and a benchmark suite.

Significance. If the reported results hold, SwiftF0 would be a practically valuable contribution: it is very compact, fast on CPU, robust to noise, and the synthetic dataset idea addresses a real weakness in current speech pitch labels. The open-source implementation and benchmark suite are important reproducibility assets. However, the headline 'state-of-the-art' claim is not currently supported by the evidence, because the evaluation is averaged over only three datasets, one of which is generated by the same TTS pipeline used for training, and the aggregate metric is author-defined with unexamined constants. The core method is plausible and the work is not internally contradictory, but the strength of the empirical claims substantially exceeds what the paper currently demonstrates.

major comments (4)
  1. [§4, Tables 1–2] All results are reported as averages over only three datasets, with no per-dataset breakdown, no variance, and no confidence intervals. Since one of those datasets (SpeechSynth) is generated by the same LightSpeech pipeline family used for training, the reader cannot determine whether the 91.80% HM and the >12-point margin over CREPE are driven by an in-distribution benchmark. Please report per-dataset scores for every metric (at least for SwiftF0 and CREPE), add variability estimates over held-out segments, and include an evaluation on a natural-speech corpus with reference pitch that was not used in training. Without this, the 'new state-of-the-art' claim is not empirically verifiable.
  2. [§3.8, HM metric] The harmonic mean HM = 6 / Σ(1/c_i) uses author-chosen constants (500, 10, 5) and an author-selected set of six components. The text says 500 was chosen to prevent HM from dropping to zero and 10/5 were chosen to weight octave and gross errors, but no sensitivity analysis is given. The paper's central comparison is based on this metric, so the ranking may not be robust to reasonable alternative coefficient choices. Please provide a sensitivity analysis over these constants (e.g., vary each by a factor of 2) and show that the ranking of SwiftF0 versus CREPE and pYIN is unchanged. In addition, report the six component metrics per dataset, not only as averages.
  3. [§3.6, SpeechSynth test set] The SpeechSynth test set is described only as 'independently generated' and 'not seen during training.' This guarantees non-overlap of instances, not independence of distribution: training and test come from the same LightSpeech generator trained on the same speech corpora. A model can learn TTS artifacts rather than general pitch structure and still look strong on this test set. Please provide evidence of distribution shift, such as per-dataset results on a held-out natural-speech dataset (e.g., a PTDB-TUG/MIR-1k split excluded from training), or a comparison of SwiftF0's performance on SpeechSynth versus natural speech. This is a generalization-risk concern, not an internal inconsistency, but it is load-bearing for the headline claim.
  4. [§3.4 vs §3.5] The training loss Lcents minimizes L1 error on the expected log-frequency,  log(f)  = Σ p_b log(f_b). At inference, however, the local expected value is computed as a weighted average of linear bin center frequencies,  ​f = Σ p̃_b f_b. The paper states that 'the continuous estimate ... is computed using the same expected log-frequency formulation as in the regression loss,' which is inaccurate. This mismatch means the training objective does not directly optimize the inference decoding. Please either align the inference equation with the training objective (e.g., use the expected log-frequency and exponentiate), or add a loss term that matches the local expected-value decoding, and re-run the experiments if this changes results.
minor comments (5)
  1. [§3.5] The voicing confidence threshold is given only as 'approximately 90%.' The exact value should be specified for reproducibility, and the sensitivity to this threshold should be reported since voicing F1 is part of HM.
  2. [Figure 1] The bar chart labels in Figure 1 are confusing: the axis and legend mix 'Raw Chroma Accuracy,' 'Note Accuracy,' and 'Octave Accuracy.' Clarify which metric each panel shows.
  3. [References] The Biaobei reference appears as '[bia, 2017]' in the text but the bibliography entry lacks a key in the reference list; also check the citation key consistency for all URLs (e.g., the [SPTK developers, 2025] entry).
  4. [§3.3] The local expected value window half-width w=9 is stated without justification. A sentence on how this was chosen, or a small ablation, would strengthen the method section.
  5. [§4.3] Runtime results are given only as averages with no indication of variance or the specific CPU model. Since speed is a major contribution, report at least the hardware details and ideally standard deviations over repeated runs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is self-contained and the evaluation, while having validity limitations, does not reduce to the paper's own inputs by construction.

full rationale

I walked the paper's claimed derivation chain: SwiftF0 is a trained STFT-CNN with a classification-plus-regression objective, and its reported accuracy is measured against held-out audio (Vocadito, Bach10-mf0-synth, and an independently generated SpeechSynth test set) using a six-component harmonic mean. I found no step where a prediction is equivalent to a fitted input by construction. The harmonic mean constants (500, 10, 5) are author-chosen metric weights, not parameters fitted to SwiftF0's outputs, and the paper does not claim to derive them from first principles. The fact that SpeechSynth is generated by the same TTS family used in training makes the test split distributionally close to training, and the lack of per-dataset breakdowns and confidence intervals is a legitimate external-validity concern, but it is not circularity: the test instances are distinct and the model's outputs are not defined in terms of the test labels. The local expected value at inference is aligned with the regression loss, but this is an optimization-design choice, not a reduction of the reported result to the loss definition. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The central accuracy claim therefore has independent empirical content, even though its generalization beyond the evaluated domains remains uncertain.

Assumptions & free parameters 8 free parameters · 5 assumptions · 1 invented entities

The central results depend on a number of hand-selected settings: STFT parameters, frequency band, pitch-bin count, loss weight, inference window, confidence threshold, augmentation ranges, and metric coefficients. Most are reasonably motivated but not justified by sensitivity analysis. The most consequential assumption is that the evaluation domains and the author-defined metric are neutral arbiters of a broad state-of-the-art claim.

free parameters (8)
  • STFT window length and hop = N=1024, H=256
    Chosen as best balance between efficiency and performance after experiments (Section 3.2).
  • Frequency band = fmin=46.875 Hz, fmax=2093.75 Hz
    Design choice to cover vocal and instrumental pitch while removing 74% of bins (Section 3.2).
  • Number of pitch bins = B=200 (approx. 33.1 cents per bin)
    Empirically no improvement beyond 200 bins (Section 3.3).
  • Loss weight = lambda=1
    Equal weighting of classification and regression; variation said to have minimal impact (Section 3.4).
  • Local expected value window half-width = w=9
    Adopted for inference without data-driven justification (Section 3.5).
  • Voicing confidence threshold = ~0.90
    Used to discard low-confidence frames; no calibration analysis (Section 3.5).
  • HM metric coefficients = CA decay 500, OA scale 10, GEA scale 5, RPA tolerance 50 cents, gross error threshold 200 cents
    Hand-chosen; 500 chosen to prevent HM dropping to zero, others chosen by perceived importance (Section 3.8).
  • Augmentation ranges = gain [-6,+6] dB, SNR [10,30] dB, alpha in [0,1]
    Heuristic choices to create realistic noisy training examples (Section 3.7).
assumptions (5)
  • domain assumption STFT magnitude spectrogram is a sufficient input representation for pitch estimation.
    Invoked in Section 3.2 to justify using spectral input rather than raw waveform.
  • domain assumption Algorithmically-derived labels in PTDB-TUG (RAPT), MIR-1k (YIN), and Vocadito (pYIN) are accurate enough to serve as training targets.
    Section 3.6 acknowledges label impurities but still uses these datasets for training.
  • domain assumption LightSpeech-generated phoneme-level TTS provides exact ground-truth pitch contours.
    Section 3.6 introduces SpeechSynth as having perfectly accurate pitch labels, but does not provide independent validation.
  • ad hoc to paper The harmonic mean of six metrics with the chosen constants is a valid single measure of pitch tracker quality.
    Section 3.8 defines the metric and constants; no external consensus or sensitivity analysis is given.
  • domain assumption Training only on voiced frames, with a fixed confidence threshold at inference, is sufficient for unvoiced handling.
    Section 3.7 and Section 3.5 describe this pipeline; the paper notes the confidence scores are not true voicing probabilities.
invented entities (1)
  • SpeechSynth synthetic speech dataset
    purpose: Provide exact, on-demand ground-truth pitch curves for training and evaluation.
    No release URL or independent validation is provided; the evaluation test set is generated by the same TTS family used in training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SwiftF0: Fast and Accurate Monophonic Pitch Detection." pith.science (2026). https://pith.science/paper/VOS22KH7

@misc{pith2026250818440,
  author       = {Pith},
  title        = {Pith review of: SwiftF0: Fast and Accurate Monophonic Pitch Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VOS22KH7}},
  note         = {Machine review of arXiv:2508.18440}
}
read the original abstract

Accurate and real-time monophonic pitch estimation in noisy conditions, particularly on resource-constrained devices, remains an open challenge in audio processing. We present \emph{SwiftF0}, a novel, lightweight neural model that sets a new state-of-the-art for monophonic pitch estimation. Through training on diverse speech, music, and synthetic datasets with extensive data augmentation, SwiftF0 achieves robust generalization across acoustic domains while maintaining computational efficiency. SwiftF0 achieves a 91.80\% harmonic mean (HM) at 10 dB SNR, outperforming baselines like CREPE by over 12 percentage points and degrading by only 2.3 points from clean audio. SwiftF0 requires only 95,842 parameters and runs approximately 42x faster than CREPE on CPU, making it ideal for efficient, real-time deployment. To address the critical lack of perfectly accurate ground truth pitch in speech corpora (which typically rely on algorithmic estimators or laryngograph signals), we introduce \emph{SpeechSynth}. This synthetic speech dataset, generated by a phoneme-level TTS model, provides exact, on-demand ground-truth pitch curves, enabling more robust model training and evaluation. Furthermore, we propose a unified metric, combining six complementary performance measures for comprehensive and reliable pitch evaluation, and release an open-source pitch benchmark suite. A live demo of SwiftF0 is available at https://swift-f0.github.io/, the source code at https://github.com/lars76/swift-f0, and the benchmark framework at https://github.com/lars76/pitch-benchmark.

Figures

Figures reproduced from arXiv: 2508.18440 by the authors.

Figure 1
Figure 1. SwiftF0 is a new monophonic pitch estimation algorithm achieving superior pitch accuracy compared to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pitch detection model architecture showing data flow from audio input through STFT, frequency band [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. shows the mel spectrogram and F0 traces for the word “the” (/D@/). 0.000 0.015 0.030 0.045 0.060 0.075 0.090 0.105 0.120 0.135 0.150 Time 75 100 125 150 175 200 225 250 275 300 Mel Scale (Hz) 212.5 215.0 217.5 220.0 222.5 225.0 227.5 Pitch (Hz) Praat SwiftF0 CREPE pYIN PENN ð [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Mel spectrogram and F0 contours for the word “now” (/nAU:/). SwiftF0 maintains a continuous pitch track, whereas other methods introduce gaps or false detections. 95,842 parameters, a frequency band selection strategy that reduces input size by 74%, and the elimination…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Is Natural Always Appropriate? Investigating Naturalness and Appropriateness Across Different Domains for TTS Evaluation

    eess.AS 2026-06 unverdicted novelty 6.0 of 10

    Appropriateness of TTS varies independently across domains while naturalness scores penalize stylized speech and reward spontaneity.

Reference graph

Works this paper leans on

15 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [7]

    Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu

    doi:10.1109/ICASSP.2014.6854950. Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. FastSpeech 2: Fast and high-quality end-to-end text to speech,

    work page arXiv 2014

  2. [8]

    Renqian Luo, Xu Tan, Rui Wang, Tao Qin, Jinzhu Li, Sheng Zhao, Enhong Chen, and Tie-Yan Liu

    URL https://arxiv.org/abs/2006.04558. Renqian Luo, Xu Tan, Rui Wang, Tao Qin, Jinzhu Li, Sheng Zhao, Enhong Chen, and Tie-Yan Liu. LightSpeech: Lightweight and fast text to speech with neural architecture search,

    arXiv 2006

  3. [11]

    Biaobei dataset

    URL https://arxiv.org/abs/2010.11567. Biaobei dataset. https://www.data-baker.com/datasets/freeDatasets,

    arXiv 2010

  4. [14]

    SPTK developers

    doi:10.1016/j.wocn.2018.07.001. SPTK developers. Speech Signal Processing Toolkit (SPTK),

    doi:10.1016/j.wocn.2018.07.001 2018

  5. [2002]

    Matthias Mauch and Simon Dixon

    doi:10.1121/1.1458024. Matthias Mauch and Simon Dixon. pYIN: A fundamental frequency estimator using probabilistic threshold distributions. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 659–663,

    doi:10.1121/1.1458024

  6. [2008]

    URL https://asa.scitation

    doi:10.1121/1.2916590. URL https://asa.scitation. org/doi/10.1121/1.2916590. Max Morrison, Caedon Hsieh, Nathan Pruyne, and Bryan Pardo. Cross-domain neural pitch and periodicity estimation,

    work page doi:10.1121/1.2916590

  7. [2010]

    Justin Salamon, Rachel M

    doi:10.1109/TASL.2009.2026503. Justin Salamon, Rachel M. Bittner, Jordi Bonada, Juan J. Bosch, Emilia Gómez, and Juan P. Bello. An analysis/synthesis framework for automatic F0 annotation of multitrack datasets. In Proceedings of the 18th International Society for Music Information Retrieval Conference (ISMIR) , Suzhou, China, October

    work page arXiv 2009

  8. [2014]

    URL https://ieeexplore.ieee.org/document/ 6853681

    doi:10.1109/ICASSP.2014.6853681. URL https://ieeexplore.ieee.org/document/ 6853681. David Talkin. A robust algorithm for pitch tracking (RAPT). In W. B. Kleijn and K. K. Paliwal, editors, Speech Coding and Synthesis, pages 495–518. Elsevier, Amsterdam,

    work page arXiv 2014

Show all 15 references
  1. [2015]

    Yannick Jadoul, Benjamin Thompson, and Bas de Boer

    doi:10.1109/W ASPAA.2015.7336899. Yannick Jadoul, Benjamin Thompson, and Bas de Boer. Introducing Parselmouth: A Python interface to Praat. Journal of Phonetics, 71:1–15,

    2015

  2. [2017]

    Peter Foster, Siddharth Sigtia, Sacha Krstulovic, Jon Barker, and Mark D

    DataBaker (Beijing) Technol- ogy Co., Ltd. Peter Foster, Siddharth Sigtia, Sacha Krstulovic, Jon Barker, and Mark D. Plumbley. Chime-home: A dataset for sound source recognition in a domestic environment. In 2015 IEEE Workshop on Applications of Signal Processing to Audio and ...

    2015

  3. [2018]

    12 SwiftF0 A PREPRINT Paul Boersma and David Weenink

    URL https://arxiv.org/abs/1802.06182. 12 SwiftF0 A PREPRINT Paul Boersma and David Weenink. Praat: doing phonetics by computer [Computer program] ,

    arXiv

  4. [2021]

    Yao Shi, Hui Bu, Xin Xu, Shaoji Zhang, and Ming Li

    URL https://arxiv.org/abs/2110.05580. Yao Shi, Hui Bu, Xin Xu, Shaoji Zhang, and Ming Li. AISHELL-3: A multi-speaker mandarin TTS corpus and the baselines,

    arXiv

  5. [2022]

    org/abs/2203.09893

    URL https://arxiv. org/abs/2203.09893. Sander Dieleman and Benjamin Schrauwen. End-to-end learning for music audio. In 2014 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 6964–6968,

    2014 arXiv

  6. [2023]

    URL https://zenodo.org/doi/10.5281/zenodo. 7499098. 14

    doi

  7. [2025]

    Version 6.4.39, retrieved 13 July 2025 from https://praat.org

    URL https: //praat.org. Version 6.4.39, retrieved 13 July 2025 from https://praat.org. Alain de Cheveigné and Hideki Kawahara. YIN, a fundamental frequency estimator for speech and music. The Journal of the Acoustical Society of America , 111(4):1917–1930,

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.