Can We Trust Machine Learning? The Reliability of Features from Open-Source Speech Analysis Tools for Speech Modeling

Tahiya Chowdhury; Veronica Romero

REVIEW 4 major objections 6 minor 39 references

Can We Trust Machine Learning? The Reliability of Features from Open-Source Speech Analysis Tools for Speech Modeling

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read OpenSMILE and Praat return significantly different values for the same speech audio, and the choice of tool changes machine-learning predictions about autism diagnosis.

desk verdict A well-intentioned cautionary study whose central pitch comparisons may be invalid due to an apparent unit mismatch between OpenSMILE and Praat; it needs major clarification before the numbers can be trusted. read the letter →

arxiv 2506.11072 v1 pith:AGPJOTJX submitted 2025-06-02 eess.AS cs.CLcs.CYcs.SDstat.AP

Tahiya Chowdhury , Veronica Romero This is my paper

classification eess.AScs.CLcs.CYcs.SDstat.AP

keywords speechfeatureextractionOpenSMILEPraatautismspectrumdisordermachinelearningreliabilityreproducibilityrateprosodicfeatures

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that feature-extraction tool choice is an uncontrolled source of variance in behavioral speech modeling. Using the same utterance-level audio from autism diagnostic sessions, the authors show that OpenSMILE and Praat return statistically different values for every feature examined—mean pitch, pitch variability, pitch range, loudness, and speech rate—with p < 0.001. These differences are not neutral: two-way ANOVAs find group differences (by gender and by diagnosis) that appear only with one tool, and random-forest classifiers trained on the two tools' features perform differently across tasks and diagnostic groups. The authors do not claim one tool is correct; they argue that default open-source feature extraction lacks validation, so published clinical machine-learning results may not be comparable, and they call for reporting tool parameters and domain-relevant verification.

What carries the argument

The engine is a paired comparison design on a shared audio corpus. Each utterance-level audio file from ADOS-2 sessions is processed by both tools under default settings; the five features are aggregated to task level, then compared with paired t-tests and two-way ANOVAs with tool-by-group interaction, followed by Bonferroni post-hoc tests. The same features feed separate leave-one-user-out random forests to measure downstream classification consequences. The sanity check that exposes the mechanism is the speech-rate comparison: Praat's intensity-based syllable counting yields behaviorally impossible rates on noisy automated diarization output, whereas OpenSMILE's heuristic voiced-segment mapping stays near plausible values.

What would settle it

Take the same ADOS-2 recordings, produce utterance-level audio via manual diarization and forced alignment, and re-extract the five features with both tools. If Praat's speech-rate estimates fall to plausible values and the cross-tool differences largely disappear, the paper's reliability conclusion is an artifact of preprocessing rather than a property of the tools; if the differences persist on clean audio, the claim of tool-level unreliability is confirmed.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a mismatch: five speech features extracted from identical audio by OpenSMILE's eGeMAPS set and by Praat via Parselmouth are all significantly different (paired t-test, p < 0.001), with mean differences such as 156.9 Hz for mean pitch and 48.4 syllables per second for speech rate. The speech-rate gap is behaviorally implausible—Praat sometimes reports a mean of 54.12 syllables per second when typical adult speech is 4–6—and the authors attribute Praat's instability to its intensity-based syllable detection being sensitive to diarization noise, silence, and clipped audio. Group comparisons reinforce the point: Praat alone detected gender differences in several tasks and diagnostic differences in speech rate during the Cartoon task, while OpenSMILE did not. When the features are fed to leave-one-user-out random-forest classifiers, performance varies by tool, task, and diagnostic group (e.g., Praat's TD recall in Cartoons is 0.18 vs. ASD 0.77), so no single metric or tool gives a stable picture. The conclusion is that unvalidated default feature extraction can change both the measured behavior and the model's apparent accuracy.

Load-bearing premise

The main load-bearing assumption is that the utterance-level audio produced by automatic speaker diarization is clean enough that the large cross-tool differences reflect genuine tool behavior rather than segmentation artifacts such as silence, clipping, or misattributed turns.

Editorial extensions

If this is right

Studies that use different speech tools on comparable clinical audio cannot be assumed to measure the same construct; cross-study effect sizes may be inflated or obscured by tool differences.
Demographic or diagnostic differences found with a single tool should be treated as provisional until replicated with another tool or a validated reference.
Classification results should be reported per diagnostic group and per task, since aggregate accuracy hides large recall disparities, such as the TD recall of 0.18 versus ASD recall of 0.77 in the Cartoon task with Praat.
Method sections in clinical speech machine learning should state tool version, parameter set, and preprocessing choices as a reproducibility requirement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the cross-tool gap is driven largely by diarization artifacts, then re-running the comparison on manually segmented or forced-aligned audio should shrink the speech-rate gap; this is a testable implication the paper does not pursue.
The same tool-variance problem likely affects other clinical speech domains such as dementia and depression, and other tools such as librosa and ProsodyPro, so a shared benchmark corpus with reference acoustic values would help separate tool noise from behavioral signal.
A pragmatic design rule follows: report both tools or a calibrated third measure when the construct is speech rate, since intensity-based syllable counting is artifact-prone on automatically diarized clinical recordings.
The paper's per-task and per-group performance tables suggest that model selection should optimize for clinically meaningful error balance, such as ASD recall, rather than average F1, because tool choice flips which group is disadvantaged.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look, but with one serious caveat that the reader's report missed. The paper does something the field needs: it takes real autism-assessment audio, runs two standard open-source speech feature extractors on the same utterances, and shows that downstream classifier results differ by task and group. The recommendation—report not just the tool but the exact parameter configuration—is solid and well motivated.

The problem is that the headline claim, that all five features are significantly different between tools, may be an artifact of comparing unlike quantities. The eGeMAPS configuration cited for OpenSMILE outputs pitch as F0semitoneFrom27.5Hz, a semitone scale, while Praat returns Hz. The paper never states which OpenSMILE pitch field was used. The reported mean pitch difference of ~157 Hz is suspiciously close to the offset you would get comparing a semitone value near 36 against a Hz value near 193 for typical adolescent speech. Pitch Std and Pitch Range show the same pattern. If that is what happened, then three of the five features in Table 1 are not measuring the same thing at all, and the paired t-test results for those features are meaningless. The paper also doesn't provide code or configuration files, so the reader can't resolve this from the text.

The other soft spots are more minor and mostly disclosed: the speech-rate outlier is acknowledged as a diarization artifact, the sample is small and gender-imbalanced, and multiple ANOVAs are run without correction. But the unit issue is load-bearing. It doesn't sink the paper's general message—tool choice matters—but it does mean the specific quantitative results in Section 4.1 should not be quoted until the authors clarify their feature definitions and reproduce the comparison with matched units.

Is it worth peer review? Yes, conditionally. The question is important, the paper is honest about its limitations, and the authors are not overclaiming clinical significance. But a serious referee needs to require exact feature names, unit conversion, and ideally a release of the extraction scripts. If the unit mismatch is confirmed, the paper becomes an even more useful cautionary tale about how easy it is to make this mistake. I would not cite the specific numbers in my own work yet, but I might bring it to a reading group as a case study in tool pitfalls.

Referee Report

4 major / 6 minor

Summary. The manuscript compares five speech features (mean F0, F0 standard deviation, F0 range, loudness, and speech rate) extracted with OpenSMILE (eGeMAPSv02) and Praat (Parselmouth) from automatically diarized ADOS-2 recordings of 29 adolescents (14 ASD, 15 TD; 21 male, 8 female) across 14 tasks. Using task-level aggregates, it reports large statistically significant differences between tools for all five features (Table 1), ANOVA-based demographic/tool effects (Table 2), and leave-one-user-out random forest classification of ASD versus TD for each task (Table 3). The paper concludes that open-source speech tools are not interchangeable, that feature-extraction choices affect downstream group-level conclusions and classification performance, and that researchers should report their exact parameter choices.

Significance. If the central comparison were clean, this would be a useful and timely finding: feature extraction is an under-audited link in the clinical speech-ML pipeline, and the paper shows that task, diagnostic group, and tool choice can all shift surface conclusions. The comparison is empirical and external; no fitted parameter enters the headline pairwise differences; and the authors explicitly disclose the diarization/segmentation confound in Section 4.1 and Section 5.1 rather than hiding it. The practical recommendation to report tool versions, configurations, and preprocessing choices is well motivated. However, the value of the paper hinges on whether the headline differences reflect genuine measurement divergence. As written, that is not established for the pitch and loudness features because the OpenSMILE output definitions are not stated, and the speech-rate row is explicitly contaminated by segmentation artifacts. The study is also small (n=29), with multiple-testing concerns in the ANOVA section and no uncertainty quantification in the classification section. The topic deserves attention, but the strong claims need correction and re-analysis before they can stand.

major comments (4)

[Section 3.1 / Section 4.1 / Table 1] The manuscript says only that 'for pitch, we used fundamental frequency (F0)', but it does not state which OpenSMILE eGeMAPS output field was captured. The eGeMAPSv02 configuration exposes pitch as F0semitoneFrom27.5Hz, a semitone-scale value, whereas Praat pitch is in Hz; the mean differences in Table 1 (Mean Pitch: 156.94, Pitch Std: 31.84, Pitch Range: 125.62) are exactly the pattern expected from combining values in these two different units. The same issue likely affects the loudness comparison: eGeMAPS Loudness is a perceptual loudness measure in sone, while the paper's Praat 'loudness' is based on intensity in dB. A paired t-test on raw values in different units is not a test of whether the tools measure the same construct. The authors must specify the exact OpenSMILE features used; if semitone/sone values were used, the comparisons must be redone on a common scale (for example, converting semitone values back to Hz) or re-scoped to the features that genuinely share units. At minimum, a correlation or rank-correlation analysis after unit conversion would show whether the tools agree up to a monotone transformation.
[Section 4.1 / Figure 1 / Table 1] The speech-rate row is not a clean comparison of tool behavior. The paper reports an OpenSMILE mean of 5.78 syllables/second versus a Praat mean of 54.12 syllables/second, which the authors themselves describe as behaviorally impossible and attribute to diarization noise, silence, clipped audio, and misattributed turns. Because this artifact drives the 48.37 mean difference in Table 1, the speech-rate result cannot be cited as evidence of how the tools' speech-rate algorithms differ on valid audio. The authors should redo this part of the analysis after filtering implausible syllable counts or using manually validated utterance boundaries, and should report medians or trimmed means and the number of utterances excluded. The acknowledgment in Section 4.1 is helpful, but it does not cure the fact that the headline Table 1 includes an artifact-dominated value.
[Section 3.2 / Section 4.2 / Table 2] The demographic comparison runs a large number of tests without a stated global correction: five features, fourteen tasks, and two group factors. Table 2 then reports individual p<0.05 effects, several with partial eta-squared values around 0.07 to 0.17. Many of these would not survive a family-wise or false-discovery-rate correction. In addition, the text says 'Bonferroni adjustments' but cites Reference [28], which is the Benjamini-Hochberg false-discovery-rate paper, and the post-hoc columns report significance for one tool without exact p-values for the other. The authors should state the total number of tests, prespecify a correction procedure, apply it consistently, and fix the citation.
[Section 4.3 / Table 3] The statement that 'OpenSMILE generally performed worse than Praat' on F1 is not supported by Table 3: OpenSMILE has equal or higher F1 in many task/group cells (for example, Creating, Cartoons, Demonstration, Telling, Description, Conversation for ASD, Social for TD, Construction for TD, and Joint for both groups). The table reports point estimates only, with no confidence intervals, error bars, or significance tests, and the leave-one-user-out procedure on 29 participants produces noisy per-task estimates. The qualitative conclusion that performance varies by tool, task, and group may be true, but the specific 'generally worse' claim needs summary statistics across tasks and uncertainty quantification before it can be accepted.

minor comments (6)

[Table 1 caption] The caption contains a typo: 'extracted form Praat and OpenSMILE' should be 'extracted from Praat and OpenSMILE'.
[Section 3.1] The conversion factor for OpenSMILE speech rate, 'assuming 1.5 words per segment', is stated without a source or a sensitivity check; it is a constant and cannot by itself create cross-tool differences, but it does affect the reported OpenSMILE speech-rate values and should be justified.
[Reproducibility] There is no code or data availability statement. For a paper whose central message is that parameter choices matter, releasing the exact eGeMAPS configuration, Praat/Parselmouth version, pitch floor/ceiling settings, and diarization parameters would substantially increase reproducibility.
[Section 4.2] The wording 'only present when using Praat (p=0.2801)' is ambiguous; the sentence likely means that the OpenSMILE comparison gave p=0.2801, but both p-values should be written explicitly.
[Table 3] The column headers 'Precision (Praat — OpenSMILE)' do not align clearly with the ASD/TD subcolumns, and the doubling of numbers per cell is confusing; clarify the layout and define which number belongs to which tool.
[Section 4.1 statistical reporting] The paired-sample t-tests are reported only as p<0.001; please report the t-values and degrees of freedom, and clarify whether the tests are computed per task or on the aggregated means used in Table 1.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the cross-tool feature comparison is empirical and externally defined.

full rationale

The paper's central claim—that OpenSMILE and Praat yield significantly different values for nominally identical speech features—is an empirical comparison of two external tools, not a derivation from a fitted parameter. No quantity used in the analysis is defined in terms of the outcome it is used to explain. The speech-rate estimate uses a fixed heuristic conversion (1.5 words per segment, then to syllables per second), but this constant is applied to only one tool and does not by construction determine the magnitude or direction of the cross-tool difference; the reported Praat outliers arise from tool behavior (intensity-based syllable detection) and are disclosed as artifact-sensitive in Section 4.1. The only self-citation ([2], Chowdhury, Romero and Stent, Interspeech 2023) appears as background for prior autism-classification work and in Future Work about clinician speech; it is not used to justify the reliability comparison or to forbid alternatives. Potential unit mismatches (e.g., OpenSMILE eGeMAPS F0 in semitones versus Praat pitch in Hz) would be a correctness or validity risk rather than circularity, since the paper does not construct the difference from an input relation. The disclosed limitations (automatic diarization, English-only data, gender imbalance) concern internal validity, not circularity. No prediction in the paper reduces by construction to its inputs, so no circular step is identified.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central comparison relies on four domain assumptions: automatic diarization yields usable utterances, the five features are commensurable across tools, default classifier settings are adequate, and the Praat speech-rate outlier is an artifact. One heuristic constant, 1.5 words per voiced segment, enters the OpenSMILE speech-rate measure. No new entities are introduced.

free parameters (1)

OpenSMILE speech-rate conversion factor (words per voiced segment) = 1.5
Section 3.1: OpenSMILE speech rate is estimated as voiced segments per second, multiplied by an assumed 1.5 words per segment, then converted to syllables per second. The constant comes from prior work [22] and is not validated for this adolescent sample.

assumptions (4)

domain assumption Automatic diarization output is sufficiently accurate for utterance-level feature extraction.
Section 2 relies on pyannote and PyDub to segment participant speech; Section 4.1 acknowledges that diarization noise can interfere with speech-rate estimates, so the validity of the comparison depends on this assumption.
domain assumption The five features are intended to measure the same constructs in both tools.
Section 3.1 defines comparable features, pitch mean, pitch standard deviation, pitch range, loudness, and speech rate, but the tools use different algorithms; the cross-tool comparison assumes commensurability.
domain assumption Random forest with scikit-learn default hyperparameters is adequate for the classification comparison.
Section 3.3 deliberately uses default settings; this makes the model reproducible in principle but assumes defaults are sensible for 29 participants and five features per task.
domain assumption Praat's speech-rate outlier mainly reflects artifact sensitivity rather than true speech behavior.
Section 4.1 says a mean of 54 syllables per second is behaviorally impossible and attributes it to segmentation artifacts; this interpretation is asserted, not tested with manual diarization.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Can We Trust Machine Learning? The Reliability of Features from Open-Source Speech Analysis Tools for Speech Modeling." pith.science (2026). https://pith.science/paper/AGPJOTJX

@misc{pith2026250611072,
  author       = {Pith},
  title        = {Pith review of: Can We Trust Machine Learning? The Reliability of Features from Open-Source Speech Analysis Tools for Speech Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AGPJOTJX}},
  note         = {Machine review of arXiv:2506.11072}
}

read the original abstract

Machine learning-based behavioral models rely on features extracted from audio-visual recordings. The recordings are processed using open-source tools to extract speech features for classification models. These tools often lack validation to ensure reliability in capturing behaviorally relevant information. This gap raises concerns about reproducibility and fairness across diverse populations and contexts. Speech processing tools, when used outside of their design context, can fail to capture behavioral variations equitably and can then contribute to bias. We evaluate speech features extracted from two widely used speech analysis tools, OpenSMILE and Praat, to assess their reliability when considering adolescents with autism. We observed considerable variation in features across tools, which influenced model performance across context and demographic groups. We encourage domain-relevant verification to enhance the reliability of machine learning models in clinical applications.

Figures

Figures reproduced from arXiv: 2506.11072 by the authors.

**Figure 1.** Mean speech rate measured by OpenSMILE and Praat. Note that average adult speech rate is between 4 and 6 syllables/second. 4.3. Do classification results vary based on the tools used? To investigate how tools and features extracted by them influence model performance in classifying diagnostic labels, we compared the performance for both tools by the two diagnostic groups separately for each of the 14 tasks (in [P… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 37 canonical work pages

[28]

Syllables per word in typical and delayed speech acquisition,

P. Flipsen Jr, “Syllables per word in typical and delayed speech acquisition,”Clinical linguistics & phonetics, vol. 20, no. 4, pp. 293–301, 2006

work page 2006
[1]

Introduction Clinical conditions that manifest neurological, cognitive, and behavioral symptoms are typically diagnosed with standardized tools. Consider Autism Spectrum Disorder, which refers to a set of developmental conditions that restrict an individual’s ability for social communication and interaction, among other things. Currently, the clinical mea...

work page Pith review arXiv 2025
[2]

Data We utilized data collected during sessions of the Autism Di- agnostic Observation Schedule - Second Edition (ADOS-2), a standardized assessment tool designed to evaluate ASD-related impairments [23]. In this evaluation, an adolescent (‘Partici- pant’) interacts with a certified adult assessor through a series of semi-structured activities that assess...

work page
[3]

Method We first extracted speech features from the audio recordings of the participants using the two speech processing tools consid- ered here:OpenSMILE[9] andPraat[16] to compare them across demographic groups and their performance in machine learning classification models. 3.1. Features For OpenSMILE, we used the Geneva Minimalistic Acoustic Parameter ...

work page
[4]

Results 4.1. Are features measured by OpenSMILE and Praat the same way? To answer whether the two tools under consideration, OpenS- MILE and Praat, provide the same measure of each feature, we calculated the difference in features measured by the two tools for each participant by task. Since the difference in feature val- ues for various tasks was negligi...

work page
[5]

Researchers have delved into the use of such tools with sometimes minimal understanding

Discussion The current project is a first step toward better understanding the measures provided by open-source speech analysis tools and how they are affecting our understanding of mental health dis- orders such as Autism. Researchers have delved into the use of such tools with sometimes minimal understanding. For ex- ample, the two tools described have ...

work page
[6]

Future Work We plan to first deal with the limitations of our approach here and delve into more careful and informed setting of parame- ters when using open-source tools. Next, we acknowledge that conversation is an interactive activity, and plan to also analyze the behaviors of the psychologist administering the diagnostic, as previous research has found...

work page
[7]

V ocal markers of autism: Assessing the generalizability of machine learning models,

A. Rybner, E. T. Jessen, M. D. Mortensen, S. N. Larsen, R. Grossman, N. Bilenberg, C. Cantio, J. R. M. Jepsen, E. Weed, A. Simonsen, and R. Fusaroli, “V ocal markers of autism: Assessing the generalizability of machine learning models,”Autism Research, vol. 15, no. 6, pp. 1018–1030, 2022. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10. 10...

work page 2022

Show all 39 references

[8]

Parameter selection for analyzing conversations with autism spectrum disorder,

T. Chowdhury, V . Romero, and A. Stent, “Parameter selection for analyzing conversations with autism spectrum disorder,” inInter- speech 2023, 2023, pp. 2643–2647

2023
[9]

Classifying autism from crowd- sourced semistructured speech recordings: machine learning model comparison study,

N. A. Chi, P. Washington, A. Kline, A. Husic, C. Hou, C. He, K. Dunlap, and D. P. Wall, “Classifying autism from crowd- sourced semistructured speech recordings: machine learning model comparison study,”JMIR pediatrics and parenting, vol. 5, no. 2, p. e35406, 2022

2022
[10]

V oice acoustics allow classifying autism spectrum disorder with high accuracy,

F. Briend, C. David, S. Silleresi, J. Malvy, S. Ferr ´e, and M. Lati- nus, “V oice acoustics allow classifying autism spectrum disorder with high accuracy,”Translational Psychiatry, vol. 13, no. 1, p. 250, 2023

2023
[11]

Audio-visual recognition of emotional engagement of people with dementia

L. Steinert, F. Putze, D. K ¨uster, and T. Schultz, “Audio-visual recognition of emotional engagement of people with dementia.” inInterspeech, 2021, pp. 1024–1028

2021
[12]

Cross-lingual features for alzheimer’s dementia detection from speech,

T. Melistas, L. Kapelonis, N. Antoniou, P. Mitseas, D. Sgouropou- los, T. Giannakopoulos, A. Katsamanis, and S. Narayanan, “Cross-lingual features for alzheimer’s dementia detection from speech,” inInterspeech 2023, 2023, pp. 3008–3012

2023
[13]

Repeatability of commonly used speech and lan- guage features for clinical applications,

G. M. Stegmann, S. Hahn, J. Liss, J. Shefner, S. B. Rutkove, K. Kawabata, S. Bhandari, K. Shelton, C. J. Duncan, and V . Berisha, “Repeatability of commonly used speech and lan- guage features for clinical applications,”Digital biomarkers, vol. 4, no. 3, pp. 109–122, 2020

2020
[14]

Are reported accuracies in the clinical speech machine learning literature overoptimistic?

V . Berisha, C. Krantsevich, G. Stegmann, S. Hahn, and J. Liss, “Are reported accuracies in the clinical speech machine learning literature overoptimistic?” inInterspeech 2022, 2022, pp. 2453– 2457

2022
[15]

Opensmile: The mu- nich versatile and fast open-source audio feature extractor,

F. Eyben, M. W ¨ollmer, and B. Schuller, “Opensmile: The mu- nich versatile and fast open-source audio feature extractor,” in Proceedings of the 18th ACM International Conference on Mul- timedia. New York, NY , USA: Association for Computing Ma- chinery, 2010, p. 1459–1462

2010
[16]

Analyzing short term dynamic speech fea- tures for understanding behavioral traits of children with autism spectrum disorder

Y .-K. Kim, R. Lahiri, M. Nasir, S. H. Kim, S. Bishop, C. Lord, and S. S. Narayanan, “Analyzing short term dynamic speech fea- tures for understanding behavioral traits of children with autism spectrum disorder.” inInterspeech, 2021, pp. 2916–2920

2021
[17]

Automati- cally predicting perceived conversation quality in a pediatric sam- ple enriched for autism,

Y . Yang, S. Cho, M. Covello, A. Knox, O. Bastani, J. Weimer, E. Dobriban, R. Schultz, I. Lee, and J. Parish-Morris, “Automati- cally predicting perceived conversation quality in a pediatric sam- ple enriched for autism,” inInterspeech 2023, 2023, pp. 4603– 4607

2023
[18]

Acoustic changes in speech prosody produced by children with autism after robot- assisted speech training,

S. Chen, B. X. Wang, Y . Hong, F. Zhou, A. Chan, P. yi Tang, B. Li, C. Wen, J. Cheung, Y . Liu, and Z. Chen, “Acoustic changes in speech prosody produced by children with autism after robot- assisted speech training,” inInterspeech 2024, 2024, pp. 2480– 2484

2024
[19]

Is voice a marker for autism spectrum disorder? a system- atic review and meta-analysis,

R. Fusaroli, A. Lambrechts, D. Bang, D. M. Bowler, and S. B. Gaigg, “Is voice a marker for autism spectrum disorder? a system- atic review and meta-analysis,”Autism Research: Official Journal of the International Society for Autism Research, vol. 10, no. 3, pp. 384–407, 2017

2017
[20]

Leakage and the reproducibility crisis in machine-learning-based science,

S. Kapoor and A. Narayanan, “Leakage and the reproducibility crisis in machine-learning-based science,”Patterns, vol. 4, no. 9, 2023

2023
[21]

Toward a cumulative science of vocal markers of autism: A cross-linguistic meta-analysis-based inves- tigation of acoustic markers in American and Danish autistic chil- dren,

R. Fusaroli, R. Grossman, N. Bilenberg, C. Cantio, J. R. M. Jepsen, and E. Weed, “Toward a cumulative science of vocal markers of autism: A cross-linguistic meta-analysis-based inves- tigation of acoustic markers in American and Danish autistic chil- dren,”Autism Research: Off...

2022
[22]

Praat: doing phonetics by computer,

“Praat: doing phonetics by computer,”http://www. praat. org/, 2007

2007
[23]

Quantification of speech and syn- chrony in the conversation of adults with autism spectrum disor- der,

K. Ochi, N. Ono, K. Owada, M. Kojima, M. Kuroda, S. Sagayama, and H. Yamasue, “Quantification of speech and syn- chrony in the conversation of adults with autism spectrum disor- der,”PloS one, vol. 14, no. 12, p. e0225377, 2019

2019
[24]

Quantitative analysis of pitch in speech of children with neu- rodevelopmental disorders,

G. Kiss, J. P. v. Santen, E. Prud’Hommeaux, and L. M. Black, “Quantitative analysis of pitch in speech of children with neu- rodevelopmental disorders,” inProceedings of INTERSPEECH, 2012

2012
[25]

Acoustic-prosodic correlates of ‘awkward’prosody in story retellings from adolescents with autism,

D. Bone, M. P. Black, A. Ramakrishna, R. B. Grossman, and S. S. Narayanan, “Acoustic-prosodic correlates of ‘awkward’prosody in story retellings from adolescents with autism,” inProceedings of INTERSPEECH, 2015, pp. 1616–1620

2015
[26]

Analysis of en- gagement behavior in children during dyadic interactions using prosodic cues,

R. Gupta, D. Bone, S. Lee, and S. Narayanan, “Analysis of en- gagement behavior in children during dyadic interactions using prosodic cues,”Comput. Speech Lang., vol. 37, no. C, p. 47–66, may 2016

2016
[27]

Abnormal speech spectrum and increased pitch vari- ability in young autistic children,

Y . S. Bonneh, Y . Levanon, O. Dean-Pardo, L. Lossos, and Y . Adini, “Abnormal speech spectrum and increased pitch vari- ability in young autistic children,”Frontiers in human neuro- science, vol. 4, p. 237, 2011

2011
[29]

The Autism diagnostic ob- servation schedule—generic: A standard measure of social and communication deficits associated with the spectrum of Autism,

C. Lord, S. Risi, L. Lambrecht, E. H. Cook, B. L. Leventhal, P. C. DiLavore, A. Pickles, and M. Rutter, “The Autism diagnostic ob- servation schedule—generic: A standard measure of social and communication deficits associated with the spectrum of Autism,” Journal of Autism and...

2000
[30]

pyannote.audio: neural building blocks for speaker diarization,

H. Bredin, R. Yin, J. M. Coria, G. Gelly, P. Korshunov, M. Lavechin, D. Fustes, H. Titeux, W. Bouaziz, and M.-P. Gill, “pyannote.audio: neural building blocks for speaker diarization,” inProceedings of the IEEE International Conference on Acous- tics, Speech, and Signal Proces...

2020
[31]

[Online]

(2022) Pydub. [Online]. Available: https://github.com/jiaaro/ pydub

2022
[32]

The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing,

F. Eyben, K. R. Scherer, B. W. Schuller, J. Sundberg, E. Andr ´e, C. Busso, L. Y . Devillers, J. Epps, P. Laukka, S. S. Narayanan, and K. P. Truong, “The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing,”IEEE Transactions on Affect...

2016
[33]

Introducing Parsel- mouth: A Python interface to Praat,

Y . Jadoul, B. Thompson, and B. de Boer, “Introducing Parsel- mouth: A Python interface to Praat,”Journal of Phonetics, vol. 71, pp. 1–15, 2018

2018
[34]

Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing,

Y . Benjamini and Y . Hochberg, “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing,” Journal of the Royal Statistical Society: Series B (Methodologi- cal), vol. 57, no. 1, pp. 289–300, 12 2018

2018
[35]

Why do we need sex-balanced studies of autism?

C. W. Nordahl, “Why do we need sex-balanced studies of autism?”Autism Research, vol. 16, no. 9, pp. 1662–1669, 2023

2023
[36]

librosa: Audio and music signal analysis in python,

B. McFee, C. Raffel, D. Liang, D. P. Ellis, M. McVicar, E. Batten- berg, and O. Nieto, “librosa: Audio and music signal analysis in python,” inProceedings of the 14th python in science conference, vol. 8, 2015

2015
[37]

Prosodypro—a tool for large-scale systematic prosody analysis,

Y . Xu, “Prosodypro—a tool for large-scale systematic prosody analysis,”TRASP 2013, p. 7, 2013

2013
[38]

Hearing me hear- ing you: Reciprocal effects between child and parent language in autism and typical development,

R. Fusaroli, E. Weed, D. Fein, and L. Naigles, “Hearing me hear- ing you: Reciprocal effects between child and parent language in autism and typical development,”Cognition, vol. 183, pp. 1–18, Feb. 2019

2019
[39]

Interpersonal synchrony across vocal and lexical modalities in interactions involving children with autism spectrum disorder,

R. Lahiri, M. Nasir, M. Kumar, S. H. Kim, S. Bishop, C. Lord, and S. Narayanan, “Interpersonal synchrony across vocal and lexical modalities in interactions involving children with autism spectrum disorder,”JASA express letters, vol. 2, no. 9, 2022

2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [28]

Syllables per word in typical and delayed speech acquisition,

P. Flipsen Jr, “Syllables per word in typical and delayed speech acquisition,”Clinical linguistics & phonetics, vol. 20, no. 4, pp. 293–301, 2006

work page 2006

[2] [1]

Introduction Clinical conditions that manifest neurological, cognitive, and behavioral symptoms are typically diagnosed with standardized tools. Consider Autism Spectrum Disorder, which refers to a set of developmental conditions that restrict an individual’s ability for social communication and interaction, among other things. Currently, the clinical mea...

work page Pith review arXiv 2025

[3] [2]

Data We utilized data collected during sessions of the Autism Di- agnostic Observation Schedule - Second Edition (ADOS-2), a standardized assessment tool designed to evaluate ASD-related impairments [23]. In this evaluation, an adolescent (‘Partici- pant’) interacts with a certified adult assessor through a series of semi-structured activities that assess...

work page

[4] [3]

Method We first extracted speech features from the audio recordings of the participants using the two speech processing tools consid- ered here:OpenSMILE[9] andPraat[16] to compare them across demographic groups and their performance in machine learning classification models. 3.1. Features For OpenSMILE, we used the Geneva Minimalistic Acoustic Parameter ...

work page

[5] [4]

Results 4.1. Are features measured by OpenSMILE and Praat the same way? To answer whether the two tools under consideration, OpenS- MILE and Praat, provide the same measure of each feature, we calculated the difference in features measured by the two tools for each participant by task. Since the difference in feature val- ues for various tasks was negligi...

work page

[6] [5]

Researchers have delved into the use of such tools with sometimes minimal understanding

Discussion The current project is a first step toward better understanding the measures provided by open-source speech analysis tools and how they are affecting our understanding of mental health dis- orders such as Autism. Researchers have delved into the use of such tools with sometimes minimal understanding. For ex- ample, the two tools described have ...

work page

[7] [6]

Future Work We plan to first deal with the limitations of our approach here and delve into more careful and informed setting of parame- ters when using open-source tools. Next, we acknowledge that conversation is an interactive activity, and plan to also analyze the behaviors of the psychologist administering the diagnostic, as previous research has found...

work page

[8] [7]

V ocal markers of autism: Assessing the generalizability of machine learning models,

A. Rybner, E. T. Jessen, M. D. Mortensen, S. N. Larsen, R. Grossman, N. Bilenberg, C. Cantio, J. R. M. Jepsen, E. Weed, A. Simonsen, and R. Fusaroli, “V ocal markers of autism: Assessing the generalizability of machine learning models,”Autism Research, vol. 15, no. 6, pp. 1018–1030, 2022. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10. 10...

work page 2022