The efficacy of remote video-based behavioral skills training and awareness training on public speaking performance.

This study evaluated the efficacy of (a) remote video-based behavioral skills training (BST) with added speech outlines on teaching public speaking behaviors and (b) remote video-based awareness training (AT) on speech-disfluency rates. A multiple-baseline design across speech behaviors was used to evaluate the training. Remote video-based BST and AT were effective at teaching public speaking behaviors and reducing speech disfluencies, respectively, for both participants. In addition, performance generalized to increased audience size. Although expert ratings of perceived public speaking effectiveness improved following BST, the ratings did not improve and some worsened following AT. Both participants reported satisfaction with video-based BST and AT. One participant reported greater comfort, confidence, overall ability, and less anxiety as a public speaker following BST. Both participants reported greater improvements in those categories following AT. Our results suggest that public speaking behaviors can be taught using remote video-based BST and speech disfluencies can be reduced using remote video-based AT.

The efficacy of remote video-based training on public speaking.

This study evaluated the efficacy of remote video-based behavioral skills training (BST) on teaching public speaking behaviors to 4 participants. A multiple-baseline design across speech behaviors was used to evaluate the effects of the training. Remote video-based BST was effective at increasing public speaking behaviors for all participants. In addition, performance generalized to an increased audience size. An external expert in communications rated the participants as more effective public speakers following training. All participants reported satisfaction with the training and expressed greater comfort, confidence, overall ability, and less anxiety as a public speaker following training. We also measured potential collateral effects of teaching public speaking behaviors on speech disfluencies. Although remote video-based BST was effective for all participants, it did not produce a change in the rate of speech disfluencies. Our findings indicate that public speaking behaviors can be taught using a remote video-based BST package.

Deriving gradient measures of child speech from crowdsourced ratings.

Recent research has demonstrated that perceptual ratings aggregated across multiple non-expert listeners can reveal gradient degrees of contrast between sounds that listeners might transcribe identically. Aggregated ratings have been found to correlate strongly with acoustic gold standard measures both when individual raters use a continuous rating scale such as visual analog scaling (Munson et al., 2012) and when individual raters provide binary ratings (McAllister Byun, Halpin, & Szeredi, 2015). In light of evidence that inexperienced listeners use continuous scales less consistently than experienced listeners, this study investigated the relative merits of binary versus continuous rating scales when aggregating responses over large numbers of naive listeners recruited through online crowdsourcing. Stimuli were words produced by children in treatment for misarticulation of North American English /r/. Each listener rated the same 40 tokens two times: once using Visual Analog Scaling (VAS) and once using a binary rating scale. The gradient rhoticity of each item was then estimated using (a) VAS click location, averaged across raters; (b) the proportion of raters who assigned the "correct /r/" label to each item in the binary rating task (pˆ). First, we validate these two measures of rhoticity against each other and against an acoustic gold standard. Second, we explore the range of variability in individual response patterns that underlie these group-level data. Third, we integrate statistical, theoretical, and practical considerations to offer guidelines for determining which measure to use in a given situation.

Online crowdsourcing for efficient rating of speech: a validation study.

UNLABELLED: Blinded listener ratings are essential for valid assessment of interventions for speech disorders, but collecting these ratings can be time-intensive and costly. This study evaluated the validity of speech ratings obtained through online crowdsourcing, a potentially more efficient approach. 100 words from children with /r/ misarticulation were electronically presented for binary rating by 35 phonetically trained listeners and 205 naïve listeners recruited through the Amazon Mechanical Turk (AMT) crowdsourcing platform. Bootstrapping was used to compare different-sized samples of AMT listeners against a "gold standard" (mode across all trained listeners) and an "industry standard" (mode across bootstrapped samples of three trained listeners). There was strong overall agreement between trained and AMT listeners. The "industry standard" level of performance was matched by bootstrapped samples with n = 9 AMT listeners. These results support the hypothesis that valid ratings of speech data can be obtained in an efficient manner through AMT. Researchers in communication disorders could benefit from increased awareness of this method. LEARNING OUTCOMES: Readers will be able to (a) discuss advantages and disadvantages of data collection through the crowdsourcing platform Amazon Mechanical Turk (AMT), (b) describe the results of a validity study comparing samples of AMT listeners versus phonetically trained listeners in a speech-rating task.