Finding the experts in the crowd: Validity and reliability of crowdsourced measures of children's gradient speech contrasts.

Perceptual ratings aggregated across multiple nonexpert listeners can be used to measure covert contrast in child speech. Online crowdsourcing provides access to a large pool of raters, but for practical purposes, researchers may wish to use smaller samples. The ratings obtained from these smaller samples may not maintain the high levels of validity seen in larger samples. This study aims to measure the validity and reliability of crowdsourced continuous ratings of child speech, obtained through Visual Analog Scaling, and to identify ways to improve these measurements. We first assess overall validity and interrater reliability for measurements obtained from a large set of raters. Second, we investigate two rater-level measures of quality, individual validity and intrarater reliability, and examine the relationship between them. Third, we show that these estimates may be used to establish guidelines for the inclusion of raters, thus impacting the quality of results obtained when smaller samples are used.

Bias in the perception of phonetic detail in children's speech: A comparison of categorical and continuous rating scales.

Previous research has shown that continuous rating scales can be used to assess phonetic detail in children's productions, and could potentially be used to detect covert contrasts. Two experiments examined whether continuous rating scales have the additional benefit of being less susceptible to task-related biasing than categorical phonetic transcriptions. In both experiments, judgements of children's productions of /s/ and /θ/ were interleaved with two types of rating tasks designed to induce bias: continuous judgements of a parameter whose variation is itself relatively more continuous (gender typicality of their speech) in one biasing condition, and categorical judgements of a parameter that is relatively less continuous (the vowel they produced) in the other biasing condition. One experiment elicited continuous judgements of /s/ and /θ/ productions, while the other elicited categorical judgements. The results of Experiment 1 showed that the influence of acoustic characteristics on continuous judgements of /s/ and /θ/ was stable across biasing conditions. In contrast, the results of Experiment 2 showed that the influence of acoustic characteristics on categorical judgements of /s/ and /θ/differed systematically across biasing conditions. These results suggest that continuous judgements are psychometrically superior to categorical judgements, as they are more resistant to task-related bias.

Using ultrasound tongue imaging to identify covert contrasts in children's speech.

Ultrasound tongue imaging has become a promising technique for detecting covert contrasts, due to the developments in data analysis methods that allow for processing information on tongue shape from young children. An important feature concerning analyses of ultrasound data from children who are likely to produce covert contrasts is that the data are likely to be collected without head-to-transducer stabilisation, due to the speakers' age. This article is a review of the existing methods applicable in analysing data from non-stabilised recordings. The article describes some of the challenges of ultrasound data collection from children, and analysing these data, as well as possible ways to address those challenges. Additionally, there are examples from typical and disordered productions featuring covert contrasts, with illustrations of quantifying differences in tongue shape between target speech sounds.

Covert contrast in velar fronting: An acoustic and ultrasound study.

There is growing evidence that speech sound acquisition is a gradual process, with instrumental measures frequently revealing covert contrast in errors perceived to involve phonemic substitution. Ultrasound imaging has the potential to expand our understanding of covert contrast by showing whether a child uses different tongue shapes while producing sounds that are perceived as neutralised. This study used an ultrasound measure (Dorsum Excursion Index) and acoustic measures (VOT and spectral moments of the burst) to investigate overt and covert contrast between velar and alveolar stops in child speech. Participants were two children who produced a perceptually overt velar-alveolar contrast and two children who neutralised the contrast via velar fronting. Both acoustic and ultrasound measures revealed significant differences between perceptually distinct velar and alveolar targets. One child with velar fronting demonstrated covert contrast in one acoustic and one ultrasound measure; the other showed no evidence of contrast. Clinical implications are discussed in this article.