Strengths and challenges of the COSMIN tools in outcome measures appraisal: A case example for speech-language therapy.

BACKGROUND: The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) is an international initiative that offers standardized and validated tools to guide the appraisal of patient-reported outcome measures in healthcare. AIMS: To explore the use of a new set of tools from the COSMIN to appraise studies on one outcome measure available to speech and language therapists (SLTs). METHODS & PROCEDURES: We used the COSMIN tools to appraise seven studies and a user manual that reported the measurement properties of the Focus on the Outcomes of Children Under Six (FOCUS), a validated measure of pre-schoolers' communicative participation that is used in various contexts around the world. OUTCOMES & RESULTS: Using COSMIN guidelines, the FOCUS was categorized as a 'category A' tool because there was a sufficient level of evidence to support its content validity and internal consistency. According to the COSMIN guidelines, this means that the FOCUS can be recommended for use. The quality of evidence supporting measurement properties of the FOCUS received a rating of 'moderate', meaning users can have moderate confidence in its measurement properties. Since these ratings from the COSMIN tools may be unclear to users of the FOCUS, we have provided more specific recommendations. CONCLUSIONS & IMPLICATIONS: The COSMIN tools offer detailed standards to support the appraisal of outcome measures available to SLTs. However, several limitations were observed, and recommendations to support the application of the COSMIN tools are provided. What this paper adds What is already known on the subject Collecting outcome data is essential to ensure speech and language therapy is effective. Until the development of COSMIN there was a lack of standards in the way the measurement properties of outcome measure instruments were appraised. What this paper adds to existing knowledge This paper used the FOCUS, a measure of pre-schoolers' communicative participation outcomes in speech and language therapy, as a case example to illustrate the applications of the COSMIN tools. In doing so, the strengths and limitations of the current COSMIN tools in appraising the quality of outcome measure instruments are emphasized. What are the potential or actual clinical implications of this work? The COSMIN tools offer a step-by-step, standardized approach to appraise various measurement properties in outcome instruments. Due to existing limitations of the COSMIN tools, appraisal should provide clear and specific recommendations so users of outcome measures (e.g., SLTs, researchers) can identify the appropriate uses of each instrument.

Assessing communicative participation in preschool children with the Focus on the Outcomes of Communication Under Six: a scoping review.

AIM: To describe uses of the Focus on the Outcomes of Communication Under Six (FOCUS) in research with children with and without various communication disorders since its publication in 2010. METHOD: Six databases were searched for the term 'Focus on the Outcomes of Communication Under Six'. With additional searches we ascertained 70 articles, of which 25 met inclusion criteria for full review and data extraction. RESULTS: The FOCUS has been used in research across multiple countries, purposes, populations, contexts, and versions. Evaluative studies have described: the development of children's communicative participation skills and factors that impact the development of communicative participation; the impact of specific interventions on communicative participation; how FOCUS captures change relative to measures of impairment; and how FOCUS performs when used at different intervals. Adaptations have included: use of the FOCUS as a descriptive or discriminative tool; use with children outside the validated age range; use of select items; and use with typically developing children. INTERPRETATION: The FOCUS is used worldwide in research and practice, and much has been learned about children's communicative participation. Future research is needed to explore the relationship between children's impairments and their communicative participation, develop a FOCUS App, and develop and validate a FOCUS for school-age children.

Validity of a streamlined version of the Focus on the Outcomes of Communication Under Six: Process and outcome.

BACKGROUND: The Focus on the Outcomes of Communication Under Six (FOCUS) is a 50-item outcome measure based on the framework and concepts of World Health Organization's International Classification of Functioning. The FOCUS has been shown to capture participation-level changes associated with speech and language therapy in children 18 to 72 months old and has established validity and reliability. However, there were reasons to try to reduce the measure without losing any of its proven value as a change-detecting instrument. METHODS: Using data from 18,931 preschool children, we used item response models and a test of redundancy to reduce the original 50-item FOCUS to 34 items. We then assessed the correlations between FOCUS-50 and FOCUS-34 scores on these children. RESULTS: Findings show that a shortened (34-item) version of FOCUS can be treated as equivalent to the full 50-item version. CONCLUSION: The correlation between change scores on the two versions is .98. We suggest that in situations where only a total score is of interest, the FOCUS-34 can be used in place of the full FOCUS.

Factors Contributing to Preschoolers' Communicative Participation Outcomes: Findings From a Population-Based Longitudinal Cohort Study in Ontario, Canada.

Purpose: The aim of this study was to identify predictors of communicative participation outcomes for a large cohort of preschoolers with speech and language impairments. Method: A secondary analysis of longitudinal program evaluation data from Ontario, Canada's Preschool Speech and Language Program was done. Data available for 46,872 children 18-67 months of age (M = 41.76 months, SD = 11.92; 68% boys, 32% girls) were previously used to predict children's communicative participation skill development in 5 levels of function. Demographic and intervention-based variables were added to the models to identify new predictors of growth. Results: Three demographic and 3 intervention-based variables were statistically significant predictors of children's communicative participation outcomes. Clinically significant predictors included participation in an early learning environment, receipt of speech-language interventions, and the amount of time spent in intervention. These variables impacted predicted outcomes differently, depending on a child's level of communicative function. Conclusions: This population-based study of preschoolers with speech and language impairments identified predictors of growth in communicative participation skills-an outcome important and meaningful to families but not often explored. A broad picture emerged of factors that may influence the development of communicative participation skills and may be used to predict outcomes for preschoolers. Given the large sample size, these robust findings may be used to predict outcomes outside the Preschool Speech and Language Program as well. Supplemental Material: https://doi.org/10.23641/asha.6024422.

Moving research tools into practice: the successes and challenges in promoting uptake of classification tools.

PURPOSE: In this paper, we present our experiences - both successes and challenges - in implementing evidence-based classification tools into clinical practice. We also make recommendations for others wanting to promote the uptake and application of new research-based assessment tools. METHOD: We first describe classification systems and the benefits of using them in both research and practice. We then present a theoretical framework from Implementation Science to report strategies we have used to implement two research-based classification tools into practice. We also illustrate some of the challenges we have encountered by reporting results from an online survey investigating 58 Speech-language Pathologists' knowledge and use of the Communication Function Classification System (CFCS), a new tool to classify children's functional communication skills. RESULT AND CONCLUSIONS: We offer recommendations for researchers wanting to promote the uptake of new tools in clinical practice. Specifically, we identify structural, organizational, innovation, practitioner, and patient-related factors that we recommend researchers address in the design of implementation interventions. Roles and responsibilities of both researchers and clinicians in making implementations science a success are presented. Implications for rehabilitation Promoting uptake of new and evidence-based tools into clinical practice is challenging. Implementation science can help researchers to close the knowledge-to-practice gap. Using concrete examples, we discuss our experiences in implementing evidence-based classification tools into practice within a theoretical framework. Recommendations are provided for researchers wanting to implement new tools in clinical practice. Implications for researchers and clinicians are presented.

A population-based study of communicative participation in preschool children with speech-language impairments.

AIM: To develop statistical models of communicative participation development of preschool children and explore variations by level of function. METHOD: This was a secondary analysis of data from a longitudinal study of preschool children with speech and language impairments (n=46 872; age range 18-67mo, mean age [SD] 41.76mo [11.92]; 67% male) accessing publicly funded services in Ontario, Canada. Two measures were used: Focus on the Outcomes of Communication Under Six (FOCUS), measuring changes in communicative participation skills, and the Communication Function Classification System (CFCS), classifying communicative function into one of five levels. We used mixed effects modeling to fit growth curves for children in each CFCS level. Models allowed for variation in initial FOCUS score at 18 months, rate of growth with age, and rate of acceleration/deceleration with age. RESULTS: Starting FOCUS score (18mo) varied inversely with CFCS level at entry to the program. Growth was initially rapid and then leveled off for children in Levels I to III. Growth was less rapid for children in Level IV, but leveled off, and was slow but continual for children in Level V. INTERPRETATION: This work can help us to move beyond traditional impairment-based thinking and shows that children can make meaningful communicative changes regardless of their function.

Validity of the Communication Function Classification System for use with preschool children with communication disorders.

AIM: To evaluate construct and predictive validity of the Communication Function Classification System (CFCS) for use with preschool children with a range of speech and language disorders. METHOD: Seventy-seven preschool children with speech and language disorders (50 males, 27 females; mean 2y 7mo, standard deviation [SD] 1y) participated in this cohort study. Preschool children had speech and language, language-only, or speech-only disorders. Together with parent input, speech-language pathologists (SLPs) completed the CFCS at time 1. Parents and SLPs then independently completed a validated change-detecting functional communication outcome measure, the Focus on the outcomes of Communication Under Six (FOCUS), three times: at assessment (time 1), at the start of treatment (time 2), and at the end of treatment (time 3). RESULTS: There was a significant negative correlation between CFCS classifications and FOCUS scores at all three measurement points for the ratings by both parents and SLPs (correlations ranged from -0.60 to -0.76). As expected, no correlations between CFCS classifications and FOCUS change scores were statistically significant. INTERPRETATION: This study provides evidence of construct and predictive validity of the CFCS, demonstrating its value as a discriminative tool for use with preschool children with a range of speech and language disorders.

Communicative participation changes in pre-school children receiving augmentative and alternative communication intervention.

PURPOSE: This paper reports changes in communicative participation skills-systematically measured and described-in an empirical observational case series of eight children receiving augmentative and alternative communication (AAC) interventions. METHOD: The eight children (seven boys, one girl), ranging from 1 year 4 months to 4 years 11 months (mean = 2.8 years; SD = 1.32 years) received varied AAC interventions (i.e. sign language, assistive technology, PECS), averaging 15 hours of treatment over a 12-month period. Parents completed an outcome measure (FOCUS) three times: at the start, mid-point (6 months) and end of the intervention period (after 12 months). They also completed the ASQ-SE at the start and end of intervention. RESULT: FOCUS scores increased over the treatment interval, indicating improvement in real-world communication skills as observed by their parents. The ASQ-SE items that pertained to communication also improved, while the items that did not correspond to communication did not. This divergence suggests that the communicative participation improvements resulted from treatment rather than general developmental gains. The largest improvements were noted in receptive language/listening, pragmatics and social/play skills. Improvements in intelligibility were also measured for several children. CONCLUSION: These results suggest that AAC intervention facilitated improvements in communicative participation skills in pre-school children.

Validation of the Focus on the Outcomes of Communication under Six outcome measure.

AIM: The aim of this study was to establish the construct validity of the Focus on the Outcomes of Communication Under Six (FOCUS(©) ),a tool designed to measure changes in communication skills in preschool children. METHOD: Participating families' children (n=97; 68 males, 29 females; mean age 2y 8mo; SD 1.04y, range 10mo-4y 11mo) were recruited through eight Canadian organizations. The children were on a waiting list for speech and language intervention. Parents completed the Ages and Stages Questionnaire - Social/Emotional (ASQ-SE) and the FOCUS three times: at assessment and at the start and end of treatment. A second sample (n=28; 16 males 12 females) was recruited from another organization to correlate the FOCUS scores with speech, intelligibility and language measures. Second sample participants ranged in age from 3 years 1 month to 4 years 9 months (mean 3y 11mo; SD 0.41y). At the start and end of treatment, children were videotaped to obtain speech and language samples. Parents and speech-language pathologists (SLPs) independently completed the FOCUS tool. SLPs who were blind to the pre/post order of the videotapes analysed the samples. RESULTS: The FOCUS measured significantly more change (p<0.01) during treatment than during the waiting list period. It demonstrated both convergent and discriminant validity against the ASQ-SE. The FOCUS change corresponded to change measured by a combination of clinical speech and language measures (κ=0.31, p<0.05). CONCLUSION: The FOCUS shows strong construct validity as a change-detecting instrument.

Exploring the outcomes of a novel computer-assisted treatment program targeting expressive-grammar deficits in preschoolers with SLI.

PURPOSE: The impact of a newly designed computer-assisted treatment (C-AT) program, My Sentence Builder, for the remediation of expressive-grammar deficits in children with specific language impairment (SLI) was explored. This program was specifically designed with features to directly address expressive-grammar difficulties, thought to be associated with hypothesized deficits in verbal working memory (VWM). METHOD: Thirty-four preschoolers with deficits in expressive-grammar morphology participated. Using the randomization procedure of consecutive sampling, participants were recruited. Twenty-two participants were consecutively assigned to one of two treatment groups, C-AT or non C-AT (nC-AT). The nC-AT utilized conventional language stimulation procedures containing features which have been traditionally used to address expressive-grammar deficits. A group of equivalent children awaiting treatment and chosen from the same sample of children as the treatment participants served as a control group. Blind assessments of outcomes were completed pre-, post-, and 3-months post-treatment in a formal and informal context. RESULTS: C-AT and nC-AT participants significantly outperformed controls pre-to-post to 3-months post-treatment in both assessment contexts. No significant differences in treatment gains were found between C-AT and nC-AT. CONCLUSION: Results suggested that treatments designed to directly address expressive-grammar deficits were better than no treatment for preschool SLI. Further, use of a C-AT program may be another feasible treatment method for this disorder population. LEARNING OUTCOMES: As a result of this activity, the reader will recognize that: (1) expressive-grammar treatment is better than no treatment for immediate and continued language growth, (2) use of a C-AT program containing specific features designed to directly address expressive-grammar deficits is another viable, but not necessarily a better treatment option for the remediation of expressive-grammar deficits in preschool children with SLI, and (3) different outcome contexts yield distinct yet equally important findings about growth in children's expressive-grammar skills with treatment.

Development of the FOCUS (Focus on the Outcomes of Communication Under Six), a communication outcome measure for preschool children.

AIM: Our aim was to develop an outcome measure, called Focus on the Outcomes of Communication Under Six (FOCUS), that captures real-world changes in preschool children's communication. Conceptually grounded in the World Health Organization International Classification of Functioning, Disability and Health framework, the FOCUS items were derived from observations of change provided by parents and clinicians after speech-language therapy. METHOD: Participating families (n=165) were drawn from a convenience sample at partner institutions in Ontario, Newfoundland and Labrador, and Nova Scotia. All children had speech, language comprehension, and/or language production disorders and were receiving speech-language therapy. The age of the children ranged from 1 year 2 months to 5 years 6 months (mean=3.8y; SD=0.91y), and 119 children were male. Three test phases were completed. The measure was revised according to item analysis and parent/clinician feedback after phase 1 (n=74) and phase 2 (n=65). In phase 3 (n=26), the Pediatric Quality of Life Inventory (PedsQL), a health-related quality of life measure, was added to establish construct validity. RESULTS: In phase 1, item analysis revealed high internal consistency for both parents (Cronbach's alpha=0.87) and clinicians (Cronbach alpha=0.97). These values indicated redundancy, so 31 items were cut. Five items for young children were added. In phase 2, internal consistency remained high for both parents (Cronbach alpha=0.98) and clinicians (Cronbach alpha=0.83), indicating redundancy of items. Twenty-seven items were removed. In phase 3, parents and clinicians reliably scored the FOCUS in 10 minutes. They stated that it provided an accurate snapshot of the child's communication. Internal consistency for parents remained high (Cronbach alpha=0.96). Children with higher FOCUS scores at the end of treatment had higher PedsQL total scores (r=0.466, p=0.029). INTERPRETATION: The FOCUS is a usable measure of a child's ability to communicate and participate in his or her community. It demonstrates high internal consistency and construct validity.

Predicted and observed outcomes in preschool children following speech and language treatment: parent and clinician perspectives.

UNLABELLED: Parents of 210 preschool children (age 2-5.7) and their clinicians were asked to describe their expectations for therapy and the changes they observed following treatment. Based on content analysis of the parents' and clinicians' responses, it was apparent that the comments aligned with the World Health Organization's (WHO) International Classification of Functioning, Disability and Health-Child and Youth version (ICF-CY) framework. Parents and clinicians observed positive changes across the ICF-CY domains of Body Functions, Activities and Participation and Personal Factors following therapy. Parents noted twice as many changes in the Participation and Personal Factors domains as clinicians. Parents described improvements in play, socialization, confidence and behaviour at home, school and in the community-changes not typically captured by other preschool speech and language outcome measures. New outcome measures need to be based on actual observations of change by both parents and clinicians to ensure that they measure a sufficiently broad-based range of skills. LEARNING OUTCOMES: The reader will better understand (1) the parents' expectations for therapy (2) the types of changes that can be associated with speech and language therapy and (3) the need for broad-based outcome measures that can evaluate speech and language outcomes.

Clinical use of the Mayo-Portland Adaptability Inventory in rehabilitation after paediatric acquired brain injury.

The Mayo-Portland Adaptability Inventory (MPAI; designed to be administered by clinicians) is a popular measure of disability following head injury in adults. Its acceptability, validity, and reliability were assessed for use with children. There were 335 children and adolescents (215 males, 120 females) aged between 1 and 19 years at injury (median age 9y 8mo [SD 5y]) in our sample. The test was acceptable to respondents, rapidly and easily administered, and required only small modifications. It demonstrated validity against client and parent reports of major symptoms. It demonstrated test-retest reliability within the limitations of our data and excellent interrater accord. Consequently, the MPAI is recommended for paediatric use for evaluating rehabilitation needs and therapy outcome.