Psychometric Evaluation of a New Digitally Animated Child Self-Report Assessment Instrument: The Interactive Child Distress Screener.

The present study aimed to provide the first psychometric evaluation of the newly developed, digitally animated assessment instrument: the Interactive Child Distress Screener (ICDS). The latent factor structure of the novel ICDS was first established using exploratory factor analysis (EFA) on 15 pairs of animated items using a community sample (N = 266) of child-parent dyads. EFA results support a two-factor structure representing two broad domains of internalizing and externalizing difficulties (r = .52) and comprised 12 items. The reliability of the factors was strong with ordinal alpha and omega coefficients above .84 and .87, respectively, for each of the subscales. Convergent validity for the overall sample was supported with established child and parent-reported measures of internalizing and externalizing problems; however, the ICDS factors demonstrated convergence greater in magnitude with other child-reported measures such as the Me and My School Survey. Satisfaction and utility ratings by children indicated that the digital format was highly acceptable.

Co-design and qualitative validation of animated assessment item content for a child-reported digital distress screener.

PURPOSE: The Interactive Child Distress Screener (ICDS) is a novel, digital screening tool that is currently under development and aims to broadly assess self-reported emotional and behavioural distress in children aged five to 11 years. This study implemented a generative participatory codesign and iterative refinement process to qualitatively validate the content of 30 animated assessment items developed for the ICDS by assessing their acceptability and accuracy from the child's perspective. METHODS: The participants (N = 62) were five to 11-year-old children. Individual interviews were conducted with each child to determine acceptability and validity of animated items and facilitate the co-design refinement process of the animated assessment items. RESULTS: Twenty-two out of 30 (73%) items met ≥80% satisfaction and accuracy consensus in their original format, six items (20%) required one round of refinement before meeting consensus, and two items (7%) required two rounds of refinements. Combined acceptability of animated items was high, ranging from 4.1 to 5 out of 5 across all items. CONCLUSION: Participants were able to accurately identify and understand socio-emotional and behavioural constructs when depicted as animated items. Acceptability was high, even in first iterations when accuracy of understanding required refinement. This study highlighted the importance and benefits of iterative participatory design methodology in ensuring assessment items developed for children are understood, accepted and likely to be effective in obtaining accurate self-report.

The Interactive Child Distress Screener: Development and Preliminary Feasibility Testing.

BACKGROUND: Early identification of child emotional and behavioral concerns is essential for the prevention of mental health problems; however, few suitable child-reported screening measures are available. Digital tools offer an exciting opportunity for obtaining clinical information from the child's perspective. OBJECTIVE: The aim of this study was to describe the initial development and pilot testing of the Interactive Child Distress Screener (ICDS). The ICDS is a Web-based screening instrument for the early identification of emotional and behavioral problems in children aged between 5 and 12 years. METHODS: This paper utilized a mixed-methods approach to (1) develop and refine item content using an expert review process (study 1) and (2) develop and refine prototype animations and an app interface using codesign with child users (study 2). Study 1 involved an iterative process that comprised the following four steps: (1) the initial development of target constructs, (2) preliminary content validation (face validity, item importance, and suitability for animation) from an expert panel of researchers and psychologists (N=9), (3) item refinement, and (4) a follow-up validation with the same expert panel. Study 2 also comprised four steps, which are as follows: (1) the development of prototype animations, (2) the development of the app interface and a response format, (3) child interviews to determine feasibility and obtain feedback, and (4) refinement of animations and interface. Cognitive interviews were conducted with 18 children aged between 4 and 12 years who tested 3 prototype animated items. Children were asked to describe the target behavior, how well the animations captured the intended behavior, and provide suggestions for improvement. Their ability to understand the wording of instructions was also assessed, as well as the general acceptability of character and sound design. RESULTS: In study 1, a revised list of 15 constructs was generated from the first and second round of expert feedback. These were rated highly in terms of importance (mean 6.32, SD 0.42) and perceived compatibility of items (mean 6.41, SD 0.45) on a 7-point scale. In study 2, overall feedback regarding the character design and sounds was positive. Children's ability to understand intended behaviors varied according to target items, and feedback highlighted key objectives for improvements such as adding contextual cues or improving character detail. These design changes were incorporated through an iterative process, with examples presented. CONCLUSIONS: The ICDS has potential to obtain clinical information from the child's perspective that may otherwise be overlooked. If effective, the ICDS will provide a quick, engaging, and easy-to-use screener that can be utilized in routine care settings. This project highlights the importance of involving an expert review and user codesign in the development of digital assessment tools for children.