Development and validation of the SickKids Enterprise-wide Data in Azure Repository (SEDAR).

Objectives: To describe the processes developed by The Hospital for Sick Children (SickKids) to enable utilization of electronic health record (EHR) data by creating sequentially transformed schemas for use across multiple user types. Methods: We used Microsoft Azure as the cloud service provider and named this effort the SickKids Enterprise-wide Data in Azure Repository (SEDAR). Epic Clarity data from on-premises was copied to a virtual network in Microsoft Azure. Three sequential schemas were developed. The Filtered Schema added a filter to retain only SickKids and valid patients. The Curated Schema created a data structure that was easier to navigate and query. Each table contained a logical unit such as patients, hospital encounters or laboratory tests. Data validation of randomly sampled observations in the Curated Schema was performed. The SK-OMOP Schema was designed to facilitate research and machine learning. Two individuals mapped medical elements to standard Observational Medical Outcomes Partnership (OMOP) concepts. Results: A copy of Clarity data was transferred to Microsoft Azure and updated each night using log shipping. The Filtered Schema and Curated Schema were implemented as stored procedures and executed each night with incremental updates or full loads. Data validation required up to 16 iterations for each Curated Schema table. OMOP concept mapping achieved at least 80 % coverage for each SK-OMOP table. Conclusions: We described our experience in creating three sequential schemas to address different EHR data access requirements. Future work should consider replicating this approach at other institutions to determine whether approaches are generalizable.

Interprofessional peer-assisted learning for pharmacy and physical therapy students using inhalers and inhalation devices.

BACKGROUND: Peer-assisted learning (PAL) is exchanging knowledge between learners often from similar professional levels. Limited evidence exists on the effectiveness of PAL between different healthcare professions. The purpose of this study is to evaluate the knowledge, confidence, and perception of students engaged in an interprofessional PAL activity with pharmacy students instructing physical therapy students on the proper technique, cleaning/storage and therapeutic knowledge on inhaler devices for treatment of pulmonary conditions. METHODS: Pharmacy and physical therapy students completed a survey before and immediately after the PAL activity. As instructors, pharmacy students rated their experience with inhalers, their confidence if they were to assist clients on the use of inhaler devices and confidence in teaching peers. Physical therapy students completed surveys on inhaler knowledge with 10 scenario-based multiple-choice questions, and their confidence if they were to assist clients with inhaler devices. The knowledge questions were grouped into three categories: storage and cleaning of inhalers (3 questions), technique of using inhalers (4 questions), and therapeutic knowledge of drugs given by inhalation (3 questions). RESULTS: 102 physical therapy and 84 pharmacy students completed the activity and surveys. For the physical therapy students, the mean improvement of the total score for knowledge-based questions was 3.6 ± 1.8 (p < 0.001). The question with the fewest number of correct answers (13%) before the PAL activity had the highest number of correct answers post-activity (95%). Prior to the activity, no physical therapy students felt certain/very certain about their knowledge on inhalers, yet after PAL activity this proportion increased to 35%. The percent of pharmacy students reporting their confidence as "certain" and "very certain" in teaching peers increased from 46% before the activity to 90% afterwards. Pharmacy students rated the monitoring and follow-up of inhaler devices as the lowest expectation for physical therapists to play a role. Steps taken to prepare for this PAL activity were also discussed. CONCLUSIONS: Interprofessional PAL can increase knowledge and confidence of healthcare students reciprocally learning and teaching in joint activities. Allowing such interactions facilitate students to build interprofessional relationships during their training, which can increase communication and collaboration to foster an appreciation for each other's roles in clinical practice.

Monte Carlo simulation with confusion matrix paradigm - A sample of internal consistency indices.

Monte Carlo simulation is a common method of providing empirical evidence to verify statistics used in psychological studies. A representative set of conditions should be included in simulation studies. However, several recently published Monte Carlo simulation studies have not included the conditions of the null distribution of the statistic in their evaluations or comparisons of statistics and, therefore, have drawn incorrect conclusions. This present study proposes a design based on a common statistic evaluation procedure in psychology and machine learning, using a confusion matrix with four cells: true positive, true negative, false negative modified, and false positive modified. To illustrate this design, we employ an influential Monte Carlo simulation study by Trizano-Hermosilla and Alvarado (2016), which concluded that the Omega-indexed internal consistency should be preferred over other alternatives. Our results show that Omega can report an acceptable level of internal consistency (i.e., > 0.7) in a population with no relationship between every two items in some conditions, providing novel empirical evidence for comparing internal consistency indices.

Autism spectrum disorder screening in preschools.

LAY ABSTRACT: With professional training and regular opportunities to observe children interacting with their peers, preschool teachers are in a good position to notice children's autism spectrum disorder symptomatology. Yet even when a preschool teacher suspects that a child may have autism spectrum disorder, fear of false alarm may hold the teacher back from alerting the parents, let alone suggesting them to consider clinical assessment for the child. A valid and convenient screening tool can help preschool teachers make more informed and hence more confident judgment. We set out to develop a screening tool that capitalizes on peer interaction as a naturalistic "stress test" to identify children more likely than their peers to have autism spectrum disorder. A total of 304 3- to 4-year-olds were observed at school with an 84-item preliminary checklist; data-driven item reduction yielded a 13-item Classroom Observation Scale. The Classroom Observation Scale scores correlated significantly with Autism Diagnostic Observation Schedule-2 scores. To validate the scale, another 322 2- to 4-year-olds were screened using the Classroom Observation Scale. The screen-positive children and randomly selected typically developing peers were assessed for autism spectrum disorder 1.5 years later. The Classroom Observation Scale as used by teachers and researchers near preschool onset predicted autism spectrum disorder diagnoses 1.5 years later. This user-friendly 13-item Classroom Observation Scale enables teachers and healthcare workers with little or no clinical training to identify, with reliable and valid results, preschoolers more likely than their peers to have autism spectrum disorder.

Psychometric validation of the Chinese patient- and family satisfaction in the intensive care unit questionnaires.

PURPOSE: To assess the psychometric properties of the Chinese version of the 23-item Patient Satisfaction in the Intensive Care Unit (PS-ICU-23) and 24-item Family Satisfaction in the Intensive Care Unit (FS-ICU-24) questionnaires. MATERIALS AND METHODS: Patients (n = 243) discharged from an intensive care unit (ICU) in Hong Kong and family members (n = 237) completed the translated questionnaires. We used confirmatory factor analysis to assess the construct validity and measurement equivalence across groups (gender, mechanical ventilation, casemix, length of stay in ICU), coefficient alpha for internal consistency (reliability) and concordance correlation coefficient (ρc) for agreement between patients' and family members' perspective on satisfaction. RESULTS: A three-factor model provided a better fit than the two-factor model for both PS-ICU-23 and FS-ICU-24 questionnaires. Factorial equivalence was present across groups in both questionnaires. Internal consistency was adequate for PS-ICU-23 (coefficient alpha overall domain 0.85; care subscale, 0.83; information subscale, 0.90; decision-making process subscale, 0.67) and for FS-ICU-24 (coefficient alpha overall scale 0.86; care subscale, 0.84; information subscale, 0.89; decision-making process subscale, 0.65). Overall ICU satisfaction agreement was moderate (0.40). CONCLUSIONS: Both Chinese PS-ICU-23 and FS-ICU-24 questionnaires have sound psychometric properties but family satisfaction may not be a good proxy for patient satisfaction in ICU.

Probability of bivariate superiority: A non-parametric common-language statistic for detecting bivariate relationships.

Researchers often focus on bivariate normal correlation (r) to evaluate bivariate relationships. However, these techniques assume linearity and depend on parametric assumptions. We propose a new nonparametric statistical model that can be more intuitively understood than the conventional r: probability of bivariate superiority (PBS). Our development of Bp, the estimator of a PBS relationship, extends Dunlap's (1994) common-language transformation of r (CLr) by providing a method to directly estimate PBS-the probability that when x is above (or below) the mean of all X, its paired y score will also be above (or below) the mean of all Y. Probability of superiority is an important form of bivariate relationship that until now could only be accurately estimated when data met the parametric assumptions for r. We specify the copula that forms the theoretical basis for PBS, provide an algorithm for estimating PBS from a sample, and describe the results of a Monte Carlo experiment that evaluated our algorithm across 448 data conditions. The PBS estimate, Bp, is robust to violations of parametric assumptions and offers a useful method for evaluating the significance of probability-of-superiority relationships in bivariate data. It is critical to note that Bp estimates a different form of bivariate relationship than does r. Our working examples show that a PBS effect can be significant in the absence of a significant correlation, and vice versa. In addition to utilizing the PBS model in future research, we suggest that this new statistical procedure be used to find theoretically important but previously overlooked effects from past studies.

Limited Usefulness of Capture Procedure and Capture Percentage for Evaluating Reproducibility in Psychological Science.

In psychological science, there is an increasing concern regarding the reproducibility of scientific findings. For instance, Replication Project: Psychology (Open Science Collaboration, 2015) found that the proportion of successful replication in psychology was 41%. This proportion was calculated based on Cumming and Maillardet (2006) widely employed capture procedure (CPro) and capture percentage (CPer). Despite the popularity of CPro and CPer, we believe that using them may lead to an incorrect conclusion of (a) successful replication when the population effect sizes in the original and replicated studies are different; and (b) unsuccessful replication when the population effect sizes in the original and replicated studies are identical but their sample sizes are different. Our simulation results show that the performances of CPro and CPer become biased, such that researchers can easily make a wrong conclusion of successful/unsuccessful replication. Implications of these findings are considered in the conclusion.

Probability-of-Superiority SEM (PS-SEM)-Detecting Probability-Based Multivariate Relationships in Behavioral Research.

In behavioral research, exploring bivariate relationships between variables X and Y based on the concept of probability-of-superiority (PS) has received increasing attention. Unlike the conventional, linear-based bivariate relationship (e.g., Pearson's correlation), PS defines that X and Y can be related based on their likelihood-e.g., a student who is above mean in SAT has 63% likelihood of achieving an above-mean college GPA. Despite its increasing attention, the concept of PS is restricted to a simple bivariate scenario (X-Y pair), which hinders the development and application of PS in popular multivariate modeling such as structural equation modeling (SEM). Therefore, this study addresses an empirical-based simulation study that explores the potential of detecting PS-based relationship in SEM, called PS-SEM. The simulation results showed that the proposed PS-SEM method can detect and identify PS-based when data follow PS-based relationships, thereby providing a useful method for researchers to explore PS-based SEM in their studies. Conclusions, implications, and future directions based on the findings are also discussed.

Effect size measures in a two-independent-samples case with nonnormal and nonhomogeneous data.

In psychological science, the "new statistics" refer to the new statistical practices that focus on effect size (ES) evaluation instead of conventional null-hypothesis significance testing (Cumming, Psychological Science, 25, 7-29, 2014). In a two-independent-samples scenario, Cohen's (1988) standardized mean difference (d) is the most popular ES, but its accuracy relies on two assumptions: normality and homogeneity of variances. Five other ESs-the unscaled robust d (d r* ; Hogarty & Kromrey, 2001), scaled robust d (d r ; Algina, Keselman, & Penfield, Psychological Methods, 10, 317-328, 2005), point-biserial correlation (r pb ; McGrath & Meyer, Psychological Methods, 11, 386-401, 2006), common-language ES (CL; Cliff, Psychological Bulletin, 114, 494-509, 1993), and nonparametric estimator for CL (A w ; Ruscio, Psychological Methods, 13, 19-30, 2008)-may be robust to violations of these assumptions, but no study has systematically evaluated their performance. Thus, in this simulation study the performance of these six ESs was examined across five factors: data distribution, sample, base rate, variance ratio, and sample size. The results showed that A w and d r were generally robust to these violations, and A w slightly outperformed d r . Implications for the use of A w and d r in real-world research are discussed.

Interprofessional Peer Teaching of Pharmacy and Physical Therapy Students.

OBJECTIVE: To evaluate an interprofessional peer-teaching activity during which physical therapy students instructed undergraduate pharmacy students on 3 ambulatory devices (canes, crutches, walkers). DESIGN: The pre/post evaluation of 2 pharmacy undergraduate classes included 220 students, 110 per year. After pharmacy students completed a 10-point, knowledge-based pretest, they participated in a hands-on activity with physical therapy students teaching them about sizing, use, and safety of canes, crutches, and walkers. A 10-point posttest was completed immediately afterward. ASSESSMENT: The mean difference of pre/post scores was 3.5 (SD 1.9) for the peer-led teaching, and 3.8 (SD 2.2) for the peer learning group. Students had positive responses regarding the learning exercise and recommended further peer teaching. CONCLUSION: The peer-learning activity involving physical therapy students teaching pharmacy students was an effective method of improving knowledge and skills regarding basic ambulatory devices.

Evaluating Person Fit for Cognitive Diagnostic Assessment.

Methods evaluating person fit for cognitive diagnostic assessment are an important area of research because failing to detect misfitting responses can lead to the misinterpretation of students' attribute profiles, which may result in faulty remediation decisions. This article aims to examine ways of detecting person misfit for cognitive diagnostic assessments. The authors first investigated whether the well-known lz statistic, developed under the framework of item response theory, can be extended for use in the context of cognitive diagnostic models. The authors also introduce a new person fit statistic, response conformity index (RCI), developed for detecting misfitting response patterns for cognitive diagnostic assessments. The authors conduct both simulation and real data studies to compare the detection rates of lz and our new statistic.

Bootstrap confidence intervals for the mean correlation corrected for Case IV range restriction: a more adequate procedure for meta-analysis.

In this study, we proposed to use the nonparametric bootstrap procedure to construct the confidence interval for the mean correlation r corrected for Case IV range restriction in meta-analysis (i.e., ; Hunter, Schmidt, & Le, 2006). A comprehensive Monte Carlo study was conducted to evaluate the accuracy of the parametric confidence interval and 3 nonparametric bootstrap confidence intervals for r(c4). Of the 4 intervals, our results showed that the bootstrap bias-corrected and accelerated percentile interval (BCaI) yielded the most accurate results across different data situations. In addition, the mean-corrected correlation r(c4) was found to be more accurate than the uncorrected estimate. Implications of the mean-corrected correlation r(c4) and BCaI in organizational studies are also discussed.

Evaluating the performance of different procedures for constructing confidence intervals for coefficient alpha: a simulation study.

Reliability is one of the most important aspects of testing in educational and psychological measurement. The construction of confidence intervals for reliability coefficients has important implications for evaluating the accuracy of the sample estimate of reliability and for comparing different tests, scoring rubrics, or training procedures for raters or observers. The present simulation study evaluated and compared various parametric and non-parametric methods for constructing confidence intervals of coefficient alpha. Six factors were manipulated: number of items, number of subjects, population coefficient alpha, deviation from essentially parallel condition, item response distribution and type. The coverage and width of different confidence intervals were compared across simulation conditions.

Bootstrap standard error and confidence intervals for the correlations corrected for indirect range restriction.

The standard Pearson correlation coefficient, r, is a biased estimator of the population correlation coefficient, ρ(XY) , when predictor X and criterion Y are indirectly range-restricted by a third variable Z (or S). Two correction algorithms, Thorndike's (1949) Case III, and Schmidt, Oh, and Le's (2006) Case IV, have been proposed to correct for the bias. However, to our knowledge, the two algorithms did not provide a procedure to estimate the associated standard error and confidence intervals. This paper suggests using the bootstrap procedure as an alternative. Two Monte Carlo simulations were conducted to systematically evaluate the empirical performance of the proposed bootstrap procedure. The results indicated that the bootstrap standard error and confidence intervals were generally accurate across simulation conditions (e.g., selection ratio, sample size). The proposed bootstrap procedure can provide a useful alternative for the estimation of the standard error and confidence intervals for the correlation corrected for indirect range restriction.

Psychometric properties of the Spence Children's Anxiety Scale in a Hong Kong Chinese community sample.

This study evaluated the psychometric properties of a widely used self-report anxiety scale-the Spence Children's Anxiety Scale (SCAS) and the associated parent-report version (PSCAS)-in a Hong Kong Chinese community sample. While good psychometric properties of SCAS and PSCAS had been documented in Western cultural contexts (e.g., Australia), no systematic psychometric evaluation of the Chinese-translated SCAS and PSCAS has been published. In this study, psychometric properties of SCAS and PSCAS were examined with respect to four criteria: (a) factor structure, (b) descriptive statistics, (c) convergent validity with an anxiety cognition measure, and (d) internal consistency. Psychometric properties of SCAS and PSCAS for a Chinese community sample were found to be highly comparable with those published on Australian samples, thus providing a solid conceptual foundation for use of the Chinese version of SCAS and PSCAS.

Effectiveness of group cognitive-behavioral treatment for childhood anxiety in community clinics.

This study evaluated the effectiveness of cognitive-behavioral treatment for childhood anxiety in a community clinic setting in Hong Kong, China. Forty-five clinically-referred children (age 6-11 years) were randomly assigned to either a cognitive-behavioral treatment program or a waitlist-control condition. Children in the treatment condition showed significant reduction in anxiety symptoms-both statistically and clinically-whereas children in the waitlist condition did not. After the waitlist period was over, the control group also received the treatment program and showed a similar reduction in symptoms. For the full sample of 45 children, the effectiveness of the intervention was significant immediately after treatment and in 3- and 6-month follow-ups. In addition, children's anxiety cognition and their ability to cope with anxiety-provoking situations fully mediated the treatment gains. These results offer empirical support for cognitive-behavioral treatment programs in a non-Western cultural context and plausible mediators for how cognitive-behavioral therapy works.