Exploring the Factor Structure of the Social Cognition and Object Relations-Global Rating Method: Support for Two- and Three-Factor Models.

The Social Cognition and Object Relations Scales-Global Rating Method (SCORS-G) contains 8 scales for coding narrative content. This study explores the factor structure of this measure using college (n = 171), outpatient (n = 239), and inpatient (n = 78) samples. Participants told stories to the Thematic Apperception Test (TAT; Murray, 1943) cards. Stories were transcribed and coded by blind raters using the SCORS-G. Cases were randomly assigned to an exploratory or validation group. Exploratory factor analysis with the exploratory group suggested 2- and 3-factor models. The Emotional Investment in Relationships (EIR) scale did not obtain a primary loading on any factor and was not included in subsequentmodels. After modifications, confirmatory factor analysis indicated good-to-adequate fit for 2- and 3-factor models. Both models showed good fit in the validation group and met criteria for invariance across models. Findings indicated that some SCORS-G scales tap cognitive-structural elements, whereas others assess affective-relational components of narratives. We found mild support separating the affective-relational scales in terms of internal representations for the self and others and relationships. The results reported here indicate that clinicians and researchers can calculate a separate cognitive-structural composite score and an affective-relational composite score when using the SCORS-G to rate TAT stories.

Dimensional measure of self and interpersonal functioning: Comparisons with treatment alliance and readiness for inpatient psychotherapy.

The social cognition and object relations scale-global rating method is a clinical rating system assessing 8 domains of self and interpersonal functioning. It can be applied to score numerous forms of narrative data. In this study, we investigate the SCORS-G relationship to measures of alliance and readiness for psychotherapy with an adolescent inpatient sample. Seventy-two psychiatrically hospitalized adolescents were consented and subsequently rated by their individual and group therapist using the SCORS-G. The unit psychiatrist also completed an assessment of patients' readiness for inpatient psychotherapy. The patients completed a self-report of their alliance with the inpatient treatment team as a whole. SCORS-G ratings were positively correlated with assessments of readiness for inpatient psychotherapy and patient-reported alliance. This study further demonstrates the clinical utility of the SCORS-G with adolescents.

Reliability and Validity of Prototype Diagnosis for Adolescent Psychopathology.

The current study investigated the interrater reliability and validity of prototype ratings of 5 common adolescent psychiatric disorders: attention-deficit/hyperactivity disorder, conduct disorder, major depressive disorder, generalized anxiety disorder, and posttraumatic stress disorder. One hundred fifty-seven adolescent inpatient participants consented to participate in this study. We compared ratings from 2 inpatient clinicians, blinded to each other's ratings and patient measures, after their separate initial diagnostic interview to assess interrater reliability. Prototype ratings completed by clinicians after their initial diagnostic interview with adolescent inpatients and outpatients were compared with patient-reported behavior problems and parents' report of their child's behavioral problems. Prototype ratings demonstrated good interrater reliability. Clinicians' prototype ratings showed predicted relationships with patient-reported behavior problems and parent-reported behavior problems. Prototype matching seems to be a possible alternative for psychiatric diagnosis. Prototype ratings showed good interrater reliability based on clinicians unique experiences with the patient (as opposed to video-/audio-recorded material) with no training.

Construct Validity of the Relationship Profile Test: Links With Measures of Psychopathology and Adult Attachment.

This study assessed the construct validity of the Relationship Profile Test (RPT; Bornstein & Languirand, 2003 ) with a substance abuse sample. One hundred-eight substance abuse patients completed the RPT, Experiences in Close Relationships Scale-Short Form (Wei, Russell, Mallinckrodt, & Vogel, 2007 ), Personality Assessment Inventory (Morey, 1991 ), and Symptom Checklist-90-Revised (Derogatis, 1983 ). Results suggest that the RPT has good construct validity when compared against theoretically related broadband measures of personality, psychopathology, and adult attachment. Overall, health dependency was negatively related to measures of psychopathology and insecure attachment, and overdependence was positively related to measures of psychopathology and attachment anxiety. Many of the predictions regarding RPT detachment and the criterion measures were not supported. Implications of these findings are discussed.

Clinical validity of a dimensional assessment of self- and interpersonal functioning in adolescent inpatients.

The Social Cognition and Object Relations Scale-Global Version (SCORS-G) is a clinical rating system assessing 8 domains of self- and interpersonal relational experience that can be applied to narrative response data (e.g., Thematic Apperception Test [TAT; Murray, 1943], early memories narratives) or oral data (e.g., psychotherapy narratives, relationship anecdotal paradigms). In this study, 72 psychiatrically hospitalized adolescents consented and were rated by their individual and group therapist using the SCORS-G. Clinicians also rated therapy engagement, personality functioning, quality of peer relationships, school functioning, global assessment of functioning (GAF), history of eating-disordered behavior, and history of nonsuicidal self-injury. SCORS-G composite ratings achieved an acceptable level of interrater reliability and were associated with theoretically predicted variables (e.g., engagement in therapy, history of nonsuicidal self-injury). SCORS-G ratings also incrementally improved the prediction of therapy engagement and global functioning beyond what was accounted for by GAF scores. This study further demonstrates the clinical utility of the SCORS-G with adolescents.

Development and Initial Validation of a Brief Symptom Measure.

UNLABELLED: Self-report measures of psychiatric symptomatology are important components of treatment monitoring and service evaluation programs. However, the currently available measures have numerous limitations including being symptom or disorder specific, suited to a limited range of clinical settings, and having excessive burden. Consequently, there is a need for a brief and psychometrically robust measure of global symptomatology that is applicable across diverse clinical settings, therapeutic modalities and patient populations. This paper presents the development and initial validation of such a scale, the Brief Symptom Measure-25 (BSM-25). We report findings from multiple samples examining the reliability, validity, sensitivity to change and factor structure of the new instrument. The results suggest that the BSM-25 has good reliability, is suitable to multiple levels of care, sensitive to treatment induced change and has promising validity. Exploratory bifactor modelling revealed that all items loaded strongly on a general factor (bifactor) while also forming two minor group factors. Potential limitations of this study along with future research and clinical applications of the BSM-25 are discussed. KEY PRACTITIONER MESSAGE: The BSM-25 is a broad measure of symptom severity that is easy to administer and score, appropriate for divers patient populations, and suitable for monitoring progress in routine clinical practice.

Validation of a measure of alliance for an adolescent inpatient setting.

UNLABELLED: The link between alliance and treatment outcome is robust. Nevertheless, few, if any, self-report measures exist to assess the alliance between hospitalized adolescents and their treatment team as a whole. The present study looks to extend the use of a brief self-report measure of inpatient treatment alliance designed for adult inpatients to be used with adolescents. The scale is designed incorporating items that tap the three factors of alliance (bond, goals and collaboration) to assess the alliance that the patient has with his or her treatment team. Our results show that the Inpatient-Treatment Alliance Scale is unifactoral, shows good psychometrics and is linked in theoretically meaningful ways to global clinician ratings of engagement in individual psychotherapy. KEY PRACTITIONER MESSAGE: Inpatient treatment of adolescents requires the assessment of alliance to be between the patient and his or her treatment team rather than an individual clinician. Assessment of the alliance can benefit clinicians treating hospitalized adolescents especially because these patients are difficult to engage with in treatment. This study shows that the Inpatient-Treatment Alliance Scale is a promising measure for assessing treatment alliance on an adolescent inpatient setting.

Assessing overall functioning with adolescent inpatients.

The current study sought to evaluate the validity and reliability of a brief measure of overall functioning for adolescents. Clinicians were asked to complete the Overall Functioning Scale (OFS) for 72 adolescents consecutively admitted to the adolescent psychiatric inpatient service of a community safety net medical center. The results revealed that this new measure is related to the patients' length of stay, clinician-rated measures of social cognition and object relations, Global Assessment of Functioning (GAF) score at admission, as well as global rating of engagement in individual psychotherapy. The results also showed that the OFS was related to the patients' history of nonsuicidal self-harm as well as treatment outcome as assessed by measures of psychological health and well-being as well as symptoms. Hierarchical regressions reveal that the OFS shows incremental validity greater than the admission GAF score in predicting length of stay. The results also showed that the OFS demonstrates interrater reliability in the excellent range (intraclass correlation coefficient(1,2)) of 0.88. Clinical implications of the use of this tool and areas of future research are discussed.

Measuring outcomes in adult outpatient psychiatry.

Despite growing pressure for accountability, mental health professionals continue to debate the value of routinely measuring treatment outcomes. This paper sought to move the outcomes measurement debate forward by reviewing some of the current limitations in outcomes methodology and by providing initial strategies to address them. Using these strategies, we evaluated outcomes for a large diagnostically diverse group of adult outpatients receiving treatment as usual (TAU) within an academic medical centre. Initial self-report and clinician-rated assessments were obtained from 5546 patients, and follow-up data were obtained from 1572 (28%) patients. Using the subset of patients with the follow-up data, we determined treatment effect sizes, rates of reliable improvement (and deterioration) and rates of clinically significant improvement for all patients and for specific diagnostic groups (depression, anxiety, substance abuse, mood disorders not otherwise specified, bipolar and psychotic conditions). TAU outcomes for depression and anxiety were also compared with benchmarks derived from the randomized controlled trial literature. Lastly, the impact of patient or sample characteristics on outcome was explored. Overall, these findings generally support the benefit of TAU over no treatment while also highlighting both the utility and limitations inherent in the current approaches to evaluating treatment outcomes. Suggestions for improving outcomes measurement are provided.

A psychometric evaluation of the Personality Assessment Inventory - short form clinical scales in an inpatient psychiatric sample.

Few studies have assessed the psychometric properties of the Personality Assessment Inventory short-form (PAI-SF) clinical scales, and none have conducted these evaluations using participants from psychiatric inpatient units. The present study evaluated item-level tests of scaling assumptions of the PAI-SF using a large (N=503) clinical sample of participants who completed the PAI during their admission to a psychiatric inpatient unit. Internal consistency reliability was high across scales, and tests of item-scale convergence and discrimination generally confirmed hypothesized item groupings. Scale-level correlations supported unique variance being measured by each scale. Finally, agreement between the PAI short- and full-form scales was found to be high. The results are discussed with regards to scale interpretation.

Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes.

OBJECTIVES: To measure participation outcomes with a computerized adaptive test (CAT) and compare CAT and traditional fixed-length surveys in terms of score agreement, respondent burden, discriminant validity, and responsiveness. DESIGN: Longitudinal, prospective cohort study of patients interviewed approximately 2 weeks after discharge from inpatient rehabilitation and 3 months later. SETTING: Follow-up interviews conducted in patient's home setting. PARTICIPANTS: Adults (N=94) with diagnoses of neurologic, orthopedic, or medically complex conditions. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Participation domains of mobility, domestic life, and community, social, & civic life, measured using a CAT version of the Participation Measure for Postacute Care (PM-PAC-CAT) and a 53-item fixed-length survey (PM-PAC-53). RESULTS: The PM-PAC-CAT showed substantial agreement with PM-PAC-53 scores (intraclass correlation coefficient, model 3,1, .71-.81). On average, the PM-PAC-CAT was completed in 42% of the time and with only 48% of the items as compared with the PM-PAC-53. Both formats discriminated across functional severity groups. The PM-PAC-CAT had modest reductions in sensitivity and responsiveness to patient-reported change over a 3-month interval as compared with the PM-PAC-53. CONCLUSIONS: Although continued evaluation is warranted, accurate estimates of participation status and responsiveness to change for group-level analyses can be obtained from CAT administrations, with a sizeable reduction in respondent burden.

Effectively Assessing Treatment Needs in Incarcerated Seriously Mentally Ill Individuals: The Utility of the Personality Assessment Inventory-Level of Care Index.

Early identification of treatment needs in incarcerated individuals with serious mental illness has significant implications. Validated assessment instruments to guide treatment are lacking in correctional settings. Hierarchical linear modeling was used to examine the predictive validity of the Level of Care Index (LOCI) in 35 inmates admitted to a specialized treatment unit. The LOCI score was predictive of levels of depressogenic psychopathology and psychological well-being as well as changes in these constructs over time. These results validate the use of the LOCI in a correctional setting and demonstrate the utility of the LOCI to identify treatment trajectories.

Cognitive status and profile validity on the Personality Assessment Inventory (PAI) in offenders with serious mental illness.

Cognitive impairment among seriously mentally ill offenders has implications for legal matters (e.g., competency to stand trial), as well as clinical treatment and care. Thus, being able to identify potential cognitive concerns early in the adjudication process can be important when deciding on further interventions. In this study, we examined the validity scales of the Personality Assessment Inventory (PAI), scores on the Wechsler Adult Intelligence Scale-IV (WAIS-IV), and competency findings in male inmates (n=61) diagnosed with a serious mental illness. Lower scores on the WAIS-IV significantly (p=0.001) predicted invalid, versus valid, PAI profiles, with working memory impairment being the most significant (p=0.004) predictor of an invalid profile. Ancillary analyses on a smaller sample (n=18) indicate that those with invalid PAI profiles were more likely to be deemed legally incompetent (p=0.03). These findings suggest that the PAI validity scales may be informative in detecting cognitive concerns and help clinicians make determinations about competency restoration and treatment.

Psychometric evaluation of the SF-36 health survey in Medicare managed care.

Data quality and scoring assumptions for the SF-36 Health Survey were evaluated among the elderly and disabled, using 1998 Cohort I baseline Medicare HOS data (n=177,714). Missing data rates were low, and scoring assumptions were met. Internal consistency reliability was 0.83 to 0.93 for the eight scales and 0.94 and 0.89, respectively, for the physical (PCS) and mental (MCS) component summary measures. Results declined with increased risk factors (e.g., older age, more chronic conditions), but were well above accepted standards for all subgroups. These findings support using standard algorithms for scoring the SF-36 in the HOS and subgroup analyses of HOS data.

Treatment as usual (TAU) for depression: a comparison of psychotherapy, pharmacotherapy, and combined treatment at a large academic medical center.

Depression is among the most prevalent and burdensome psychiatric disorders in the United States (Kessler et al., Achieves of General Psychiatry 62:617-627, 2005). There is substantial empirical support regarding efficacy of pharmacotherapy, psychotherapy, and combined treatment (both pharmacotherapy and psychotherapy) for treating depression. However, far less is known about the effectiveness of these treatments for real-world patients treated within routine clinical care settings (Cahill et al., The British Journal of Clinical Psychology 49:421-453, 2010). This study seeks to explore the effectiveness of treatment as usual (TAU) for depression in a large cohort of psychiatric outpatients receiving psychotherapy, pharmacotherapy, or combined treatment within an academic medical center. Initial and follow-up self-report assessments were analyzed for 1,322 patients receiving treatment for depression. Using these data, we determined treatment effect sizes, rates of reliable improvement (and deterioration), and rates of clinically significant improvement for psychotherapy, pharmacotherapy, and combined care. On average, all treatments produced significant improvement with effect sizes surpassing our no-treatment benchmark. No significant between-group (treatment) differences in self-report outcomes were found. The rates of reliable change were similar for all treatment groups consistent with past research. The present findings support the effectiveness of psychotherapy, pharmacotherapy, and combined treatment as routinely provided within a large academic medical center for the treatment of real-world patients suffering with depression.

Psychometric properties of the Rosenberg Self-Esteem Scale: overall and across demographic groups living within the United States.

The purpose of this study was twofold: (a) to evaluate the scaling assumptions and component structure of and present normative data for the Rosenberg Self-Esteem Scale (RSES) using a sample of US adults (N = 503), both overall and across demographic subgroups and (b) to provide new data regarding the relationship between the two RSES subcomponents of self-competence (SC) and self-liking (SL), and other demographic and clinical variables. As hypothesized, all psychometric tests supported the underlying structure of the RSES. Overall RSES scores varied significantly across age, racial and ethnic, education, employment status, income, and marital status groups. Furthermore, differences between SC and SL were also found across groups differing in gender, age, employment status, and marital status groups. The implications and limitations of this study are discussed, with an emphasis on clinical relevance.

An item-level psychometric analysis of the personality assessment inventory: clinical scales in a psychiatric inpatient unit.

Multi-item multiscale self-report measures are increasingly used in inpatient assessments. When considering a measure for this setting, it is important to evaluate the psychometric properties of the clinical scales and items to ensure that they are functioning as intended in a highly distressed clinical population. The present study examines scale properties for a self-report measure frequently employed in inpatient assessments, the Personality Assessment Inventory (PAI). In addition to examining internal consistency statistics, this study extends prior PAI research by considering key issues related to inpatient assessment (e.g., scale distinctiveness, ceiling effects). Coefficient alphas, interitem correlations, and item- scale relationships suggest that the PAI clinical scales and subscales are internally consistent. Items for respective clinical scales generally showed significantly higher item-scale correlations with their intended scale (as compared with their item-scale correlation with scales they were not intended to measure). In addition, scales' coefficient alpha scores were higher than their interscale correlations. Taken as a whole, these results support the hypothesis that PAI scales were measuring relatively distinct constructs in this inpatient sample. Findings are discussed with regard to the implications for scale interpretation in inpatient assessment, functioning of individual scales and subscales, and functioning of specific items. Limitations of the present study and directions for future research are discussed.

Development and initial psychometric evaluation of the participation measure for post-acute care (PM-PAC).

OBJECTIVE: To evaluate a patient-reported participation measure constructed within the framework of the International Classification of Functioning, Disability, and Health. DESIGN: The Participation Measure for Post-Acute Care (PM-PAC) contains 51 items that assess participation in nine domains: mobility; role functioning; community, social, and civic life; domestic life/self-care; economic life; interpersonal relationships; communication; work; and education. Self-reported data were collected by interview with 395 noninstitutionalized rehabilitation patients. Psychometric analyses were conducted to test assumptions underlying the scaling and scoring of PM-PAC scales and to evaluate reliability and validity. RESULTS: In general, analyses supported assignment of items to hypothesized scales. However, the community, social, and civic life, interpersonal relationships, and communication domains were highly correlated. Item response theory models indicated that items were directed primarily at people with worse than average limitations in participation than the average patient in this rehabilitation sample. Test and retest scale scores did not differ significantly (intraclass correlation coefficient range, 0.61-0.86). Groups with more severe conditions scored worse on PM-PAC scales, as hypothesized. CONCLUSIONS: PM-PAC is a promising new measure of patient-reported participation as defined by the International Classification of Functioning, Disability, and Health. Information about PM-PAC items from this study will be useful in developing a computerized, adaptive measure of participation.

Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: I. Activity outcomes.

OBJECTIVE: To examine score agreement, precision, validity, efficiency, and responsiveness of a computerized adaptive testing (CAT) version of the Activity Measure for Post-Acute Care (AM-PAC-CAT) in a prospective, 3-month follow-up sample of inpatient rehabilitation patients recently discharged home. DESIGN: Longitudinal, prospective 1-group cohort study of patients followed approximately 2 weeks after hospital discharge and then 3 months after the initial home visit. SETTING: Follow-up visits conducted in patients' home setting. PARTICIPANTS: Ninety-four adults who were recently discharged from inpatient rehabilitation, with diagnoses of neurologic, orthopedic, and medically complex conditions. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Summary scores from AM-PAC-CAT, including 3 activity domains of movement and physical, personal care and instrumental, and applied cognition were compared with scores from a traditional fixed-length version of the AM-PAC with 66 items (AM-PAC-66). RESULTS: AM-PAC-CAT scores were in good agreement (intraclass correlation coefficient model 3,1 range, .77-.86) with scores from the AM-PAC-66. On average, the CAT programs required 43% of the time and 33% of the items compared with the AM-PAC-66. Both formats discriminated across functional severity groups. The standardized response mean (SRM) was greater for the movement and physical fixed form than the CAT; the effect size and SRM of the 2 other AM-PAC domains showed similar sensitivity between CAT and fixed formats. Using patients' own report as an anchor-based measure of change, the CAT and fixed length formats were comparable in responsiveness to patient-reported change over a 3-month interval. CONCLUSIONS: Accurate estimates for functional activity group-level changes can be obtained from CAT administrations, with a considerable reduction in administration time.

Activity outcome measurement for postacute care.

BACKGROUND: Efforts to evaluate the effectiveness of a broad range of postacute care services have been hindered by the lack of conceptually sound and comprehensive measures of outcomes. It is critical to determine a common underlying structure before employing current methods of item equating across outcome instruments for future item banking and computer-adaptive testing applications. OBJECTIVE: To investigate the factor structure, reliability, and scale properties of items underlying the Activity domains of the International Classification of Functioning, Disability and Health (ICF) for use in postacute care outcome measurement. METHODS: We developed a 41-item Activity Measure for Postacute Care (AM-PAC) that assessed an individual's execution of discrete daily tasks in his or her own environment across major content domains as defined by the ICF. We evaluated the reliability and discriminant validity of the prototype AM-PAC in 477 individuals in active rehabilitation programs across 4 rehabilitation settings using factor analyses, tests of item scaling, internal consistency reliability analyses, Rasch item response theory modeling, residual component analysis, and modified parallel analysis. RESULTS: Results from an initial exploratory factor analysis produced 3 distinct, interpretable factors that accounted for 72% of the variance: Applied Cognition (44%), Personal Care & Instrumental Activities (19%), and Physical & Movement Activities (9%); these 3 activity factors were verified by a confirmatory factor analysis. Scaling assumptions were met for each factor in the total sample and across diagnostic groups. Internal consistency reliability was high for the total sample (Cronbach alpha = 0.92 to 0.94), and for specific diagnostic groups (Cronbach alpha = 0.90 to 0.95). Rasch scaling, residual factor, differential item functioning, and modified parallel analyses supported the unidimensionality and goodness of fit of each unique activity domain. CONCLUSIONS: This 3-factor model of the AM-PAC can form the conceptual basis for common-item equating and computer-adaptive applications, leading to a comprehensive system of outcome instruments for postacute care settings.