Developing a team science workshop for early-career investigators.

INTRODUCTION: High impact biomedical research is increasingly conducted by large, transdisciplinary, multisite teams in an increasingly collaborative environment. Thriving in this environment requires robust teamwork skills, which are not acquired automatically in the course of traditional scientific education. Team science skills training does exist, but most is directed at clinical care teams, not research teams, and little is focused on the specific training needs of early-career investigators, whose early team leadership experiences may shape their career trajectories positively or negatively. Our research indicated a need for team science training designed specifically for early-career investigators. METHODS: To address this need, we designed and delivered a 2-day workshop focused on teaching team science skills to early-career investigators. We operationalized team science competencies, sought the advice of team science experts, and performed a needs assessment composed of a survey and a qualitative study. Through these multiple approaches, we identified and grouped training priorities into three broad training areas and developed four robust, hands-on workshop sessions. RESULTS: Attendees comprised 30 pre- and post-doc fellows (TL1) and early-career faculty (KL2 and K12). We assessed impact with a pre- and post-workshop survey adapted from the Team Skills Scale. Results from the pre- and post-test Wilcoxon signed-rank analysis (n = 25) showed statistically significant improvement in team science skills and confidence. Open-ended responses indicated that the workshop focus was appropriate and well targeted to the trainees' needs. CONCLUSIONS: Although team science education is still very much in its infancy, these results suggest that training targeted to early-career investigators improves team skills and may foster improved collaboration.

Sowing the "CEED"s of a more diverse biomedical workforce: The Career Education and Enhancement for Health Care Research Diversity (CEED) program at the University of Pittsburgh.

PURPOSE: The need to diversify the biomedical research workforce is well documented. The importance of fostering the careers of fledgling underrepresented background (URB) biomedical researchers is evident in light of the national and local scarcity of URB scientists in biomedical research. The Career Education and Enhancement for Health Care Research Diversity (CEED) program at the University of Pittsburgh Institute for Clinical Research Education (ICRE) was designed to promote career success and help seal the "leaky pipeline" for URB researchers. In this study, we aimed to quantify CEED's effect on several key outcomes by comparing CEED Scholars to a matched set of URB ICRE trainees not enrolled in CEED using data collected over 10 years. METHOD: We collected survey data on CEED Scholars from 2007 to 2017 and created a matched set of URB trainees not enrolled in CEED using propensity score matching in a 1:1 ratio. Poisson regression was used to compare the rate of publications between CEED and non-CEED URB trainees after adjusting for baseline number of publications. RESULTS: CEED has 45 graduates. Seventy-six percent are women, 78% are non-White, and 33% are Hispanic/Latino. Twenty-four CEED Scholars were matched to non-CEED URB trainees. Compared to matched URB trainees, CEED graduates had more peer-reviewed publications (p=0.0261) and were more likely to be an assistant professor (p=0.0145). CONCLUSIONS: Programs that support URB researchers can help expand and diversify the biomedical research workforce. CEED has been successful despite the challenges of a small demographic pool.

A Program to Support Scholarship During Internal Medicine Residency Training: Impact on Academic Productivity and Resident Experiences.

Problem: Although scholarship during residency training is an important requirement from the Accreditation Council for Graduate Medical Education, efforts to support resident scholarship have demonstrated inconsistent effects and have not comprehensively evaluated resident experiences. Intervention: We developed the Leadership and Discovery Program (LEAD) to facilitate scholarship among all non-research-track categorical internal medicine (IM) residents. This multifaceted program set expectations for all residents to participate in a scholarly project, supported faculty to manage the program, facilitated access to faculty mentors, established a local resident research day to highlight scholarship, and developed a didactic lecture series. Context: We implemented LEAD at a large university training program. We assessed resident scholarship before and after LEAD implementation using objective metrics of academic productivity (i.e., scientific presentations, peer-reviewed publications, and both presentations and publications). We compared these metrics in LEAD participants and a similar historical group of pre-LEAD controls. We also assessed these outcomes over the same two periods in research track residents who participated in research training independent from and predating LEAD (research track controls and pre-LEAD research track controls). We conducted focus groups to qualitatively assess resident experiences with LEAD. Outcome: Compared to 63 pre-LEAD controls, greater proportions of 52 LEAD participants completed scientific presentations (48.1% vs. 28.6%, p = .03) and scientific presentations and peer-reviewed publications (23.1% vs. 9.5%, p = .05). No significant differences existed for any academic productivity metrics among research track controls and pre-LEAD research track controls (p > .23, all comparisons). Perceived facilitators of participation in LEAD included residents' desire for research experiences and opportunities to publish prior to fellowship training; the main barrier to participation was feeling overwhelmed due to the time constraints imposed by clinical training. Suggestions for improvement included establishing clearer programmatic expectations and providing lists of potential mentors and projects. Lessons Learned: Implementation of a multifaceted program to support scholarship during residency was associated with significant increases in academic productivity among IM residents. Residents perceived that programs to support scholarship during residency training should outline clear expectations and identify available mentors and projects for residents who are challenged by the time constraints of clinical training.

Assessing an assessment: The review and redesign of a competency-based mid-degree evaluation.

INTRODUCTION: Little has been published about competency-based education in academic medicine, in particular how competencies are or should be assessed. This paper re-examines a competency-based assessment for M.S. students in clinical research, and "assesses the assessment" 4 years into its implementation. METHODS: Data were gathered from student surveys and interviews with program advisors, and common themes were identified. We then made refinements to the assessment, and student surveys were administered to evaluate the impact of the changes. RESULTS: Research results suggested the need to improve communication, time the assessment to align with skills development and opportunities for planning, streamline, and clarify expectations with examples and templates. After implementing these changes, data suggest that student satisfaction has improved without any reduction in academic rigor. CONCLUSION: The effective implementation of competency-based training in clinical and translational research requires the development of a scholarly literature on effective methods of assessment. This paper contributes to that nascent body of research.

The teams of early-career investigators: A qualitative pilot study.

Introduction: Early team experiences can influence the professional trajectories of early-career investigators profoundly, yet they remain underexplored in the team science literature, which has focused primarily on large, multisite teams led by established researchers. To better understand the unique challenges of teams led by early-career investigators, we conducted a qualitative pilot study. Methods: Interviews were conducted with the principal investigator and members of 5 teams led by KL2 and K12 scholars at the University of Pittsburgh. A code book was developed and thematic analysis was conducted. Results: Seven distinct themes emerged. Interview subjects reported a high level of trust and strong communication patterns on their teams; however, the data also suggested underlying tensions that have the potential to escalate into larger problems if unaddressed. Conclusions: This study yields a deeper understanding of teams led by early-career investigators, which can help us provide appropriately targeted training and support.

Improvement in Quality Metrics by the UPMC Enhanced Care Program: A Novel Super-Utilizer Program.

The aim was to evaluate pre-post quality of care measures among super-utilizer patients enrolled in the Enhanced Care Program (ECP), a primary care intensive care program. A pre-post analysis of metrics of quality of care for diabetes, hypertension, cancer screenings, and connection to mental health care for participants in the ECP was conducted for patients enrolled in ECP for 6 or more months. Patients enrolled in ECP showed statistically significant improvements in hemoglobin A1c, retinal exams, blood pressure measurements, and screenings for colon cancer, and trends toward improvement in diabetic foot exams and screenings for cervical and breast cancer. There was a significant increase in connecting patients to mental health care. This study shows that super-utilizer patients enrolled in the ECP had significant improvements in quality metrics from those prior to enrollment in ECP.

Automated Communication Tools and Computer-Based Medication Reconciliation to Decrease Hospital Discharge Medication Errors.

This study sought to determine the effects of automated primary care physician (PCP) communication and patient safety tools, including computerized discharge medication reconciliation, on discharge medication errors and posthospitalization patient outcomes, using a pre-post quasi-experimental study design, in hospitalized medical patients with ≥2 comorbidities and ≥5 chronic medications, at a single center. The primary outcome was discharge medication errors, compared before and after rollout of these tools. Secondary outcomes were 30-day rehospitalization, emergency department visit, and PCP follow-up visit rates. This study found that discharge medication errors were lower post intervention (odds ratio = 0.57; 95% confidence interval = 0.44-0.74; P < .001). Clinically important errors, with the potential for serious or life-threatening harm, and 30-day patient outcomes were not significantly different between study periods. Thus, automated health system-based communication and patient safety tools, including computerized discharge medication reconciliation, decreased hospital discharge medication errors in medically complex patients.

Assessing Competencies in a Master of Science in Clinical Research Program: The Comprehensive Competency Review.

Competencies in Master of Science Clinical Research programs are becoming increasingly common. However, students and programs can only benefit fully from competency-based education if students' competence is formally assessed. Prior to a summative assessment, students must have at least one formative, formal assessment to be sure they are developing competence appropriate for their stage of training. This paper describes the comprehensive competency review (CCR), a milestone for MS students in Clinical Research at the University of Pittsburgh's Institute for Clinical Research Education. The CCR involves metacognitive reflection of the student's learning as a whole, written evidence of each competency, a narrative explaining the choice of evidence for demonstrating competencies, and a meeting in which two faculty members review the evidence and solicit further oral evidence of competence. CCRs allow for individualized feedback at the midpoint in degree programs, providing students with confidence that they will have the means and strategies to develop competence in all areas by the summative assessment of competence at their thesis defense. CCRs have also provided programmatic insight on the need for curricular revisions and additions. These benefits outweigh the time cost on the part of students and faculty in the CCR process.

Effectiveness of a physical activity and weight loss intervention for middle-aged women: healthy bodies, healthy hearts randomized trial.

BACKGROUND: Physical inactivity is a significant risk factor for cardiovascular disease and remains highly prevalent in middle-aged women. OBJECTIVE: We hypothesized that an interventionist-led (IL), primary-care-based physical activity (PA) and weight loss intervention would increase PA levels and decrease weight to a greater degree than a self-guided (SG) program. DESIGN: We conducted a randomized trial. PARTICIPANTS: Ninety-nine inactive women aged 45-65 years and with BMI ≥ 25 kg/m(2) were recruited from three primary care clinics. INTERVENTIONS: The interventionist-led (IL) group (n = 49) had 12 weekly sessions of 30 min discussions with 30 min of moderate-intensity PA. The self-guided (SG) group (n = 50) received a manual for independent use. MAIN MEASURES: Assessments were conducted at 0, 3, and 12 months; PA and weight were primary outcomes. Weight was measured with a standardized protocol. Leisure PA levels were assessed using the Modifiable Activity Questionnaire. Differences in changes by group were analyzed with a t-test or Wilcoxon rank-sum test. Mixed models were used to analyze differences in changes of outcomes by group, using an intention-to-treat principle. KEY RESULTS: Data from 98 women were available for analysis. At baseline, mean (SD) age was 53.9 (5.4) years and 37 % were black. Mean weight was 92.3 (17.7) kg and mean BMI was 34.7 (5.9) kg/m(2). Median PA level was 2.8 metabolic equivalent hours per week (MET-hour/week) (IQR 0.0, 12.0). At 3 months, IL women had a significantly greater increase in PA levels (7.5 vs. 1.9 MET-hour/week; p = 0.02) than SG women; there was no significant difference in weight change. At 12 months, the difference between groups was no longer significant (4.7 vs. 0.7 MET-hour/week; p = 0.38). Mixed model analysis showed a significant (p = 0.048) difference in PA change between groups at 3 months only. CONCLUSIONS: The IL intervention was successful in increasing the physical activity levels of obese, inactive middle-aged women in the short-term. No significant changes in weight were observed.

Characterization of investigators' approach to translational research: a qualitative study.

BACKGROUND: Little is known about how investigators approach their research programs along the translational research continuum. Many consider the translational continuum to be linear, with research beginning at the bench and concluding with research at the bedside or in the community. We aimed to understand if translational investigators approach and view their research in this fashion. METHODS: We conducted semistructured individual interviews with 16 graduates of the University of Pittsburgh's Multidisciplinary Clinical Research Scholars Program (KL2) in 2012. RESULTS: Our research revealed three characteristic models. The first model we called "linear" and represented the traditional approach. The second we called "holistic"; these investigators began with central research questions and sought to explore them in every direction of translation, not necessarily taking linear steps. The third model we called "technical"; in this model, investigators focused on a unique technology or methodology and applied it across multiple research contexts. CONCLUSION: This study found that there are multiple ways that translational investigators approach their research program. Better understanding of these models can help educators and mentors guide investigators so that they can be more productive in their clinical or translational research career.

PaTH: towards a learning health system in the Mid-Atlantic region.

The PaTH (University of Pittsburgh/UPMC, Penn State College of Medicine, Temple University Hospital, and Johns Hopkins University) clinical data research network initiative is a collaborative effort among four academic health centers in the Mid-Atlantic region. PaTH will provide robust infrastructure to conduct research, explore clinical outcomes, link with biospecimens, and improve methods for sharing and analyzing data across our diverse populations. Our disease foci are idiopathic pulmonary fibrosis, atrial fibrillation, and obesity. The four network sites have extensive experience in using data from electronic health records and have devised robust methods for patient outreach and recruitment. The network will adopt best practices by using the open-source data-sharing tool, Informatics for Integrating Biology and the Bedside (i2b2), at each site to enhance data sharing using centrally defined common data elements, and will use the Shared Health Research Information Network (SHRINE) for distributed queries across the network.

Shortening the Work Preference Inventory for use with physician scientists: WPI-10.

The Work Preference Inventory (WPI) is a four-factor, 30-item measure that assesses work motivation. Used to help individuals choose appropriate career paths, its length contributes to response burden, especially when combined with other measures. We aimed to develop a shortened, valid, and reliable version of the WPI. Trainees at the University of Pittsburgh's Institute for Clinical Research Education completed the 30-item WPI between 2007 and 2012. We conducted exploratory and confirmatory factor analyses to reduce the number of items. Of the 402 eligible trainees, 371 (92%) provided data for the exploratory factor analysis (EFA), and 134 of the eligible 144 trainees (93%) provided data for the confirmatory factor analysis (CFA). EFA revealed four factors that were roughly equivalent to those of the original. CFA used the three items with the highest loadings on each factor, with two items removed due to low loadings and R-squareds, resulting in a 10-item scale. Cronbach's alpha for each of the four factors ranged from 0.68 to 0.76. Factors in the WPI-10 were strongly and significantly associated with factors in the original WPI, indicating strong validity of the shortened measure. The WPI-10 shows evidence for similar validity and reliability to the original instrument while reducing respondent burden.

Measurement of social capital among clinical research trainees.

While physical and human capital are established as important predictors of success among early-career clinical investigators, less is known about the role of social capital. The authors aimed to develop a brief scale to assess social capital in this population and test its reliability and validity. A three-item assessment was developed based on a conceptual framework and measures of social capital from other fields and was administered to 414 clinical research trainees at the University of Pittsburgh in 2007-2012. The measure exhibited good internal consistency reliability (α = 0.71) and a normal distribution. On a 10-point scale, mean social capital was 6.4 (SD = 1.7). Social capital was significantly associated with 7 of the 9 expected constructs: sex, age, confidence in research skills, work-related motivation, burnout, and social support. Exploratory multivariable regression analysis demonstrated that social capital was most strongly associated with higher research confidence (ß = 0.35, p < 0.001), higher extrinsic motivation (ß = 0.50, p = 0.003), and lower burnout (ptrend = 0.02). This three-item scale measures social capital in this population with adequate internal consistency reliability, face validity, and construct validity. This brief assessment provides a tool that may be valuable to benchmark social capital of clinical research trainees and to better contextualize programmatic and trainee outcomes.

Doctoral programs to train future leaders in clinical and translational science.

PURPOSE: Although the National Institutes of Health (NIH) has made extensive investments in educational programs related to clinical and translational science (CTS), there has been no systematic investigation of the number and characteristics of PhD programs providing training to future leaders in CTS. The authors undertook to determine the number of institutions that, having had received NIH-funded Clinical and Translational Science Awards (CTSAs), currently had or were developing PhD programs in CTS; to examine differences between programs developed before and after CTSA funding; and to provide detailed characteristics of new programs. METHOD: In 2012, CTS program leaders at the 60 CTSA-funded institutions completed a cross-sectional survey focusing on four key domains related to PhD programs in CTS: program development and oversight; students; curriculum and research; and milestones. RESULTS: Twenty-two institutions had fully developed PhD programs in CTS, and 268 students were earning PhDs in this new field; 13 institutions were planning PhD programs. New programs were more likely to have fully developed PhD competencies and more likely to include students in medical school, students working only on their PhD, students working on a first doctoral degree, and students working in T1 translational research. They were less likely to include physicians and students working in clinical or T2 research. CONCLUSIONS: Although CTS PhD programs have similarities, they also vary in their characteristics and management of students. This may be due to diversity in translational science itself or to the relative infancy of CTS as a discipline.

A shortened version of the Clinical Research Appraisal Inventory: CRAI-12.

PURPOSE: The original Clinical Research Appraisal Inventory (CRAI), which assesses the self-confidence of trainees in performing different aspects of clinical research, comprises 92 items. Completing the lengthy CRAI is time-consuming and represents a considerable burden to respondents, yet the CRAI provides useful data for evaluating research training programs. The purpose of this study is to develop a shortened version of the CRAI and to test its validity and reliability. METHOD: Trainees in clinical research degree and career development programs at the University of Pittsburgh's Institute for Clinical Research Education completed the 92-item CRAI between 2007 and 2012, inclusive. The authors conducted, first, exploratory factor analysis on a training dataset (2007-2010) to reduce the number of items and, then, confirmatory factor analyses on a testing dataset (2011-2012) to test the psychometric properties of the shortened version. RESULTS: Of 546 trainees, 394 (72%) provided study data. Exploratory factor analysis revealed six distinct factors, and confirmatory factor analysis identified the two items with the highest loadings per factor, for a total of 12 items. Cronbach alpha for the six new factors ranged from 0.80 to 0.94. Factors in the 12-item CRAI were strongly and significantly associated with factors in the 92-item CRAI; correlations ranged from 0.82 to 0.96 (P < .001 for each). CONCLUSIONS: The 12-item CRAI is faster and less burdensome to complete but retains the strong psychometric properties of the original CRAI.

Development, implementation, and evaluation of an interprofessional course in translational research.

The advancement of research from basic science discovery to clinical application requires the extensive collaboration of individuals from multiple disciplines, therefore the ability to work as an effective interprofessional team is essential for researchers in clinical and translational science (CTS). Courses that build interprofessional skills are a key component in CTS education, but the development of these courses poses numerous administrative and educational challenges. This paper describes the processes of designing, implementing, and evaluating an innovative graduate-level course that combines online lectures and in-class facilitated group discussions to promote interprofessional interactions. The course offers students the opportunity to interact with and learn from individuals in a variety of disciplines, and it requires students to engage in interprofessional group work to meet the course objectives. During the past 4 years, 96 students from the schools of medicine, pharmacy, nursing, public health, and health and rehabilitation sciences at a large urban university have completed the course. The course has been well-received, with 87% of students rating its overall quality as excellent, good, or satisfactory. The course offers educators a model to teach graduate students the skills that are essential for becoming effective CTS researchers.

Preliminary competencies for comparative effectiveness research.

The Clinical and Translational Science Award (CTSA) Workgroup for Comparative Effectiveness Research (CER) Education, Training, and Workforce Development identified a need to delineate the competencies that practitioners and users of CER for patient-centered outcomes research, should acquire. With input from CTSA representatives and collaborators, we began by describing the workforce. We recognize the workforce that conducts CER and the end users who use CER to improve the health of individuals and communities. We generated a preliminary set of competencies and solicited feedback from the CER representatives at each member site of the CTSA consortium. We distinguished applied competencies (i.e., skills needed by individuals who conduct CER) from foundational competencies that are needed by the entire CER workforce, including end users of CER. Key competency categories of relevance to both practitioners and users of CER were: (1) asking relevant research questions; (2) recognizing or designing ideal CER studies; (3) executing or using CER studies; (4) using appropriate statistical analyses for CER; and (5) communicating and disseminating CER study results to improve health. Although CER is particularly broad concept, we anticipate that these preliminary, relatively generic competencies will be used in tailoring curricula to individual learners from a variety of programmatic perspectives.

Information primary care physicians want to receive about their hospitalized patients.

BACKGROUND AND OBJECTIVES: Communication between physicians caring for hospitalized patients and those patients' primary care providers (PCPs) is often suboptimal, which can lead to diminished health care quality and safety. It is unclear what hospital information PCPs would find most valuable in their patients' continuing care, as is how and when they would prefer to receive such information. METHODS: Using the modified Delphi survey methodology, we developed a consensus list of information items PCPs want to receive about their hospitalized patients, using general internists and family physicians considered experts in primary care. Panelists rated items on a 5-point Likert scale signifying their level of agreement with the information's importance and with the information communication mode. Consensus agreement or disagreement was determined using 95% confidence intervals. RESULTS: Twelve physicians (five family physicians, seven general internists), averaging 19.6 years of primary care experience, participated in Delphi round 1; 41.6% (37 of 89) of the items were accepted by consensus, one item was rejected (receiving daily progress notes), and the remaining 51 items were equivocal. In round 2, nine physician panelists participated (four family physicians, five general internists), and six additional items were accepted. They generally preferred notification at the patient's first hospital interaction and at discharge. No consensus was found regarding communication mode; e-mail was most favored. CONCLUSIONS: We found broad areas of consensus regarding information PCPs wish to receive about their hospitalized patients that are generally consistent with previous surveys. Our findings also suggest that physicians are becoming more comfortable with patient-related electronic communications.

Are hot flashes associated with sleep disturbance during midlife? Results from the STRIDE cohort study.

OBJECTIVE: Sleep disturbance and hot flashes are common during menopause, but their association is not well understood. We sought to understand the associations among sleep disturbance and the frequency, bothersomeness, and interference of hot flashes in mid-life women. STUDY DESIGN: STRIDE is a study of women ages 40-65 years at varied menopausal stages. We examined the cross-sectional associations of sleep disturbance with the frequency and bothersomeness of hot flashes, and interference of hot flashes with work, social, and leisure activities during the 2nd year of STRIDE. MAIN OUTCOME MEASURE: Self-reported sleep disturbance. RESULTS: Of the 623 women with complete data, 370 (59%) reported having hot flashes. Bivariate analyses showed that reporting hot flashes with bother, but not hot flashes alone, was associated with sleep disturbance (odds ratio [OR] [95% confidence interval (CI)]: 2.8 [2.0-4.0] and 1.3 [0.7-2.5], respectively). In multivariable models, women reporting bothersome hot flashes were more likely to report sleep disturbance (OR [95% CI]: 2.1 [1.4-3.2]) compared to women who reported no hot flashes. When the perceived interference of hot flashes with work, social activities, and leisure activities were included in the model, the relationships between bothersome hot flashes and sleep disturbance disappeared. CONCLUSIONS: Hot flashes are not associated with sleep disturbance, unless they are bothersome. Mid-life patients should routinely be queried about the bothersomeness of their hot flashes.

The impact of menopause on health-related quality of life: results from the STRIDE longitudinal study.

PURPOSE: We examine the impact of menopausal status, beyond menopausal symptoms, on health-related quality of life (HRQoL). METHODS: Seven hundred thirty-two women aged 40-65, regardless of health condition or menopausal status, were enrolled from single general internal medicine practice. Women completed annual questionnaires including HRQoL, and menopausal status and symptoms. RESULTS: The physical health composite of the RAND-36 is lower in late peri (45.6, P < .05), early post (45.4, P < .05), and late postmenopausal women (44.6, P < .01), and those who report a hysterectomy (44.2, P < .01) compared to premenopausal women (47.1), with effect sizes of Cohen's d = .12-.23. The mental health composite of the RAND-36 is lower in late peri (44.7, P < .01), early post (44.9, P < .01), and late postmenopausal women (45.0, P < .05) and those who report a hysterectomy (44.2, P < .01) compared to premenopausal women (46.8), with effect sizes of Cohen's d = .15-.20. Findings are comparable adjusted for menopausal symptom frequency and bother. CONCLUSIONS: Over a 5-year follow-up period, we found a negative impact of menopause on some domains of HRQoL, regardless of menopausal symptoms. Clinicians should be aware of this relationship and work to improve HRQoL, rather than expect it to improve spontaneously when menopausal symptoms resolve.