A participatory approach to evaluating a national training and institutional change initiative: the BUILD longitudinal evaluation.

BACKGROUND AND PURPOSE: The National Institutes of Health (NIH) funds training programs to increase the numbers and skills of scientists who obtain NIH research grants, but few programs have been rigorously evaluated. The sizeable recent NIH investment in developing programs to increase the diversity of the NIH-funded workforce, implemented through the Diversity Program Consortium (DPC), is unusual in that it also funds a Consortium-wide evaluation plan, which spans the activities of the 10 BUilding Infrastructure Leading to Diversity (BUILD) awardees and the National Research Mentoring Network (NRMN). The purpose of this article is to describe the evaluation design and innovations of the BUILD Program on students, faculty, and institutions of the 10 primarily undergraduate BUILD sites. KEY HIGHLIGHTS OF THE PROJECT: Our approach to this multi-methods quasi-experimental longitudinal evaluation emphasizes stakeholder participation and collaboration. The evaluation plan specifies the major evaluation questions and key short- to long-term outcome measures (or Hallmarks of Success). The Coordination and Evaluation Center (CEC) embarked on a comprehensive evaluation strategy by developing a set of logic models that incorporate the Hallmarks of Success and other outcomes that were collaboratively identified by the DPC. Data were collected from each BUILD site through national surveys from the Higher Education Research Institute at UCLA (HERI), annual followup surveys that align with the HERI instruments, site visits and case studies, program encounter data ("tracker" data), and institutional data. The analytic approach involves comparing changes in Hallmarks (key outcomes) within institutions for biomedical students who participated versus those who did not participate in the BUILD program at each institution, as well as between institution patterns of biomedical students at the BUILD sites, and matched institutions that were not BUILD grantees. Case studies provide insights into the institutionalization of these new programs and help to explain the processes that lead to the observed outcomes. IMPLICATIONS: Ultimately, the results of the consortium-wide evaluation will be used to inform national policy in higher education and will provide relevant examples of institutional and educational programmatic changes required to diversify the biomedical workforce in the USA.

NIH Career Development Awards in Clinical and Translational Science Award institutions: distinguishing characteristics of top performing sites.

PURPOSE: To identify and disseminate the organizational characteristics of "top performing" National Institute of Health (NIH) Clinical and Translational Science Awards (CTSA) institutions in regards to career development, using the number of new K awards received per year to rank institutions and comparing these with non-CTSA institutions. METHODS: The authors analyzed the organizational characteristics of all 61 CTSA institutions from 2006 to 2013 using the American Association of Medical Colleges Organizational Characteristics Database and K Award funding details using NIH RePORT. RESULTS: Five of the "top 10 performing" institutions are in the western region, and six out of the ten are public schools. Three of the "top 10 performing" institutions receive most of their K awards through two funding mechanisms-the K08 (mentored clinical scientist research award) and K23 (mentored patient-oriented research career development awards). Notably, these three institutions lack a KL2 program. CONCLUSION: The CTSA network of institutions is committed to developing the next generation of physician scientists in order to meet the pressing health needs of society. Educators and evaluators within this network may need to provide training to junior investigators beyond the traditional KL2 programs in order to advance their career development as physician scientists and clinical translational researchers.

Neighborhood socio-economic disadvantage and race/ethnicity as predictors of breast cancer stage at diagnosis.

BACKGROUND: This study investigated the role of key individual- and community-level determinants to explore persisting racial/ethnic disparities in breast cancer stage at diagnosis in California during 1990 and 2000. METHODS: We examined socio-demographic determinants and changes in breast cancer stage at diagnosis in California during 1990 and 2000. In situ, local, regional, and distant diagnoses were examined by individual (age, race/ethnicity, and marital status) and community (income and education by zip code) characteristics. Community variables were constructed using the California Cancer Registry 1990-2000 and the 1990 and 2000 U.S. Census. RESULTS: From 1990 to 2000, there was an overall increase in the percent of in situ diagnoses and a significant decrease in regional and distant diagnoses. Among white and Asian/Pacific Islander women, a significant percent increase was observed for in situ diagnoses, and significant decreases in regional and distant diagnoses. Black women had a significant decrease in distant -stage diagnoses, and Hispanic women showed no significant changes in any diagnosis during this time period. The percent increase of in situ cases diagnosed between 1990 and 2000 was observed even among zip codes with low income and education levels. We also found a significant percent decrease in distant cases for the quartiles with the most poverty and least education. CONCLUSIONS: Hispanic women showed the least improvement in breast cancer stage at diagnosis from 1990 to 2000. Breast cancer screening and education programs that target under-served communities, such as the rapidly growing Hispanic population, are needed in California.

Identifying and Developing Leadership Competencies in Health Research Organizations: A Pilot Study.

We investigated leadership competencies for developing senior and emerging leaders and the perceived effectiveness of leadership development programs in Health Research Organizations (HROs). A pilot study was conducted to interview HRO executives in Southern California. Respondents represented different organizational contexts to ensure a diverse overview of strategic issues, competencies, and development needs. We analyzed qualitative and quantitative data using an innovative framework for analyzing HRO leadership development. The National Center for Healthcare Leadership 'Health Leadership Competency Model' was used as the foundation of our competency research. Top strategic issues included economic downturn and external funding, the influence of governmental policies and regulations, operating in global markets, and forming strategic alliances. High priority NCHL leadership competencies required to successfully lead an HRO include talent development, collaboration, strategic orientation, and team leadership. Senior executives need financial skills and scientific achievement; emerging leaders need technical/scientific competence, information seeking, and a strong work ethic. About half of the respondents reported having no leadership development program (LDP). Almost all reported their organization encourages mentoring, but less than one-third reported an active formalized mentoring program. We conclude that uncertainties and challenges related to healthcare reform and the continued budget deficits will require HRO restructuring to contain costs, remove barriers to innovation, and show value-add in accelerating discovery to improve clinical care, patient outcomes, and community health. Successful leaders will need to become more strategic, entrepreneurial, and resourceful in developing research alliances, executing research operations, and continually improving performance at all levels of the HRO.

Who will serve? Assessing recruitment of underrepresented minority and low-income dental students to increase access to dental care.

This article uses data from the 2007 American Dental Education Association survey of dental school seniors to assess their intentions to serve underserved populations according to the students' underrepresented minority (URM) and income status. Dental school recruitment and retention programs that concentrate exclusively on URM students will not benefit most low-income students since 83 percent of them are not URM. Recruiting URM students leads to more graduating students with intentions to serve minorities. Whether the income of URM students was high or low, about half in each income group stated that more than 25 percent of their patients would be underserved minorities, compared to 28 percent of the low-income non-URM students and 17 percent of the higher income non-URM students. However, our multivariable results suggest that recruitment of both low-income groups (URM and non-URM) rather than high income regardless of ethnicity might be especially helpful in producing graduates who choose public service. URM/income status was not significantly related to serving special care or rural populations.

What enhances underrepresented minority recruitment to dental schools?

This study examined the factors influencing the proportion of underrepresented minority students (URM) in dental schools. Using a comprehensive recruitment model, it considered the relative importance of community characteristics (population demographics, oral health policies, dental care system, and university environment), dental school characteristics (Pipeline-supported, mission, and financing), and community-based dental education (CBDE) characteristics of the dental school on recruitment of URM students. Data come from a national survey of dental school seniors and a variety of publicly available sources. Three outcome variables measure URM recruitment: percent URM, percent Hispanic, and percent African American in the first year of dental school. Multivariable results revealed that the most important factors predicting a higher percent URM in first-year classes were a higher proportion of URM clinical faculty and graduating students' perceptions that their clinical rotation experience improved their ability to care for diverse groups. For percent Hispanic in the first year, a higher proportion of URM clinical faculty and students spending more time in clinical rotations predicted greater Hispanic recruitment. Graduating students' perceptions that they were less prepared to treat diverse groups were directly associated with the proportion of Hispanic students in the class. For a higher percent of African Americans in the first-year class, the most important factors were a higher proportion of blacks in the county, support from the national Pipeline program, and graduating students' perceptions of better preparedness to integrate cultural differences into treatment planning. Higher total financial aid awarded by the school was negatively associated with recruitment of African Americans. Results suggest some improved URM recruitment strategies for dental schools.

Effect of school environment on dental students' perceptions of cultural competency curricula and preparedness to care for diverse populations.

Reports of oral health disparities among racial, ethnic, and socioeconomic sectors of the U.S. population have hastened development of strategies to address this issue. Among these strategies is revising dental school curricula in order to develop more culturally competent graduates. The present study uses data from the 2003 American Dental Education Association (ADEA) survey of dental school seniors to assess students' perceptions of the adequacy of their cultural competency training. We hypothesize that these perceptions are influenced by multiple student characteristics and contextual factors, including a school's status with respect to the Pipeline, Professions, and Practice initiative of the Robert Wood Johnson Foundation and The California Endowment. Response data from ADEA survey items reflecting student perceptions of adequacy of curriculum time devoted to cultural competency and their preparedness to treat an ethnically and culturally diverse population were analyzed to assess the influence of selected student and contextual factors. Student gender, marital status, and level of educational debt were found to influence the curriculum time variable, and students at California schools reported higher perceived preparedness levels than students at dental schools nationwide. Dental school environments promoting acceptance and respect of diverse ethnicities/cultures and student race/ethnicity were significantly associated with students' perception of the adequacy of curriculum time for cultural competency and students' perception of their preparedness to provide oral health care for racially and culturally diverse groups. The findings provide insight for development of cultural competency curricula and direction for future study in this area.

Influence of contextual environment and community-based dental education on practice plans of graduating seniors.

This study investigated senior dental students' plans to provide care to underserved racial/ethnic minority populations. Three sets of determinants were analyzed: contextual environment, community-based dental education (CBDE), and student characteristics. We analyzed data from the ADEA Survey of Dental School Seniors and administrative data sources to construct contextual variables. Multivariable results show three contextual variables predicted practice plans: greater numbers of federally qualified health centers, higher percentages of underrepresented minorities, and attending a California Pipeline dental school. Regarding CBDE predictors, it was alarming to find seniors who viewed the cultural competency curriculum as inadequate and perceived themselves as less prepared to provide oral health care to diverse populations were also those most likely to serve minority patients. Significant student characteristics included racial/ethnic minority, female gender, older age, lower parent's income, and socially conscious orientation. The study provides evidence that contextual environment, CBDE, and student characteristics were significantly associated with plans to care for underserved patients. Findings suggest if the Pipeline initiative is successful in stimulating reform in U.S. dental schools, future students will develop greater awareness regarding critical access problems and the competencies required to effectively care for diverse populations. In the long term, addressing the problem of dental care access will require the creation of policy, financial, and structural interventions to motivate providers to care for the underserved.