Exploring the merits of research performance measures that comply with the San Francisco Declaration on Research Assessment and strategies to overcome barriers of adoption: qualitative interviews with administrators and researchers.

BACKGROUND: In prior research, we identified and prioritized ten measures to assess research performance that comply with the San Francisco Declaration on Research Assessment, a principle adopted worldwide that discourages metrics-based assessment. Given the shift away from assessment based on Journal Impact Factor, we explored potential barriers to implementing and adopting the prioritized measures. METHODS: We identified administrators and researchers across six research institutes, conducted telephone interviews with consenting participants, and used qualitative description and inductive content analysis to derive themes. RESULTS: We interviewed 18 participants: 6 administrators (research institute business managers and directors) and 12 researchers (7 on appointment committees) who varied by career stage (2 early, 5 mid, 5 late). Participants appreciated that the measures were similar to those currently in use, comprehensive, relevant across disciplines, and generated using a rigorous process. They also said the reporting template was easy to understand and use. In contrast, a few administrators thought the measures were not relevant across disciplines. A few participants said it would be time-consuming and difficult to prepare narratives when reporting the measures, and several thought that it would be difficult to objectively evaluate researchers from a different discipline without considerable effort to read their work. Strategies viewed as necessary to overcome barriers and support implementation of the measures included high-level endorsement of the measures, an official launch accompanied by a multi-pronged communication strategy, training for both researchers and evaluators, administrative support or automated reporting for researchers, guidance for evaluators, and sharing of approaches across research institutes. CONCLUSIONS: While participants identified many strengths of the measures, they also identified a few limitations and offered corresponding strategies to address the barriers that we will apply at our organization. Ongoing work is needed to develop a framework to help evaluators translate the measures into an overall assessment. Given little prior research that identified research assessment measures and strategies to support adoption of those measures, this research may be of interest to other organizations that assess the quality and impact of research.

Interdisciplinarity in medical education research: myth and reality.

The medical education (Med Ed) research community characterises itself as drawing on the insights, methods, and knowledge from multiple disciplines and research domains (e.g. Sociology, Anthropology, Education, Humanities, Psychology). This common view of Med Ed research is echoed and reinforced by the narrative used by leading Med Ed departments and research centres to describe their activities as "interdisciplinary." Bibliometrics offers an effective method of investigating scholarly communication to determine what knowledge is valued, recognized, and utilized. By empirically examining whether knowledge production in Med Ed research draws from multiple disciplines and research areas, or whether it primarily draws on the knowledge generated internally within the field of Med Ed, this article explores whether the characterisation of Med Ed research as interdisciplinary is substantiated. A citation analysis of 1412 references from research articles published in 2017 in the top five Med Ed journals was undertaken. A typology of six knowledge clusters was inductively developed. Findings show that the field of Med Ed research draws predominantly from two knowledge clusters: the Applied Health Research cluster (made of clinical and health services research), which represents 41% of the references, and the Med Ed research cluster, which represents 40% of the references. These two clusters cover 81% of all references in our sample, leaving 19% distributed among the other knowledge clusters (i.e., Education, disciplinary, interdisciplinary and topic centered research). The quasi-hegemonic position held by the Applied Health and Med Ed research clusters confines the other sources of knowledge to a peripheral role within the Med Ed research field. Our findings suggest that the assumption that Med Ed research is an interdisciplinary field is not convincingly supported by empirical data and that the knowledge entering Med Ed comes mostly from the health research domain.

Problematizing assumptions about interdisciplinary research: implications for health professions education research.

This article critically examines three assumptions underlying recent efforts to advance interdisciplinary research-defined in this article as communication and collaboration between researchers across academic disciplines (e.g. Sociology, Psychology, Biology)-and examines these assumptions' implications for health professions education research (HPER). These assumptions are: (1) disciplines are silos that inhibit the free flowing of knowledge across fields and stifle innovative thinking; (2) interdisciplinary research generates a better understanding of the world as it brings together researchers from various fields of expertise capable of tackling complex problems; and (3) interdisciplinary research reduces fragmentation across groups of researchers by eliminating boundaries. These assumptions are among the new beliefs shaping the contemporary academic arena; they orient academics' and university administrators' decisions toward expanding interdisciplinary research and training, but without solid empirical evidence. This article argues that the field of HPER has largely adopted the premises of interdisciplinary research but has not yet debated the potential effects of organizing around these premises. The authors hope to inspire members of the HPER community to critically examine the ubiquitous discourse promoting interdisciplinarity, and engage in reflection about the future of the field informed by evidence rather than by unsubstantiated assumptions. For example: Should research centres and graduate programs in HPER encourage the development of interdisciplinary or disciplinary-trained researchers? Should training predominantly focus on methods and methodologies or draw more on disciplinary-based knowledge? What is the best route toward increasing the field's profile within academia and attracting the best students and researchers to engage in HPER? These are questions that merit attention at the current juncture as the future of the HPER field relies on decisions made in the present time.

Examining grounded theory through the lens of rationalist epistemology.

The objective of scientific, or more broadly, academic knowledge is to provide an understanding of the social and natural world that lies beyond common sense and everyday thinking. Academics use an array of techniques, methods and conceptual apparatuses to achieve this goal. The question we explore in this essay is the following: Does the grounded theory approach, in the constructivist version developed by Kathy Charmaz, provide the necessary methodological tools for the creation of knowledge and theories beyond everyday thinking? To conduct our analysis, we have drawn on the rationalist epistemology originally developed by Gaston Bachelard and taken up a few decades later by Pierre Bourdieu and colleagues to look at the epistemological foundation of the CGT methods as defined by Charmaz. We focussed on two distinctive epistemological features characterising constructivist grounded theory (CGT): the use of inductive reasoning to generate interpretative theory; and the primacy of subjectivity over objectivity as the preferred path to knowledge making. While the usefulness of CGT for conducting qualitative research and understanding the perspective of social actors has been acknowledged by scholars in health professions education research and other research areas, the inductivist logic on which it draws raises questions concerning the nature of the knowledge yielded by this approach. As we argue in this article, it is still unclear in what way the interpretative theory generated by CGT is not a duplication of everyday thinking expressed through meta-narratives. It is also unclear how the understanding of social phenomena can be refined if the use of inductive procedures logically implies the creation of a new theory each time a study is conducted. We engage with these questions to broaden the epistemological conversation within the health professions education research community. It is our hope that scholars in the field will engage in this epistemological conversation and advance it in new directions.

DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey.

OBJECTIVE: The San Francisco Declaration on Research Assessment (DORA) advocates for assessing biomedical research quality and impact, yet academic organizations continue to employ traditional measures such as Journal Impact Factor. We aimed to identify and prioritize measures for assessing research quality and impact. METHODS: We conducted a review of published and grey literature to identify measures of research quality and impact, which we included in an online survey. We assembled a panel of researchers and research leaders, and conducted a two-round Delphi survey to prioritize measures rated as high (rated 6 or 7 by ≥ 80% of respondents) or moderate (rated 6 or 7 by ≥ 50% of respondents) importance. RESULTS: We identified 50 measures organized in 8 domains: relevance of the research program, challenges to research program, or productivity, team/open science, funding, innovations, publications, other dissemination, and impact. Rating of measures by 44 panelists (60%) in Round One and 24 (55%) in Round Two of a Delphi survey resulted in consensus on the high importance of 5 measures: research advances existing knowledge, research plan is innovative, an independent body of research (or fundamental role) supported by peer-reviewed research funding, research outputs relevant to discipline, and quality of the content of publications. Five measures achieved consensus on moderate importance: challenges to research productivity, potential to improve health or healthcare, team science, collaboration, and recognition by professional societies or academic bodies. There was high congruence between researchers and research leaders across disciplines. CONCLUSIONS: Our work contributes to the field by identifying 10 DORA-compliant measures of research quality and impact, a more comprehensive and explicit set of measures than prior efforts. Research is needed to identify strategies to overcome barriers of use of DORA-compliant measures, and to "de-implement" traditional measures that do not uphold DORA principles yet are still in use.

What do we do? Practices and learning strategies of medical education leaders.

BACKGROUND: Continuous changes in undergraduate and postgraduate medical education require faculty to assume a variety of new leadership roles. While numerous faculty development programmes have been developed, there is little evidence about the specific practices of medical education leaders or their learning strategies to help inform their design. AIM: This study aimed to explore what medical education leaders' actually do, their learning strategies and recommendations for faculty development. METHOD: A total of 16 medical education leaders from a variety of contexts within the faculty of medicine of a large North American medical school participated in semi-structured interviews to explore the nature of their work and the learning strategies they employ. Using thematic analysis, interview transcripts were coded inductively and then clustered into emergent themes. RESULTS: Findings clustered into four key themes of practice: (1) intrapersonal (e.g., self-awareness), (2) interpersonal (e.g., fostering informal networks), (3) organizational (e.g., creating a shared vision) and (4) systemic (e.g. strategic navigation). Learning strategies employed included learning from experience and example, reflective practice, strategic mentoring or advanced training. CONCLUSIONS: Our findings illuminate a four-domain framework for understanding medical education leader practices and their learning preferences. While some of these findings are not unknown in the general leadership literature, our understanding of their application in medical education is unique. These practices and preferences have a potential utility for conceptualizing a coherent and relevant approach to the design of faculty development strategies for medical education leadership.

Meanings and perceptions of patient-centeredness in social work, nursing and medicine: a comparative study.

Answering patients' evolving, more complex needs has been recognized as a main incentive for the development of interprofessional care. Thus, it is not surprising that patient-centered practice (PCP) has been adopted as a major outcome for interprofessional education. Nevertheless, little research has focused on how PCP is perceived across the professions. This study aimed to address this issue by adopting a phenomenological approach and interviewing three groups of professionals: social workers (n = 10), nurses (n = 10) and physicians (n = 8). All the participants worked in the same department (the General Internal Medicine department of a university affiliated hospital). Although the participants agreed on a core meaning of PCP as identifying, understanding and answering patients' needs, they used many dimensions to define PCP. Overall, the participants expressed value for PCP as a philosophy of care, but there was the sense of a hierarchy of patient-centeredness across the professions, in which both social work and nursing regarded themselves as more patient-centered than others. On their side, physicians seemed inclined to accept their lower position in this hierarchy. Gieryn's concept of boundary work is employed to help illuminate the nature of PCP within an interprofessional context.

The CanMEDS role of Collaborator: How is it taught and assessed according to faculty and residents?

OBJECTIVE: To explore the perspectives of paediatric residents and faculty regarding how the Collaborator role is taught and assessed. METHODS: Using a constructivist grounded theory approach, focus groups at four Canadian universities were conducted. Data were analyzed iteratively for emergent themes. RESULTS: Residents reported learning about collaboration through faculty role modelling but did not perceive that it was part of the formal curriculum. Faculty reported that they were not trained in how to effectively model this role. Both groups reported a need for training in conflict management, particularly as it applies to intraprofessional (physician-to-physician) relationships. Finally, the participants asserted that current methods to assess residents on their performance as collaborators are suboptimal. CONCLUSIONS: The Collaborator role should be a formal part of the residency curriculum. Residents need to be better educated with regard to managing conflict and handling intraprofessional relationships. Finally, innovative methods of assessing residents on this non-medical expert role need to be created.

OBJECTIF: Explorer les points de vue des résidents et des professeurs de pédiatrie sur l'enseignement et l'évaluation du rôle de collaborateur. MÉTHODOLOGIE: Au moyen d'une démarche axée sur la théorie constructiviste, les chercheurs ont formé des groupes de travail dans quatre universités canadiennes. Ils ont analysé les données de manière itérative afin d'en faire ressortir les thèmes émergents. RÉSULTATS: Les résidents ont déclaré apprendre la collaboration par l'exemple des professeurs, mais n'avaient pas l'impression qu'elle faisait partie du programme officiel. Les professeurs ont déclaré ne pas avoir de formation sur la manière efficace de démontrer ce rôle. Les deux groupes ont rendu compte d'un besoin de formation en gestion de conflits, notamment dans le cadre des relations intraprofessionnelles (entre médecins). Enfin, les participants ont fait valoir que les méthodes actuelles d'évaluation du rendement des résidents à titre de collaborateurs sont sous-optimales. CONCLUSIONS: Le rôle de collaborateur devrait faire partie intégrante du programme de résidence. Les résidents doivent être mieux formés en matière de gestion de conflits et de relations intraprofessionnelles. Enfin, il faut créer des méthodes novatrices d'évaluation des résidents à l'égard de ce rôle d'expert non médical.

Barriers to cross-disciplinary knowledge flow: The case of medical education research.

INTRODUCTION: The medical education research field operates at the crossroads of two distinct academic worlds: higher education and medicine. As such, this field provides a unique opportunity to explore new forms of cross-disciplinary knowledge exchange. METHODS: Cross-disciplinary knowledge flow in medical education research was examined by looking at citation patterns in the five journals with the highest impact factor in 2017. To grasp the specificities of the knowledge flow in medical education, the field of higher education was used as a comparator. In total, 2031 citations from 64 medical education and 41 higher education articles published in 2017 were examined. RESULTS: Medical education researchers draw on a narrower range of knowledge communities than their peers in higher education. Medical education researchers predominantly cite articles published in health and medical education journals (80% of all citations), and to a lesser extent, articles published in education and social science journals. In higher education, while the largest share of the cited literature is internal to the domain (36%), researchers cite literature from across the social science spectrum. Findings suggest that higher education scholars engage in conversations with academics from a broader range of communities and perspectives than their medical education colleagues. DISCUSSION: Using Pierre Bourdieu's concepts of doxa and field, it is argued that the variety of epistemic cultures entering the higher education research space contributes to its interdisciplinary nature. Conversely, the existence of a relatively homogeneous epistemic culture in medicine potentially impedes cross-disciplinary knowledge exchange.

The future of medical education: a Canadian environmental scan.

OBJECTIVES: One hundred years after the Flexner report remade medical education in North America, many countries are reviewing the purpose and organisation of medical education. In Canada, a national study is being undertaken to define important issues and challenges for the future of medical education. The objectives of this paper are to describe the process of conducting an empirical environmental scan at a national level, and to present the research findings of this scan. METHODS: Thirty national key informant interviews were conducted, transcribed and coded to identify key themes. Interview data were triangulated with data sourced from 34 commissioned literature reviews and a series of national focus groups. RESULTS: Ten key issues or priorities were identified and used to generate detailed review papers used by the Association of Faculties of Medicine of Canada to create a blueprint for the evolution of medical education. The new priorities have major implications for areas ranging from admissions, curriculum content, educational process and the need to articulate the purpose and responsibilities of medical schools in society. DISCUSSION: This research provides a case study of how an empirical research approach can be used to identify and validate priorities for changes in medical education at a national level. This approach may be of interest in other countries.

When all else fails: The (mis)use of qualitative research in the evaluation of complex interventions.

RATIONAL, AIMS, AND OBJECTIVES: Qualitative research has been promoted as an important component of the evaluation of complex interventions to support the scale up and spread of health service interventions, but is currently not being maximized in practice. We aim to identify and explore the sociocultural and structural factors that impact the uses (and misuses) of qualitative research in the evaluation of complex health services interventions. METHODS: We conducted a qualitative analysis of data collected in a multiple case study of the evaluation and scale up and spread of three health service intervention. RESULTS: Our findings demonstrate the challenges of meaningfully integrating qualitative research in evaluation programmes lead by clinicians with limited qualitative expertise and operating within an environment dominated by biomedical research, even with methodological support. CONCLUSIONS: Based on these findings we encourage ongoing engagement of qualitative researchers in evaluation programmes to begin to refine our methodological understanding, while also suggesting changes to medical education and evaluation funding models to create fertile environments for interdisciplinary collaborations.

Understanding decisions to scale up: a qualitative case study of three health service intervention evaluations.

OBJECTIVE: Efforts to scale up evidence-based health care interventions are seen as a key strategy to address complex health system challenges. However, scale-up efforts have shown significant variability. We address the gap between scale-up theory and practice by exploring the socio-cultural factors at play in the evaluation and scale-up of three interventions within the clinical field. METHODS: A qualitative multiple case study was conducted to characterize the evaluation and scale-up efforts of three interventions. We interviewed 18 participants, including clinicians and researchers across the three cases. Using Pierre Bourdieu's concepts of field and capital as a theoretical lens, we conducted a thematic analysis of the data. RESULTS: Despite the espoused goals of ensuring that health service interventions are always based on high-quality evidence within the clinical field, this study demonstrates that the outcomes of the evaluations are not the only factor in the decision to engage in scale-up efforts. Important socio-cultural factors also come into play. Bourdieu uses the term capital to refer to the resources that agents compete for and with their acquisition, accumulate power and/or social standing. The type of evidence valued in the clinical field and the ability to leverage capital in demonstrating that value are also important factors. CONCLUSIONS: Determining if an intervention is effective and should be scaled up is more complex in practice than described in the literature. Efforts are needed to explicitly include the role of social processes in the current frameworks guiding scaling-up efforts.

Perspectives of clinician and biomedical scientists on interdisciplinary health research.

BACKGROUND: Interdisciplinary health research is a priority of many funding agencies. We surveyed clinician and biomedical scientists about their views on the value and funding of interdisciplinary health research. METHODS: We conducted semistructured interviews with 31 biomedical and 30 clinician scientists. The scientists were selected from the 2000-2006 membership lists of peer-review committees of the Canadian Institutes of Health Research. We investigated respondents' perspectives on the assumption that collaboration across disciplines adds value to health research. We also investigated their perspectives on funding agencies' growing support of interdisciplinary research. RESULTS: The 61 respondents expressed a wide variety of perspectives on the value of interdisciplinary health research, ranging from full agreement (22) to complete disagreement (11) that it adds value; many presented qualified viewpoints (28). More than one-quarter viewed funding agencies' growing support of interdisciplinary research as appropriate. Most (44) felt that the level of support was unwarranted. Arguments included the belief that current support leads to the creation of artificial teams and that a top-down process of imposing interdisciplinary structures on teams constrains scientists' freedom. On both issues we found contrasting trends between the clinician and the biomedical scientists. INTERPRETATION: Despite having some positive views about the value of interdisciplinary research, scientists, especially biomedical scientists, expressed reservations about the growing support of interdisciplinary research.

Biomedical scientists' perception of the social sciences in health research.

The growing interest in interdisciplinary research within the Canadian health sciences sector has been manifested by initiatives aimed at increasing the involvement of the social sciences in this sector. Drawing on Bourdieu's concept of field and Knorr-Cetina's concept of epistemic culture, this study explores the extent to which it is possible for the social sciences to integrate into, and thrive in, a field in which the experimental paradigm occupies a hegemonic position. Thirty-one semi-structured interviews were conducted to explore biomedical scientists' receptiveness toward the social sciences in general and to qualitative research in particular. We found that these respondents exhibited a predominantly negative posture toward the social sciences; however, we also found considerable variation in their judgments and explanations. Eight biomedical scientists tended to be receptive to the social sciences, 7 ambivalent, and 16 unreceptive. The main rationale expressed by receptive respondents is that the legitimacy of a method depends on its capacity to adequately respond to a research question and not on its conformity to the experimental canon. Unreceptive respondents maintained that the social sciences cannot generate valid and reliable results because they are not conducive to the experimental design as a methodological approach. Ambivalent respondents were characterized by their cautiously accepting posture toward the social sciences and, especially, by their reservations about qualitative methods. Based on the biomedical scientists' limited receptiveness, we can anticipate that the growth of the social sciences will continue to meet obstacles within the health research field in the near future in Canada.

Confined to a tokenistic status: Social scientists in leadership roles in a national health research funding agency.

The idea of interdisciplinarity has been taken up by academic and governmental organisations around the world and enacted through science policies, funding programs and higher education institutions. In Canada, interdisciplinarity led to a major transformation in health research funding. In 2000, the federal government closed the Medical Research Council (MRC) and created the Canadian Institutes of Health Research (CIHR). From the outset, CIHR's vision and goals were innovative, as it sought to include the social sciences within its purview alongside more traditional health research sectors. The extent to which it has been successful in this endeavour, however, remains unknown. The aim of our study was to examine how CIHR's intentions to foster inclusiveness and cooperation across disciplines were implemented in the agency's own organisational structure. We focused on social scientists' representation on committees and among decision-makers between 2000 and 2015, one of the key mandates of CIHR being to include the social sciences within its remit and support research in this area. We examined the composition of the Governing Council, the Institute Scientific Directors, the Chairs of the College of Reviewers, and two International Review Panels invited by CIHR. We targeted these committees and decision-makers since they hold the power to influence the field of Canadian health research through the decisions they make. Our findings show that, while CIHR was created with the mandate to support the entire spectrum of health-related research-including the social sciences-this call for inclusiveness has not yet been materialized in the agency's organisational structure. Social scientists, as well as researchers from neighbouring disciplines such as social epidemiology, health promotion and the humanities, are still confined to low levels of representation within CIHR's highest echelons. This imbalance limits social scientists' input into health research in Canada and undermines CIHR's interdisciplinary ambition.

Interdisciplinary promises versus practices in medicine: the decoupled experiences of social sciences and humanities scholars.

This paper explores social scientists' and humanities (SSH) scholars' integration within the academic medical research environment. Three questions guided our investigation: Do SSH scholars adapt to the medical research environment? How do they navigate their career within a culture that may be inconsistent with their own? What strategies do they use to gain legitimacy? The study builds on three concepts: decoupling, doxa, and epistemic habitus. Twenty-nine semi-structured interviews were conducted with SSH scholars working in 11 faculties of medicine across Canada. Participants were selected through purposeful and snowball sampling. The data were analyzed by thematic content analysis. For most of our participants, moving into medicine has been a challenging experience, as their research practices and views of academic excellence collided with those of medicine. In order to achieve some level of legitimacy more than half of our participants altered their research practices. This resulted in a dissonance between their internalized appreciation of academic excellence and their new, altered, research practices. Only six participants experienced no form of challenge or dissonance after moving into medicine, while three decided to break with their social science and humanities past and make the medical research community their new home. We conclude that the work environment for SSH scholars in faculties of medicine does not deliver on the promise of inclusiveness made by calls for interdisciplinarity in Canadian health research.

Understanding the debate on medical education research: a sociological perspective.

Since the mid-1990s, a debate has taken place among medical education scholars regarding the forms that research should take and the roles it should play. Editors of major journals in medical education and prominent researchers in the domain have repeatedly addressed the issue and have attempted to define what medical education research should be. The goal of this article is to look at the debate from a sociological perspective and to outline the social factors shaping it. An analysis of the texts published since 1990 addressing the issue shows that the debates can be deconstructed in four topics: epistemology, methodology, the primary purpose of medical education research, and the "quality" of the projects carried out in the domain. However, the debates can also be amalgamated and synthesized using the concept of "field" as developed by sociologist Pierre Bourdieu. A "field" refers to the configuration of power relations among individuals, social groups, or institutions within a domain of activities. Scientific fields are typically structured around a "bipolar" opposition pattern. At one pole stand those individuals who promote greater collaboration with nonscientists as well as research aimed at responding to practical needs. At the opposite pole stand those individuals who aspire to achieve independence of the field from such external constraints. The use of the concept of "field" allows us to understand the debate from a larger perspective and to establish parallels with similar debates in other scientific fields. In doing so, we will have the opportunity to learn from the experience of these other fields and be more reflective about the debate in which we engage.