Targeted, actionable and fair: Reviewer reports as feedback and its effect on ECR career choices.

Previous studies of the use of peer review for the allocation of competitive funding agencies have concentrated on questions of efficiency and how to make the 'best' decision, by ensuring that successful applicants are also the more productive or visible in the long term. This paper examines the components of feedback received from an unsuccessful grant application, is associated with motivating applicants career decisions to persist (reapply for funding at T1), or to switch (not to reapply, or else leave academia). This study combined data from interviews with unsuccessful ECR applicants (n = 19) to The Wellcome Trust 2009-19, and manual coding of reviewer comments received by applicants (n = 81). All applicants received feedback on their application at T0 with a large proportion of unsuccessful applicants reapplying for funding at T1. Here, peer-review-comments-as-feedback sends signals to applicants to encourage them to persist (continue) or switch (not continue) even when the initial application has failed. Feedback associated by unsuccessful applicants as motivating their decision to resubmit had three characteristics: actionable; targeted; and fair. The results lead to identification of standards of feedback for funding agencies and peer-reviewers to promote when providing reviewer feedback to applicants as part of their peer review process. The provision of quality reviewer-reports-as-feedback to applicants, ensures that peer review acts as a participatory research governance tool focused on supporting the development of individuals and their future research plans.

The relationship between parenting engagement and academic performance.

Gender differences in research productivity have been well documented. One frequent explanation of these differences is disproportionate child-related responsibilities for women. However, changing social dynamics around parenting has led to fathers taking an increasingly active role in parenting. This demands a more nuanced approach to understanding the relationship between parenting and productivity for both men and women. To gain insight into associations between parent roles, partner type, research productivity, and research impact, we conducted a global survey that targeted 1.5 million active scientists; we received viable responses from 10,445 parents (< 1% response rate), thus providing a basis for exploratory analyses that shed light on associations between parenting models and research outcomes, across men and women. Results suggest that the gendered effect observed in production may be related by differential engagement in parenting: men who serve in lead roles suffer similar penalties for parenting engagement, but women are more likely to serve in lead roles and to be more engaged across time and tasks, therefore suffering a higher penalty. Taking a period of parental leave is associated with higher levels of productivity; however, the productivity advantage dissipates after six months for the US-sample, and at 12-months for the non-US sample. These results suggest that parental engagement is a more powerful variable to explain gender differences in academic productivity than the mere existence of children, and that policies should factor these labor differentials into account.

Boundaries Between Research Ethics and Ethical Research Use in Artificial Intelligence Health Research.

The governance of ethically acceptable research in higher education institutions has been under scrutiny over the past half a century. Concomitantly, recently, decision makers have required researchers to acknowledge the societal impact of their research, as well as anticipate and respond to ethical dimensions of this societal impact through responsible research and innovation principles. Using artificial intelligence population health research in the United Kingdom and Canada as a case study, we combine a mapping study of journal publications with 18 interviews with researchers to explore how the ethical dimensions associated with this societal impact are incorporated into research agendas. Researchers separated the ethical responsibility of their research with its societal impact. We discuss the implications for both researchers and actors across the Ethics Ecosystem.

Population health AI researchers' perceptions of the public portrayal of AI: A pilot study.

This article reports how 18 UK and Canadian population health artificial intelligence researchers in Higher Education Institutions perceive the use of artificial intelligence systems in their research, and how this compares with their perceptions about the media portrayal of artificial intelligence systems. This is triangulated with a small scoping analysis of how UK and Canadian news articles portray artificial intelligence systems associated with health research and care. Interviewees had concerns about what they perceived as sensationalist reporting of artificial intelligence systems - a finding reflected in the media analysis. In line with Pickersgill's concept of 'epistemic modesty', they considered artificial intelligence systems better perceived as non-exceptionalist methodological tools that were uncertain and unexciting. Adopting 'epistemic modesty' was sometimes hindered by stakeholders to whom the research is disseminated, who may be less interested in hearing about the uncertainties of scientific practice, having implications on both research and policy.

Defining ethical standards for the application of digital tools to population health research.

There is growing interest in population health research, which uses methods based on artificial intelligence. Such research draws on a range of clinical and non-clinical data to make predictions about health risks, such as identifying epidemics and monitoring disease spread. Much of this research uses data from social media in the public domain or anonymous secondary health data and is therefore exempt from ethics committee scrutiny. While the ethical use and regulation of digital-based research has been discussed, little attention has been given to the ethics governance of such research in higher education institutions in the field of population health. Such governance is essential to how scholars make ethical decisions and provides assurance to the public that researchers are acting ethically. We propose a process of ethics governance for population health research in higher education institutions. The approach takes the form of review after the research has been completed, with particular focus on the role artificial intelligence algorithms play in augmenting decision-making. The first layer of review could be national, open-science repositories for open-source algorithms and affiliated data or information which are developed during research. The second layer would be a sector-specific validation of the research processes and algorithms by a committee of academics and stakeholders with a wide range of expertise across disciplines. The committee could be created as an off-shoot of an already functioning national oversight body or health technology assessment organization. We use case studies of good practice to explore how this process might operate.

La recherche sur la santé de la population à l'aide de méthodes fondées sur l'intelligence artificielle suscite un intérêt croissant. Ce type de recherche s'appuie sur une série de données cliniques et non cliniques pour prédire les risques sanitaires, par exemple en détectant les épidémies et en surveillant la propagation des maladies. Une part importante de cette recherche emploie des données issues des réseaux sociaux, appartenant au domaine public, ou des données secondaires anonymes relatives à la santé. Par conséquent, elles ne sont soumises à aucun contrôle de la part d'un comité d'éthique. L'usage et les règles déontologiques ont certes fait l'objet de discussions, mais l'attention portée à la gouvernance de l'éthique dans le cadre des recherches que des établissements d'enseignement supérieur ont menées sur la santé des populations reste minime. Pourtant, une telle gouvernance est essentielle pour que les spécialistes puissent prendre des décisions éthiques et garantir au public que les chercheurs agissent dans le respect de la déontologie. Nous proposons un processus de gouvernance éthique pour la recherche sur la santé de la population dans les établissements d'enseignement supérieur. Notre approche consiste à établir un rapport à la fin de la recherche, qui se concentre sur le rôle joué par les algorithmes d'intelligence artificielle dans l'accroissement de la prise de décisions. Le premier niveau de rapport pourrait comporter des registres nationaux accessibles selon le principe de science ouverte pour les algorithmes open-source ainsi que les données ou informations connexes développés durant la recherche. Le second niveau serait composé d'une validation sectorielle des algorithmes et processus de recherche par un comité d'universitaires et d'intervenants possédant une large gamme de compétences dans diverses disciplines. Ce comité pourrait être créé en tant que ramification d'un organisme national de surveillance déjà à l'œuvre, ou d'un organisme d'évaluation des technologies de la santé. Nous utilisons des études de cas pour identifier les bonnes pratiques et découvrir comment ce processus pourrait être appliqué.

Existe un interés creciente en la investigación sanitaria poblacional, que utiliza métodos basados en la inteligencia artificial. Dicha investigación se basa en una serie de datos clínicos y no clínicos para hacer predicciones sobre los riesgos sanitarios, como la identificación de epidemias y el seguimiento de la propagación de enfermedades. Gran parte de esta investigación utiliza los datos de las redes sociales de dominio público o los datos sanitarios secundarios anónimos y, por lo tanto, está exenta del escrutinio del comité de ética. Si bien se ha debatido sobre el uso y la regulación éticos de la investigación basada en tecnología digital, se ha prestado poca atención a la gobernanza ética de dicha investigación en las instituciones de enseñanza superior relacionadas con la salud poblacional. Esa gobernanza es esencial sobre cómo los académicos toman decisiones éticas y ofrece garantías al público de que los investigadores actúan de manera ética. Se propone un proceso de gobernanza ética para la investigación sanitaria poblacional en las instituciones de educación superior. El enfoque adopta la forma de una revisión una vez que la investigación ha sido completada, con especial atención a la función que los algoritmos de inteligencia artificial desempeñan en el aumento de la toma de decisiones. La primera fase de revisión podría consistir en la creación de repositorios nacionales de ciencia abierta para los algoritmos de código abierto y los datos o información afiliados que se desarrollen durante la investigación. La segunda fase consistiría en la validación de los procesos y algoritmos de investigación en un sector específico por parte de un comité de académicos y partes interesadas con una amplia gama de conocimientos especializados en todas las disciplinas. El comité podría crearse como una rama de un organismo nacional de supervisión ya en funcionamiento o de una organización de evaluación sobre tecnologías de la salud. Se utilizan estudios de casos de buenas prácticas para explorar cómo podría funcionar este proceso.

Reasoning "Uncharted Territory": Notions of Expertise Within Ethics Review Panels Assessing Research Use of Social Media.

The fast changing field of social media (SM) research presents unique challenges for research ethics committees (RECs). This article examines notions of experience and expertise in the context of REC members reviewing proposals for SM research and considers the role of the RECs in this area of review. We analyze 19 interviews with REC members to highlight that a lack of personal and professional experience of SM, compounded by a lack of institutional and professional guidelines, mean many REC members feel they do not possess sufficient expertise to review SM research. This view was supported by 14 interviews with SM researchers. REC members drew on strategies to overcome their lack of experience, although most SM researchers still found this problematic, to varying degrees. We recommend several steps to ensure REC expertise in SM research keeps pace of this fast-developing field, taking a pro-active, dialogic approach.

The Ethics Ecosystem: Personal Ethics, Network Governance and Regulating Actors Governing the Use of Social Media Research Data.

This paper examines the consequences of a culture of "personal ethics" when using new methodologies, such as the use of social media (SM) sites as a source of data for research. Using SM research as an example, this paper explores the practices of a number of actors and researchers within the "Ethics Ecosystem" which as a network governs ethically responsible research behaviour. In the case of SM research, the ethical use of this data is currently in dispute, as even though it is seemingly publically available, concerns relating to privacy, vulnerability, potential harm and consent blur the lines of responsible ethical research behaviour. The findings point to the dominance of a personal, bottom-up, researcher-led, 'ethical barometer' for making decisions regarding the permissibility of using SM data. We show that the use of different barometers by different researchers can lead to wide disparities in ethical practice - disparities which are compounded by the lack of firm guidelines for responsible practice of SM research. This has widespread consequences on the development of shared norms and understandings at all levels, and by all actors within the Ethics Ecosystem, and risks inconsistencies in their approaches to ethical decision-making. This paper argues that this governance of ethical behaviour by individual researchers perpetuates a negative cycle of academic practice that is dependent on subjective judgements by researchers themselves, rather than governed by more formalised academic institutions such as the research ethics committee and funding council guidelines.

The Evaluation Scale: Exploring Decisions About Societal Impact in Peer Review Panels.

Realising the societal gains from publicly funded health and medical research requires a model for a reflexive evaluation precedent for the societal impact of research. This research explores UK Research Excellence Framework evaluators' values and opinions and assessing societal impact, prior to the assessment taking place. Specifically, we discuss the characteristics of two different impact assessment extremes - the "quality-focused" evaluation and "societal impact-focused" evaluation. We show the wide range of evaluator views about impact, and that these views could be conceptually reflected in a range of different positions along a conceptual evaluation scale. We describe the characteristics of these extremes in detail, and discuss the different beliefs evaluators had which could influence where they positioned themselves along the scale. These decisions, we argue, when considered together, form a dominant definition of societal impact that influences the direction of its evaluation by the panel.

Comparing research investment to United Kingdom institutions and published outputs for tuberculosis, HIV and malaria: a systematic analysis across 1997-2013.

BACKGROUND: The "Unfinished Agenda" of infectious diseases is of great importance to policymakers and research funding agencies that require ongoing research evidence on their effective management. Journal publications help effectively share and disseminate research results to inform policy and practice. We assess research investments to United Kingdom institutions in HIV, tuberculosis and malaria, and analyse these by numbers of publications and citations and by disease and type of science. METHODS: Information on infection-related research investments awarded to United Kingdom institutions across 1997-2010 were sourced from funding agencies and individually categorised by disease and type of science. Publications were sourced from the Scopus database via keyword searches and filtered to include only publications relating to human disease and containing a United Kingdom-based first and/or last author. Data were matched by disease and type of science categories. Investment (United Kingdom pounds) and publications were compared to generate an 'investment per publication' metric; similarly, an 'investment per citation' metric was also developed as a measure of the usefulness of research. RESULTS: Total research investment for all three diseases was £1.4 billion, and was greatest for HIV (£651.4 million), followed by malaria (£518.7 million) and tuberculosis (£239.1 million). There were 17,271 included publications, with 9,322 for HIV, 4,451 for malaria, and 3,498 for tuberculosis. HIV publications received the most citations (254,949), followed by malaria (148,559) and tuberculosis (100,244). According to UK pound per publication, tuberculosis (£50,691) appeared the most productive for investment, compared to HIV (£61,971) and malaria (£94,483). By type of science, public health research was most productive for HIV (£27,296) and tuberculosis (£22,273), while phase I-III trials were most productive for malaria (£60,491). According to UK pound per citation, tuberculosis (£1,797) was the most productive area for investment, compared to HIV (£2,265) and malaria (£2,834). Public health research was the most productive type of science for HIV (£2,265) and tuberculosis (£1,797), whereas phase I-III trials were most productive for malaria (£1,713). CONCLUSIONS: When comparing total publications and citations with research investment to United Kingdom institutions, tuberculosis research appears to perform best in terms of efficiency. There were more public health-related publications and citations for HIV and tuberculosis than other types of science. These findings demonstrate the diversity of research funding and outputs, and provide new evidence to inform research investment strategies for policymakers, funders, academic institutions, and healthcare organizations.

Reaching "an audience that you would never dream of speaking to": influential public health researchers' views on the role of news media in influencing policy and public understanding.

While governments and academic institutions urge researchers to engage with news media, traditional academic values of public disengagement have inhibited many from giving high priority to media activity. In this interview-based study, the authors report on the views about news media engagement and strategies used by 36 peer-voted leading Australian public health researchers in 6 fields. The authors consider their views about the role and importance of media in influencing policy, their reflections on effective or ineffective media communicators, and strategies used by these researchers about how to best retain their credibility and influence while engaging with the news media. A willingness and capacity to engage with the mass media was seen as an essential attribute of influential public health researchers.

A bibliometric analysis of research on Indigenous health in Australia, 1972-2008.

OBJECTIVE: To determine the growth patterns and citation volume of research publications referring to Indigenous health in Australia from 1972 to 2008 compared to seven selected health fields. METHODS: Web of Science was used to identify all publications (n=820) referring to the health of Indigenous Australians authored by Australian researchers, 1972 to 2008. Citations for each publication were also captured. Growth was compared with selected health fields as well as with overall Australian research publications. RESULTS: Research publications referring to Indigenous health, while remaining relatively small in number, grew at an average annual rate of 14.1%, compared with 8.2% across all fields of Australian research. The growth rate shown was equal second highest in our seven categories of health and medical research. However, Indigenous publications were cited significantly less than the Australian average. CONCLUSIONS: While there has been positive growth in publications referring to Indigenous health, the attention paid to this research through citations remains disappointingly low. IMPLICATIONS: Given that research concentration and impact can be an index of how seriously a nation considers a health problem, the low visibility of Australian research examining Indigenous health does not demonstrate a level of concern commensurate with the gravity of Indigenous health problems. Further investigation for the reasons for lower citations may identify potential intervention strategies.

Identifying trustworthy experts: how do policymakers find and assess public health researchers worth consulting or collaborating with?

This paper reports data from semi-structured interviews on how 26 Australian civil servants, ministers and ministerial advisors find and evaluate researchers with whom they wish to consult or collaborate. Policymakers valued researchers who had credibility across the three attributes seen as contributing to trustworthiness: competence (an exemplary academic reputation complemented by pragmatism, understanding of government processes, and effective collaboration and communication skills); integrity (independence, "authenticity", and faithful reporting of research); and benevolence (commitment to the policy reform agenda). The emphases given to these assessment criteria appeared to be shaped in part by policymakers' roles and the type and phase of policy development in which they were engaged. Policymakers are encouraged to reassess their methods for engaging researchers and to maximise information flow and support in these relationships. Researchers who wish to influence policy are advised to develop relationships across the policy community, but also to engage in other complementary strategies for promoting research-informed policy, including the strategic use of mass media.

Bibliographic analysis of papers and authors published in Tobacco Control 1998-September 2011.

In the present work, the top 20 cited papers published in Tobacco Control between 1998 and 15 September 2011, the top 10 cited papers published after 2008 and the 50 authors whose papers have been most cited in the journal are reported. US authors dominated the most cited papers and the most cited authors, with Australian authors in second place. Papers on youth and secondhand smoke dominated the top 20 papers, although harm reduction and packaging papers appeared in the post 2008 leading cited papers.