Measuring inconsistency in research ethics committee review.

BACKGROUND: The review of human participant research by Research Ethics Committees (RECs) or Institutional Review Boards (IRBs) is a complex multi-faceted process that cannot be reduced to an algorithm. However, this does not give RECs/ IRBs permission to be inconsistent in their specific requirements to researchers or in their final opinions. In England the Health Research Authority (HRA) coordinates 67 committees, and has adopted a consistency improvement plan including a process called "Shared Ethical Debate" (ShED) where multiple committees review the same project. Committee reviews are compared for consistency by analysing the resulting minutes. METHODS: We present a description of the ShED process. We report an analysis of minutes created by research ethics committees participating in two ShED exercises, and compare them to minutes produced in a published "mystery shopper" exercise. We propose a consistency score by defining top themes for each exercise, and calculating the ratio between top themes and total themes identified by each committee for each ShED exercise. RESULTS: Our analysis highlights qualitative differences between the ShED 19, ShED 20 and "mystery shopper" exercises. The quantitative measure of consistency showed only one committee across the three exercises with more than half its total themes as top themes (ratio of 0.6). The average consistency scores for the three exercises were 0.23 (ShED19), 0.35 (ShED20) and 0.32 (mystery shopper). There is a statistically significant difference between the ShED 19 exercise, and the ShED 20 and mystery shopper exercises. CONCLUSIONS: ShED exercises are effective in identifying inconsistency between ethics committees and we describe a scoring method that could be used to quantify this. However, whilst a level of inconsistency is probably inevitable in research ethics committee reviews, studies must move beyond the ShED methodology to understand why inconsistency occurs, and what an acceptable level of inconsistency might be.

Determining the level of data sharing, and number of publications, from research databases that have been given a favourable opinion by UK research ethics committees.

OBJECTIVE: To determine data sharing and number of publications coming from research databases that have been given a favourable opinion by UK National Health Service (NHS) Research Ethics Committees (RECs). DESIGN: Cohort study. INCLUSION CRITERIA & SETTING: All research databases listed on the UK Health Research Authority's Assessment Review Portal (HARP) that had received a favourable ethics opinion as of January 2018. MAIN OUTCOME MEASURES: Publications and data access requests are either listed on HARP or notified through subsequent email correspondence. RESULTS: Out of 354 eligible databases, 34% had granted access requests and 40% had produced at least one peer-reviewed paper or conference abstract/talk. We could not establish contact with 9% of databases, and 19% reported no access requests or publications. Only 9% of databases were up to date with all annual reports. Email responses from database owners showed a range of attitudes towards data sharing. CONCLUSION: Less than half of research databases that have received a favourable opinion from NHS research ethics committees share their data and produce publications. There is also considerable variability in the operation of research databases and understanding of the purpose of research databases. This work was hampered by incomplete records due mainly to researchers not submitting annual reports.