Fundamental challenges in assessing the impact of research infrastructure.

Clinical research infrastructure is one of the unsung heroes of the scientific response to the current COVID-19 pandemic. The extensive, long-term funding into research support structures, skilled people, and technology allowed the United Kingdom research response to move off the starting blocks at pace by utilizing pre-existing platforms. The increasing focus from funders on evaluating the outcomes and impact of research infrastructure investment requires both a reframing and progression of the current models in order to address the contribution of the underlying support infrastructure. The majority of current evaluation/outcome models focus on a "pipeline" approach using a methodology which follows the traditional research funding route with the addition of quantitative metrics. These models fail to embrace the complexity caused by the interplay of previous investment, the coalescing of project outputs from different funders, the underlying infrastructure investment, and the parallel development across different parts of the system. Research infrastructure is the underpinning foundation of a project-driven research system and requires long-term, sustained funding and capital investment to maintain scientific and technological expertise. Therefore, the short-term focus on quantitative metrics that are easy to collect and interpret and that can be assessed in a roughly 5-year funding cycle needs to be addressed. The significant level of investment in research infrastructure necessitates investment to develop bespoke methodologies that develop fit-for-purpose, longer-term/continual approach(es) to evaluation. Real-world research should reflect real-world evaluation and allow for the accrual of a narrative of value indicators that build a picture of the contribution of infrastructure to research outcomes. The linear approach is not fit for purpose, the research endeavour is a complex, twisted road, and the evaluation approach needs to embrace this complexity through the development of realist approaches and the rapidly evolving data ecosystem. This paper sets out methodological challenges and considers the need to develop bespoke methodological approaches to allow a richer assessment of impact, contribution, attribution, and evaluation of research infrastructure. This paper is the beginning of a conversation that invites the community to "take up the mantle" and tackle the complexity of real-world research translation and evaluation.

EVITA 2.0, an updated framework for understanding evidence-based mental health policy agenda-setting: tested and informed by key informant interviews in a multilevel comparative case study.

BACKGROUND: Mental health remains a neglected issue on the global health policy agenda, particularly in low- and middle-income countries (LMIC), and the translation of research evidence into policy and practice is slow. The new EVITA framework was developed to improve mental health evidence uptake and policy agenda-setting in LMICs. In addition, behavioural science methods may be able to support knowledge translation to policy. METHODS: Using a mixed-methods study design, we applied and tested the newly developed EVITA 1.1 framework against three case studies related to South Africa at the district, national and international levels. In-depth interviews with 26 experts were conducted between August and November 2019, transcribed, coded and analysed in NVivo, using iterative categorization. The data were analysed against both the EVITA framework and the MINDSPACE framework for behavioural insights. RESULTS: In our case study comparison, we found that (1) research translation to the policy agenda occurs in a complex, fluid system which includes multiple "research clouds", "policy spheres" and other networks; (2) mental health research policy agenda-setting is based on key individuals and intermediaries and their interrelationships; and (3) key challenges and strategies for successful research to policy agenda impact are known, but are frequently not strategically implemented, such as including all stakeholders to overcome the policy implementation gap. Our data also suggest that behavioural science methods can be strategically applied to support knowledge translation to policy agenda-setting. CONCLUSION: We found that the EVITA framework is useful for understanding and improving mental health research policy interrelationships to support evidence uptake to the policy agenda, and that behavioural science methods are effective support mechanisms. The revised EVITA 2.0 framework therefore includes behavioural insights, for improved mental health policy agenda-setting in LMICs. More research is needed to understand whether EVITA can be applied to other LMICs and to high-income contexts.

From COVID-19 research to vaccine application: why might it take 17 months not 17 years and what are the wider lessons?

It is often said that it takes 17 years to move medical research from bench to bedside. In a coronavirus disease (COVID-19) world, such time-lags feel intolerable. In these extraordinary circumstances could years be made into months? If so, could those lessons be used to accelerate medical research when the crisis eases?To measure time-lags in health and biomedical research as well as to identify ways of reducing them, we developed and published (in 2015) a matrix consisting of overlapping tracks (or stages/phases) in the translation from discovery research to developed products, policies and practice. The matrix aids analysis by highlighting the time and actions required to develop research (and its translation) both (1) along each track and (2) from one track to another, e.g. from the discovery track to the research-in-humans track. We noted four main approaches to reducing time-lags, namely increasing resources, working in parallel, starting or working at risk, and improving processes.Examining these approaches alongside the matrix helps interpret the enormous global effort to develop a vaccine for the 2019 novel coronavirus SARS-CoV-2, the causative agent of COVID-19. Rapid progress in the discovery/basic and human research tracks is being made through a combination of large-scale funding, work being conducted in parallel (between different teams globally and through working in overlapping tracks), working at greater (but proportionate) risk to safety than usual, and adopting various new processes. The overlapping work of some of the teams involves continuing animal research whilst entering vaccine candidates into Phase I trials alongside planning their Phase II trials. The additional funding available helps to reduce some of the usual financial risks in moving so quickly. Going forward through the increasingly large human trials for safety, dosage and efficacy, it will be vital to overlap work in parallel in the often challenging public policy and clinical tracks. Thus, regulatory and reimbursement bodies are beginning and preparing rapid action to pull vaccines proving to be safe and effective through to extraordinarily rapid application to the general population. Monitoring the development of a COVID-19 vaccine using the matrix (modified as necessary) could help identify which of the approaches speeding development and deployment could be usefully applied more widely in the future.

Recurrent early stage endometrial cancer: Patterns of recurrence and results of salvage therapy.

OBJECTIVE: To analyze our institutional experience and oncologic outcomes for salvage treatment for the recurrence of early-stage endometrial cancer patients. METHODS: We included women of all ages diagnosed with FIGO stage I-II, any grade endometrial cancer from 2000 to 2016 at our institutions who were treated with at least a hysterectomy. Recurrences in the pelvis and/or vagina were considered locoregional recurrences (LRR). Overall survival (OS) was assessed using Kaplan-Meier survival analysis. Univariate (UV) and multivariate (MV) Cox proportional hazards modeling was also used. RESULTS: A total of 2691 women were analyzed. The majority had endometrioid histology (91%), stage IA disease (61%), and were grade 1 (57%). With a median follow-up of 6.1 years, the overall rate of recurrence was 7.2%, and the rate of LRR was 3.7%. Women with vaginal-only recurrences had a longer median OS after recurrence (14.0 years) compared to both pelvic (1.2 years) and distant (1.0 year) failures. For women with vaginal-only recurrences, salvage radiotherapy (RT) was the only factor associated with improved OS on MVA (HR 0.1, p = .04). For women with pelvic recurrences, salvage surgery (HR 0.3, p = .01), salvage RT (HR 0.3, p < .01), and salvage chemotherapy (HR 0.4, p = .03) were associated with improved OS. CONCLUSIONS: Failure rates for women with early-stage endometrial cancer are low. Women with vaginal-only recurrences have improved OS compared to pelvic or distant recurrences. Salvage RT appears to be an important factor for treatment of women with vaginal-only recurrences. Aggressive multimodality treatment may be beneficial for women with pelvic recurrences.

A systematic review of frameworks for the interrelationships of mental health evidence and policy in low- and middle-income countries.

BACKGROUND: The interrelationships between research evidence and policy-making are complex. Different theoretical frameworks exist to explain general evidence-policy interactions. One largely unexplored element of these interrelationships is how evidence interrelates with, and influences, policy/political agenda-setting. This review aims to identify the elements and processes of theories, frameworks and models on interrelationships of research evidence and health policy-making, with a focus on actionability and agenda-setting in the context of mental health in low- and middle-income countries (LMICs). METHODS: A systematic review of theories was conducted based on the BeHeMOTh search method, using a tested and refined search strategy. Nine electronic databases and other relevant sources were searched for peer-reviewed and grey literature. Two reviewers screened the abstracts, reviewed full-text articles, extracted data and performed quality assessments. Analysis was based on a thematic analysis. The included papers had to present an actionable theoretical framework/model on evidence and policy interrelationships, such as knowledge translation or evidence-based policy, specifically target the agenda-setting process, focus on mental health, be from LMICs and published in English. RESULTS: From 236 publications included in the full text analysis, no studies fully complied with our inclusion criteria. Widening the focus by leaving out 'agenda-setting', we included ten studies, four of which had unique conceptual frameworks focusing on mental health and LMICs but not agenda-setting. The four analysed frameworks confirmed research gaps from LMICs and mental health, and a lack of focus on agenda-setting. Frameworks and models from other health and policy areas provide interesting conceptual approaches and lessons with regards to agenda-setting. CONCLUSION: Our systematic review identified frameworks on evidence and policy interrelations that differ in their elements and processes. No framework fulfilled all inclusion criteria. Four actionable frameworks are applicable to mental health and LMICs, but none specifically target agenda-setting. We have identified agenda-setting as a research theory gap in the context of mental health knowledge translation in LMICs. Frameworks from other health/policy areas could offer lessons on agenda-setting and new approaches for creating policy impact for mental health and to tackle the translational gap in LMICs.

Estimating the returns to United Kingdom publicly funded musculoskeletal disease research in terms of net value of improved health outcomes.

BACKGROUND: Building on an approach applied to cardiovascular and cancer research, we estimated the economic returns from United Kingdom public- and charitable-funded musculoskeletal disease (MSD) research that arise from the net value of the improved health outcomes in the United Kingdom. METHODS: To calculate the economic returns from MSD-related research in the United Kingdom, we estimated (1) the public and charitable expenditure on MSD-related research in the United Kingdom between 1970 and 2013; (2) the net monetary benefit (NMB), derived from the health benefit in quality adjusted life years (QALYs) valued in monetary terms (using a base-case value of a QALY of £25,000) minus the cost of delivering that benefit, for a prioritised list of interventions from 1994 to 2013; (3) the proportion of NMB attributable to United Kingdom research; and (4) the elapsed time between research funding and health gain. The data collected from these four key elements were used to estimate the internal rate of return (IRR) from MSD-related research investments on health benefits. We analysed the uncertainties in the IRR estimate using a one-way sensitivity analysis. RESULTS: Expressed in 2013 prices, total expenditure on MSD-related research from 1970 to 2013 was £3.5 billion, and for the period used to estimate the rate of return, 1978-1997, was £1.4 billion. Over the period 1994-2013 the key interventions analysed produced 871,000 QALYs with a NMB of £16 billion, allowing for the net NHS costs resulting from them and valuing a QALY at £25,000. The proportion of benefit attributable to United Kingdom research was 30% and the elapsed time between funding and impact of MSD treatments was 16 years. Our best estimate of the IRR from MSD-related research was 7%, which is similar to the 9% for CVD and 10% for cancer research. CONCLUSIONS: Our estimate of the IRR from the net health gain to public and charitable funding of MSD-related research in the United Kingdom is substantial, and justifies the research investments made between 1978 and 1997. We also demonstrated the applicability of the approach previously used in assessing the returns from cardiovascular and cancer research. Inevitably, with a study of this kind, there are a number of important assumptions and caveats that we highlight, and these can inform future research.

ISRIA statement: ten-point guidelines for an effective process of research impact assessment.

As governments, funding agencies and research organisations worldwide seek to maximise both the financial and non-financial returns on investment in research, the way the research process is organised and funded is becoming increasingly under scrutiny. There are growing demands and aspirations to measure research impact (beyond academic publications), to understand how science works, and to optimise its societal and economic impact. In response, a multidisciplinary practice called research impact assessment is rapidly developing. Given that the practice is still in its formative stage, systematised recommendations or accepted standards for practitioners (such as funders and those responsible for managing research projects) across countries or disciplines to guide research impact assessment are not yet available.In this statement, we propose initial guidelines for a rigorous and effective process of research impact assessment applicable to all research disciplines and oriented towards practice. This statement systematises expert knowledge and practitioner experience from designing and delivering the International School on Research Impact Assessment (ISRIA). It brings together insights from over 450 experts and practitioners from 34 countries, who participated in the school during its 5-year run (from 2013 to 2017) and shares a set of core values from the school's learning programme. These insights are distilled into ten-point guidelines, which relate to (1) context, (2) purpose, (3) stakeholders' needs, (4) stakeholder engagement, (5) conceptual frameworks, (6) methods and data sources, (7) indicators and metrics, (8) ethics and conflicts of interest, (9) communication, and (10) community of practice.The guidelines can help practitioners improve and standardise the process of research impact assessment, but they are by no means exhaustive and require evaluation and continuous improvement. The prima facie effectiveness of the guidelines is based on the systematised expert and practitioner knowledge of the school's faculty and participants derived from their practical experience and research evidence. The current knowledge base has gaps in terms of the geographical and scientific discipline as well as stakeholder coverage and representation. The guidelines can be further strengthened through evaluation and continuous improvement by the global research impact assessment community.

Facial asymmetry in head and neck rhabdomyosarcoma survivors.

INTRODUCTION: Radiotherapy is essential for achieving and maintaining local control in head and neck rhabdomyosarcoma (HNRMS) patients. However, radiotherapy may cause outgrowth disturbances of facial bone and soft tissue, resulting in facial asymmetry. The aim of this study was to develop a method to visualize and measure facial asymmetry in HNRMS survivors using three-dimensional (3D) imaging techniques. METHODS: Facial deformity was evaluated in a multidisciplinary clinical assessment of 75 HNRMS survivors, treated with external beam radiotherapy (EBRT, n = 26) or Ablative surgery, MOulage brachytherapy, and REconstruction (AMORE, n = 49). Individual facial asymmetry was measured using 3D photogrammetry and expressed in a raw asymmetry index and a normalized sex-age-ethnicity-matched asymmetry signature weight. Facial asymmetry was also compared between British and Dutch controls and between survivors and their matched controls. RESULTS: Facial asymmetry was more pronounced with increasing age (P < 0.01) in British controls compared with Dutch controls (P = 0.04). Survivors developed more facial asymmetry than matched controls (P < 0.001). The clinical assessment of facial deformity correlated with the raw asymmetry index (r = 0.60, P < 0.001). DISCUSSION: 3D imaging can be used for objective measurement of facial asymmetry in HNRMS survivors. The raw asymmetry index correlated with a clinical assessment of facial deformity. Comparisons between treatment groups seemed inappropriate given the differences in facial asymmetry between British and Dutch controls. In future studies, pretreatment images could act as matched controls for posttreatment evaluation.

Quantifying the economic impact of government and charity funding of medical research on private research and development funding in the United Kingdom.

BACKGROUND: Government- and charity-funded medical research and private sector research and development (R&D) are widely held to be complements. The only attempts to measure this complementarity so far have used data from the United States of America and are inevitably increasingly out of date. This study estimates the magnitude of the effect of government and charity biomedical and health research expenditure in the United Kingdom (UK), separately and in total, on subsequent private pharmaceutical sector R&D expenditure in the UK. METHODS: The results for this study are obtained by fitting an econometric vector error correction model (VECM) to time series for biomedical and health R&D expenditure in the UK for ten disease areas (including 'other') for the government, charity and private sectors. The VECM model describes the relationship between public (i.e. government and charities combined) sector expenditure, private sector expenditure and global pharmaceutical sales as a combination of a long-term equilibrium and short-term movements. RESULTS: There is a statistically significant complementary relationship between public biomedical and health research expenditure and private pharmaceutical R&D expenditure. A 1% increase in public sector expenditure is associated in the best-fit model with a 0.81% increase in private sector expenditure. Sensitivity analysis produces a similar and statistically significant result with a slightly smaller positive elasticity of 0.68. Overall, every additional £1 of public research expenditure is associated with an additional £0.83-£1.07 of private sector R&D spend in the UK; 44% of that additional private sector expenditure occurs within 1 year, with the remainder accumulating over decades. This spillover effect implies a real annual rate of return (in terms of economic impact) to public biomedical and health research in the UK of 15-18%. When combined with previous estimates of the health gain that results from public medical research in cancer and cardiovascular disease, the total rate of return would be around 24-28%. CONCLUSION: Overall, this suggests that government and charity funded research in the UK crowds in additional private sector R&D in the UK. The implied historical returns from UK government and charity funded investment in medical research in the UK compare favourably with the rates of return achieved on investments in the rest of the UK economy and are greatly in excess of the 3.5% real annual rate of return required by the UK government to public investments generally.

A global call for action to include gender in research impact assessment.

Global investment in biomedical research has grown significantly over the last decades, reaching approximately a quarter of a trillion US dollars in 2010. However, not all of this investment is distributed evenly by gender. It follows, arguably, that scarce research resources may not be optimally invested (by either not supporting the best science or by failing to investigate topics that benefit women and men equitably). Women across the world tend to be significantly underrepresented in research both as researchers and research participants, receive less research funding, and appear less frequently than men as authors on research publications. There is also some evidence that women are relatively disadvantaged as the beneficiaries of research, in terms of its health, societal and economic impacts. Historical gender biases may have created a path dependency that means that the research system and the impacts of research are biased towards male researchers and male beneficiaries, making it inherently difficult (though not impossible) to eliminate gender bias. In this commentary, we - a group of scholars and practitioners from Africa, America, Asia and Europe - argue that gender-sensitive research impact assessment could become a force for good in moving science policy and practice towards gender equity. Research impact assessment is the multidisciplinary field of scientific inquiry that examines the research process to maximise scientific, societal and economic returns on investment in research. It encompasses many theoretical and methodological approaches that can be used to investigate gender bias and recommend actions for change to maximise research impact. We offer a set of recommendations to research funders, research institutions and research evaluators who conduct impact assessment on how to include and strengthen analysis of gender equity in research impact assessment and issue a global call for action.

Evaluating the impact of a pediatric weight-based dosing procedure in outpatient pharmacy.

OBJECTIVES: To describe the percentage of pediatric outpatient pharmacy prescriptions with inappropriate prescribing identified by a pharmacist that resulted in a change to the prescription. Secondary objectives include describing types of inappropriate prescribing errors, prevalence of Institute of Safe Medication Practices high-alert medications, patient demographics, prescriber origin, and prescription origin. METHODS: This retrospective outpatient prescription record review was approved by an institutional review board and performed at an outpatient pharmacy located in an academic teaching hospital. The study reviewed pediatric outpatient prescriptions for a 6-month period. Prescriptions with prescribing errors were identified from pediatric prescriptions sent to the problem queue and documented with appropriate pharmacist notes. RESULTS: This study demonstrated the impact of a dose checking procedure and pharmacist interventions on pediatric prescriptions. Initial results show that 3% of all pediatric prescriptions required a pharmacist intervention. Of these prescriptions, 50% resulted in a change to the original prescription. CONCLUSION: Weight-based dose checking in a pediatric outpatient pharmacy proactively prevents potential adverse events among the pediatric population. Despite this study's limitations, we believe that a pediatric dose checking procedure in community pharmacies will reduce adverse events. Further study is warranted in this field.

Apert Syndrome: Outcomes From the Australian Craniofacial Unit's Birth to Maturity Management Protocol.

The complex, progressive, multisystem nature of Apert syndrome presents many challenges to managing surgeons. Based on the pioneering work of Paul Tessier, the senior author developed a multidisciplinary birth to maturity management protocol for Apert syndrome. Between 1975 and 2014 the Australian Craniofacial Unit has treated 174 Apert syndrome patients and 28 have completed full protocol management. This paper reviews the scientific contribution made to the management of Apert syndrome by the Australian Craniofacial Unit, the development and evolution of the protocol and presents comprehensive data on the surgical and nonsurgical craniofacial interventions, and outcomes for the 28 patients who have completed the programme; 26 had normal visual acuity, 22 had normal hearing, 20 achieved normal or nearly normal speech, 24 a functional class I occlusion, 18 completed mainstream schooling of whom at least 8 went on to tertiary education, at least 13 gained employment and 15 developed good social groups. These outcomes equal or exceed those presented by other authors and provide compelling evidence of the value of protocol management in clinical outcomes, in addition to their value in international collaboration, and scientific development of future therapeutic strategies for the management of Apert syndrome.

How to increase value and reduce waste when research priorities are set.

The increase in annual global investment in biomedical research--reaching US$240 billion in 2010--has resulted in important health dividends for patients and the public. However, much research does not lead to worthwhile achievements, partly because some studies are done to improve understanding of basic mechanisms that might not have relevance for human health. Additionally, good research ideas often do not yield the anticipated results. As long as the way in which these ideas are prioritised for research is transparent and warranted, these disappointments should not be deemed wasteful; they are simply an inevitable feature of the way science works. However, some sources of waste cannot be justified. In this report, we discuss how avoidable waste can be considered when research priorities are set. We have four recommendations. First, ways to improve the yield from basic research should be investigated. Second, the transparency of processes by which funders prioritise important uncertainties should be increased, making clear how they take account of the needs of potential users of research. Third, investment in additional research should always be preceded by systematic assessment of existing evidence. Fourth, sources of information about research that is in progress should be strengthened and developed and used by researchers. Research funders have primary responsibility for reductions in waste resulting from decisions about what research to do.

A new resource for identifying and assessing the impacts of research.

The impact case studies submitted by UK Higher Education Institutions to the Research Excellence Framework (REF) in 2014 provide a rich resource of text describing impact beyond academia and across all disciplines. Using text mining techniques and qualitative assessment, the 6,679 non-redacted case studies submitted were analysed and the impact described was found to be multidisciplinary, multi-impactful, and multinational. By digging deeper into the data, the health gains from health research in terms of Quality Adjusted Life Years was also estimated. Similar analyses are possible using these case studies, but will require the data to be 're-purposed' from the original intention (i.e., for assessment purposes) for robust analysis.

Isolated port-site metastases after minimally invasive hysterectomy for endometrial cancer: outcomes of patients treated with radiotherapy.

OBJECTIVE: The management and prognosis of isolated port-site metastases after laparoscopic surgery for endometrial cancer is poorly understood and rarely described in the literature. We report a series of cases treated with radiotherapy to better characterize outcomes in these patients. METHODS: We retrospectively reviewed medical records of patients with endometrial cancer who developed isolated port-site metastases and were treated with radiation therapy at MD Anderson Cancer Center from 1996 to 2013. Seven patients met these criteria for whom treatment and outcome data were collected. RESULTS: The median interval from initial surgery to port-site recurrence was 15 months. Recurrent tumor size varied from 0.5 to 9 cm as measured on axial imaging. Six of the 7 patients underwent surgical resection of the recurrence. All received radiotherapy to a dose of 45 to 66 Gy. At a median follow-up of 2 years from the time of the port-site recurrence, the rate of disease-free survival at 1 and 2 years after the recurrence was 100% and 44%, respectively. The rate of local control and overall survival at 2 years was 100%. CONCLUSIONS: Isolated port-site metastases in the setting of endometrial cancer are associated with high rates of local control when treated with multimodality therapy including radiotherapy. Long-term disease-free outcomes in some patients suggest the potential for cure and justify aggressive local therapy. The optimal integration of surgery, chemotherapy, and radiation is unknown.

Targeted gold nanoparticles enhance sensitization of prostate tumors to megavoltage radiation therapy in vivo.

We report potent radiosensitization of prostate cancers in vitro and in vivo using goserelin-conjugated gold nanorods. Progressive receptor-mediated internalization of conjugated nanorods over time increases the radiation interaction cross-section of cells and contributes to the effects observed in vitro. The low concentrations of gold required, the long interval between injection of nanoparticles and radiation, and the use of megavoltage radiation to generate radiosensitization in vivo foretell the possibility of eventual clinical translation of this approach. FROM THE CLINICAL EDITOR: The ability of gold nanoparticles (AuNPs) to enhance the effect of physical radiation dose on tumor cells is known. This radiosensitization effect is thought to result from an increased number of photoelectric absorption events and the increased number of electrons present in gold. The authors here sought to further increase the amount and specificity of gold accumulation in prostatic cancer cells by conjugating gold nanorods to goserelin, a synthetic luteinizing hormone releasing hormone (LHRH) analogue that would bind to the LHRH receptor overexpressed in prostate cancers. It was shown that tumour cells were more sensitive to megavoltage radiation therapy. It is hoped that there would be eventual clinical translation of this approach.

How long does biomedical research take? Studying the time taken between biomedical and health research and its translation into products, policy, and practice.

BACKGROUND: The time taken, or 'time lags', between biomedical/health research and its translation into health improvements is receiving growing attention. Reducing time lags should increase rates of return to such research. However, ways to measure time lags are under-developed, with little attention on where time lags arise within overall timelines. The process marker model has been proposed as a better way forward than the current focus on an increasingly complex series of translation 'gaps'. Starting from that model, we aimed to develop better methods to measure and understand time lags and develop ways to identify policy options and produce recommendations for future studies. METHODS: Following reviews of the literature on time lags and of relevant policy documents, we developed a new approach to conduct case studies of time lags. We built on the process marker model, including developing a matrix with a series of overlapping tracks to allow us to present and measure elements within any overall time lag. We identified a reduced number of key markers or calibration points and tested our new approach in seven case studies of research leading to interventions in cardiovascular disease and mental health. Finally, we analysed the data to address our study's key aims. RESULTS: The literature review illustrated the lack of agreement on starting points for measuring time lags. We mapped points from policy documents onto our matrix and thus highlighted key areas of concern, for example around delays before new therapies become widely available. Our seven completed case studies demonstrate we have made considerable progress in developing methods to measure and understand time lags. The matrix of overlapping tracks of activity in the research and implementation processes facilitated analysis of time lags along each track, and at the cross-over points where the next track started. We identified some factors that speed up translation through the actions of companies, researchers, funders, policymakers, and regulators. Recommendations for further work are built on progress made, limitations identified and revised terminology. CONCLUSIONS: Our advances identify complexities, provide a firm basis for further methodological work along and between tracks, and begin to indicate potential ways of reducing lags.

Estimating the returns to UK publicly funded cancer-related research in terms of the net value of improved health outcomes.

BACKGROUND: Building on an approach developed to assess the economic returns to cardiovascular research, we estimated the economic returns from UK public and charitable funded cancer-related research that arise from the net value of the improved health outcomes. METHODS: To assess these economic returns from cancer-related research in the UK we estimated: 1) public and charitable expenditure on cancer-related research in the UK from 1970 to 2009; 2) net monetary benefit (NMB), that is, the health benefit measured in quality adjusted life years (QALYs) valued in monetary terms (using a base-case value of a QALY of GB£25,000) minus the cost of delivering that benefit, for a prioritised list of interventions from 1991 to 2010; 3) the proportion of NMB attributable to UK research; 4) the elapsed time between research funding and health gain; and 5) the internal rate of return (IRR) from cancer-related research investments on health benefits. We analysed the uncertainties in the IRR estimate using sensitivity analyses to illustrate the effect of some key parameters. RESULTS: In 2011/12 prices, total expenditure on cancer-related research from 1970 to 2009 was £15 billion. The NMB of the 5.9 million QALYs gained from the prioritised interventions from 1991 to 2010 was £124 billion. Calculation of the IRR incorporated an estimated elapsed time of 15 years. We related 17% of the annual NMB estimated to be attributable to UK research (for each of the 20 years 1991 to 2010) to 20 years of research investment 15 years earlier (that is, for 1976 to 1995). This produced a best-estimate IRR of 10%, compared with 9% previously estimated for cardiovascular disease research. The sensitivity analysis demonstrated the importance of smoking reduction as a major source of improved cancer-related health outcomes. CONCLUSIONS: We have demonstrated a substantive IRR from net health gain to public and charitable funding of cancer-related research in the UK, and further validated the approach that we originally used in assessing the returns from cardiovascular research. In doing so, we have highlighted a number of weaknesses and key assumptions that need strengthening in further investigations. Nevertheless, these cautious estimates demonstrate that the returns from past cancer research have been substantial, and justify the investments made during the period 1976 to 1995.

Does the level of mobility on ICU discharge impact post-ICU outcomes? A retrospective analysis.

PURPOSE: Mobilisation is a common intervention in Intensive Care (ICU). However, few studies have explored the relationship between mobility levels and outcomes. This study assessed the association of the level of mobility on ICU discharge with discharge destination from the hospital and hospital length of stay. MATERIALS AND METHODS: A retrospective analysis of data from 522 patients admitted to a single UK general ICU who were ventilated for ≥5 days was performed. The level of mobility was assessed using the Manchester Mobility Score (MMS). Multivariable regression analysed demographic and clinical variables for the independence of association with discharge destination and hospital length of stay. RESULTS: MMS ≥5 on ICU discharge was independently associated with discharge destination and hospital LOS (p < 0.001). Patients achieving MMS ≥5 on ICU discharge were more likely to be discharged home (OR 3.86 95% CI 2.1 to 6.9, p < 0.001), and had an 11.8 day shorter hospital LOS (95% CI -17.6 to -6.1, p < 0.001). CONCLUSIONS: The ability to step transfer to a chair (MMS ≥5) before ICU discharge was independently associated with discharge to usual residence and hospital LOS, irrespective of preadmission morbidity. Increasing the level of patient mobility at ICU discharge should be a key focus of rehabilitation interventions.

Mobilisation in the Intensive Care Unit (ICU) is common practice, however studies to date have not evaluated the impact on acute hospital outcomes.Achieving the ability to step to a chair on ICU discharge is an important rehabilitation milestone, and is associated with a shorter hospital length of stay and being discharged home.Rehabilitation interventions in the ICU should be targeted at progressing patients towards this milestone.

Exploring the application of machine learning to expert evaluation of research impact.

The objective of this study is to investigate the application of machine learning techniques to the large-scale human expert evaluation of the impact of academic research. Using publicly available impact case study data from the UK's Research Excellence Framework (2014), we trained five machine learning models on a range of qualitative and quantitative features, including institution, discipline, narrative style (explicit and implicit), and bibliometric and policy indicators. Our work makes two key contributions. Based on the accuracy metric in predicting high- and low-scoring impact case studies, it shows that machine learning models are able to process information to make decisions that resemble those of expert evaluators. It also provides insights into the characteristics of impact case studies that would be favoured if a machine learning approach was applied for their automated assessment. The results of the experiments showed strong influence of institutional context, selected metrics of narrative style, as well as the uptake of research by policy and academic audiences. Overall, the study demonstrates promise for a shift from descriptive to predictive analysis, but suggests caution around the use of machine learning for the assessment of impact case studies.