Implementing monitoring triggers and matching of triggered and control sites in the TEMPER study: a description and evaluation of a triggered monitoring management system.

BACKGROUND: Triggered monitoring in clinical trials is a risk-based monitoring approach where triggers (centrally monitored, predefined key risk and performance indicators) drive the extent, timing, and frequency of monitoring visits. The TEMPER study used a prospective, matched-pair design to evaluate the use of a triggered monitoring strategy, comparing findings from triggered monitoring visits with those from matched control sites. To facilitate this study, we developed a bespoke risk-based monitoring system: the TEMPER Management System. METHODS: The TEMPER Management System comprises a web application (the front end), an SQL server database (the back end) to store the data generated for TEMPER, and a reporting function to aid users in study processes such as the selection of triggered sites. Triggers based on current practice were specified for three clinical trials and were implemented in the system. Trigger data were generated in the system using data extracted from the trial databases to inform the selection of triggered sites to visit. Matching of the chosen triggered sites with untriggered control sites was also performed in the system, while data entry screens facilitated the collection and management of the data from findings gathered at monitoring visits. RESULTS: There were 38 triggers specified for the participating trials. Using these, 42 triggered sites were chosen and matched with control sites. Monitoring visits were carried out to all sites, and visit findings were entered into the TEMPER Management System. Finally, data extracted from the system were used for analysis. CONCLUSIONS: The TEMPER Management System made possible the completion of the TEMPER study. It implemented an approach of standardising the automation of current-practice triggers, and the generation of trigger data to inform the selection of triggered sites to visit. It also implemented a matching algorithm informing the selection of matched control sites. We hope that by publishing this paper it encourages other trialists to share their approaches to, and experiences of, triggered monitoring and other risk-based monitoring systems.

Triggered or routine site monitoring visits for randomised controlled trials: results of TEMPER, a prospective, matched-pair study.

BACKGROUND/AIMS: In multi-site clinical trials, where trial data and conduct are scrutinised centrally with pre-specified triggers for visits to sites, targeted monitoring may be an efficient way to prioritise on-site monitoring. This approach is widely used in academic trials, but has never been formally evaluated. METHODS: TEMPER assessed the ability of targeted monitoring, as used in three ongoing phase III randomised multi-site oncology trials, to distinguish sites at which higher and lower rates of protocol and/or Good Clinical Practice violations would be found during site visits. Using a prospective, matched-pair design, sites that had been prioritised for visits after having activated 'triggers' were matched with a control ('untriggered') site, which would not usually have been visited at that time. The paired sites were visited within 4 weeks of each other, and visit findings are recorded and categorised according to the seriousness of the deviation. The primary outcome measure was the proportion of sites with ≥1 'Major' or 'Critical' finding not previously identified centrally. The study was powered to detect an absolute difference of ≥30% between triggered and untriggered visits. A sensitivity analysis, recommended by the study's blinded endpoint review committee, excluded findings related to re-consent. Additional analyses assessed the prognostic value of individual triggers and data from pre-visit questionnaires completed by site and trials unit staff. RESULTS: In total, 42 matched pairs of visits took place between 2013 and 2016. In the primary analysis, 88.1% of triggered visits had ≥1 new Major/Critical finding, compared to 81.0% of untriggered visits, an absolute difference of 7.1% (95% confidence interval -8.3%, +22.5%; p = 0.365). When re-consent findings were excluded, these figures reduced to 85.7% versus 59.5%, (difference = 26.2%, 95% confidence interval 8.0%, 44.4%; p = 0.007). Individual triggers had modest prognostic value but knowledge of the trial-related activities carried out by site staff may be useful. CONCLUSION: Triggered monitoring approaches, as used in these trials, were not sufficiently discriminatory. The rate of Major and Critical findings was higher than anticipated, but the majority related to consent and re-consent with no indication of systemic problems that would impact trial-wide safety issues or integrity of the results in any of the three trials. Sensitivity analyses suggest triggered monitoring may be of potential use, but needs improvement and investigation of further central monitoring triggers is warranted. TEMPER highlights the need to question and evaluate methods in trial conduct, and should inform further developments in this area.

A systematic search for reports of site monitoring technique comparisons in clinical trials.

BACKGROUND: As part of a broader methodological programme of work around clinical trial monitoring, we wanted to evaluate the existing evidence for the effectiveness of different monitoring techniques. PURPOSE: To identify and evaluate prospective studies of the effectiveness of different monitoring strategies. METHODS: A systematic search of MEDLINE from 1950 onwards, using free-text terms to identify relevant published studies. We intended to extract data on details of comparative techniques, monitoring findings identified by different techniques, and recommendations or identification of areas in need of further research made by authors. RESULTS: A total of 1222 published abstracts were identified and reviewed. Of these, nine articles described methods for quality control (QC) of clinical trial activities, and one article was identified that compared the same monitoring technique at two timepoints. None included a direct comparison of different monitoring techniques and findings. LIMITATIONS: The search strategy was limited to MEDLINE. However, MEDLINE includes all the journals that tend to report trial methodological research. CONCLUSIONS: There is a lack of published empirical data that compare monitoring strategies prospectively. Assessment of the usefulness and cost-effectiveness of monitoring techniques in a variety of clinical trial settings and indications is needed.

The potential for central monitoring techniques to replace on-site monitoring: findings from an international multi-centre clinical trial.

BACKGROUND: Compliance with Good Clinical Practice (GCP) guidelines should ensure the safety of trial participants and the reliability of trial results. Over the last decade, increasing emphasis has been placed on the role of costly on-site monitoring and source data verification as processes to demonstrate that GCP is being followed, despite a lack of empirical evidence that these are effective. PURPOSE: To assess whether findings from on-site monitoring of a recent international multi-centre clinical trial could have been identified using central data review and other centralised monitoring techniques. METHODS: Findings documented in a sample of site monitoring reports, and Programme Management Board Executive (PMBe) reports, from the Microbicides Development Programme (MDP) 301 trial - a randomised placebo-controlled trial of a microbicide gel to prevent vaginally acquired HIV infection conducted in four countries in East and Southern Africa - were extracted and individually assessed to determine whether they could have been detected in the trial database or through other central means. RESULTS: Four site visit reports contained 268 monitoring findings from a review of 104 participant files covering 324 study visits. Of the 268 findings, 76 (28.4%) were also identified in the study database. Central checks, had these been in place (such as central receipt and review of back-translated documents, enrolment and testing logs, informed consent, and more complex database queries), could have identified a further 179 (66.8%); 13 (4.9%) other findings (all minor) could have been identified through a review of the participant folder at site. The four PMBe reports reviewed included six major and three critical findings from a review of over 1000 participant files: only two of these (both major) were assessed as unlikely to be identified using central monitoring techniques. LIMITATIONS: The study data used were not collected with this retrospective review in mind. It suggests that prospective work is needed to compare monitoring practices in real time. CONCLUSIONS: While there may be some categories of findings that it is not possible to identify centrally, the very large majority of findings reviewed in this analysis could be identified using central monitoring strategies. These data suggest that with better central and targeted on-site monitoring, it should be possible to identify and address most protocol and procedural compliance issues without performing intensive and costly routine on-site data monitoring.