Collection and Analysis of Adherence Information for Software as a Medical Device Clinical Trials: Systematic Review.

BACKGROUND: The rapid growth of digital health apps has necessitated new regulatory approaches to ensure compliance with safety and effectiveness standards. Nonadherence and heterogeneous user engagement with digital health apps can lead to trial estimates that overestimate or underestimate an app's effectiveness. However, there are no current standards for how researchers should measure adherence or address the risk of bias imposed by nonadherence through efficacy analyses. OBJECTIVE: This systematic review aims to address 2 critical questions regarding clinical trials of software as a medical device (SaMD) apps: How well do researchers report adherence and engagement metrics for studies of effectiveness and efficacy? and What efficacy analyses do researchers use to account for nonadherence and how appropriate are their methods? METHODS: We searched the Food and Drug Administration's registration database for registrations of repeated-use, patient-facing SaMD therapeutics. For each such registration, we searched ClinicalTrials.gov, company websites, and MEDLINE for the corresponding clinical trial and study articles through March 2022. Adherence and engagement data were summarized for each of the 24 identified articles, corresponding to 10 SaMD therapeutics. Each article was analyzed with a framework developed using the Cochrane risk-of-bias questions to estimate the potential effects of imperfect adherence on SaMD effectiveness. This review, funded by the Richard King Mellon Foundation, is registered on the Open Science Framework. RESULTS: We found that although most articles (23/24, 96%) reported collecting information about SaMD therapeutic engagement, of the 20 articles for apps with prescribed use, only 9 (45%) reported adherence information across all aspects of prescribed use: 15 (75%) reported metrics for the initiation of therapeutic use, 16 (80%) reported metrics reporting adherence between the initiation and discontinuation of the therapeutic (implementation), and 4 (20%) reported the discontinuation of the therapeutic (persistence). The articles varied in the reported metrics. For trials that reported adherence or engagement, there were 4 definitions of initiation, 8 definitions of implementation, and 4 definitions of persistence. All articles studying a therapeutic with a prescribed use reported effectiveness estimates that might have been affected by nonadherence; only a few (2/20, 10%) used methods appropriate to evaluate efficacy. CONCLUSIONS: This review identifies 5 areas for improving future SaMD trials and studies: use consistent metrics for reporting adherence, use reliable adherence metrics, preregister analyses for observational studies, use less biased efficacy analysis methods, and fully report statistical methods and assumptions.

The value of adherence information during clinical pharmaceutical trials.

BACKGROUND/AIMS: The quality of the evidence used to evaluate a drug's safety and efficacy depends, in part, on how well participants adhere to the prescribed drug-taking regime. There are multiple approaches to measure adherence in clinical trials, varying in their cost and accuracy. We demonstrate a method for evaluating the cost-effectiveness of common adherence monitoring methods, considering the costs and data quality for drugs that differ in how forgiving they are of nonadherence. METHODS: We propose a simulation approach to estimate the value of evidence about adherence, considering both costs of collection and potential errors in interpreting clinical trial results. We demonstrate the approach with a simulated clinical trial of nitrendipine, a common calcium channel blocker. We consider two trial designs, one using pretrial adherence to "enrich" the trial sample and one without an enrichment strategy. We use scenarios combining high and low values of two key properties of a clinical trial: participant adherence and drug forgiveness. RESULTS: Under the conditions of these simulations, the most cost-effective adherence monitoring approach depends on both trial participant adherence and drug forgiveness. For example, the enrichment strategy is not cost-effective for the base scenario (high forgiveness and high adherence), but is for other scenarios. We also estimate the effects of evaluable patient analysis, a controversial procedure that excludes nonadherent participants from the analyses, after a trial is completed. CONCLUSIONS: Our proposed approach can guide drug regulators and developers in designing efficient clinical trials and assessing the impact of nonadherence on trial results. It can identify cost-effective adherence-monitoring methods, given available knowledge about the methods, drug, and patients' expected adherence.

A Procedure for Eliciting Women's Preferences for Breast Cancer Screening Frequency.

BACKGROUND: We evaluate the construct validity of a proposed procedure for eliciting lay preferences among health care policy options, suited for structured surveys. It is illustrated with breast cancer screening, a domain in which people may have heterogeneous preferences. METHODS: Our procedure applies behavioral decision research principles to eliciting preferences among policy options expressed in quantitative terms. Three-hundred women older than 18 y without a history of breast cancer were recruited through Amazon MTurk. Participants evaluated 4 screening options for each of 4 groups of women, with varying risk of breast cancer. Each option was characterized by estimates of 3 primary outcomes: breast cancer deaths, false alarms, and overdiagnosis resulting in unnecessary treatment of cancers that would not progress. These estimates were based on those currently being developed by the Breast Cancer Surveillance Consortium. For each risk group, participants stated how frequently they would wish to receive screening, if the predicted outcomes applied to them. RESULTS: A preregistered test found that preferences were robust enough to be unaffected by the order of introducing and displaying the outcomes. Other tests of construct validity also suggested that respondents generally understood the task and expressed consistent preferences. Those preferences were related to participants' age and mammography history but not to measures of their numeracy, subjective numeracy, or demographics. There was considerable heterogeneity in their preferences. CONCLUSIONS: Members of the public can be engaged more fully in informing future screening guidelines if they evaluate the screening options characterized by the expected health outcomes expressed in quantitative terms. We offer and evaluate such a procedure, in terms of its construct validity with a diverse sample of women. HIGHLIGHTS: A novel survey method for eliciting lay preferences for breast cancer screening is proposed and evaluated in terms of its construct validity.Participants were generally insensitive to irrelevant task features (e.g., order of presentation) and sensitive to relevant ones (e.g., quantitative estimates of breast cancer risk, harms from screening).The proposed method elicits lay preferences in terms that can inform future screening guidelines, potentially improving communication between the public and policy makers.

Physicians' cognitive approach to prognostication after cardiac arrest.

OBJECTIVE: Elucidate how physicians formulate a neurological prognosis after cardiac arrest and compare differences between experts and general providers. METHODS: We performed semi-structured interviews with experts in post-arrest care and general physicians. We created an initial model and interview guide based on professional society guidelines. Two authors independently coded interviews based on this initial model, then identified new topics not included in it. To describe individual physicians' cognitive approach to prognostication, we created a graphical representation. We summarized these individual "mental models" into a single overall model, as well as two models stratified by expertise. RESULTS: We performed 36 interviews (17 experts and 19 generalists), most of whom practice in Europe (23) or North America (12). Participants described their approach to prognosis formulation as complex and iterative, with sequential and repeated data acquisition, interpretation, and prognosis formulation. Eventually, this cycle results in a final prognosis and treatment recommendation. Commonly mentioned factors were diagnostic test performance, time from arrest, patient characteristics. Participants also discussed factors rarely discussed in prognostication research including physician and hospital characteristics. We found no substantial differences between experts and general physicians. CONCLUSION: Physicians' cognitive approach to neurologic prognostication is complex and influenced by many factors, including some rarely considered in current research. Understanding these processes better could inform interventions designed to aid physicians in prognostication.

The Detection of Helicobacter hepaticus Using Whispering-Gallery Mode Microcavity Optical Sensors.

Current bacterial detection techniques are relatively slow, require bulky instrumentation, and usually require some form of specialized training. The gold standard for bacterial detection is culture testing, which can take several days to receive a viable result. Therefore, simpler detection techniques that are both fast and sensitive could greatly improve bacterial detection and identification. Here, we present a new method for the detection of the bacteria Helicobacter hepaticus using whispering-gallery mode (WGM) optical microcavity-based sensors. Due to minimal reflection losses and low material adsorption, WGM-based sensors have ultra-high quality factors, resulting in high-sensitivity sensor devices. In this study, we have shown that bacteria can be non-specifically detected using WGM optical microcavity-based sensors. The minimum detection for the device was 1 × 10(4) cells/mL, and the minimum time of detection was found to be 750 s. Given that a cell density as low as 1 × 10(3) cells/mL for Helicobacter hepaticus can cause infection, the limit of detection shown here would be useful for most levels where Helicobacter hepaticus is biologically relevant. This study suggests a new approach for H. hepaticus detection using label-free optical sensors that is faster than, and potentially as sensitive as, standard techniques.