Human-in-the-loop error detection in an object organization task with a social robot.

In human-robot collaboration, failures are bound to occur. A thorough understanding of potential errors is necessary so that robotic system designers can develop systems that remedy failure cases. In this work, we study failures that occur when participants interact with a working system and focus especially on errors in a robotic system's knowledge base of which the system is not aware. A human interaction partner can be part of the error detection process if they are given insight into the robot's knowledge and decision-making process. We investigate different communication modalities and the design of shared task representations in a joint human-robot object organization task. We conducted a user study (N = 31) in which the participants showed a Pepper robot how to organize objects, and the robot communicated the learned object configuration to the participants by means of speech, visualization, or a combination of speech and visualization. The multimodal, combined condition was preferred by 23 participants, followed by seven participants preferring the visualization. Based on the interviews, the errors that occurred, and the object configurations generated by the participants, we conclude that participants tend to test the system's limitations by making the task more complex, which provokes errors. This trial-and-error behavior has a productive purpose and demonstrates that failures occur that arise from the combination of robot capabilities, the user's understanding and actions, and interaction in the environment. Moreover, it demonstrates that failure can have a productive purpose in establishing better user mental models of the technology.

Application of Bayesian approaches in drug development: starting a virtuous cycle.

The pharmaceutical industry and its global regulators have routinely used frequentist statistical methods, such as null hypothesis significance testing and p values, for evaluation and approval of new treatments. The clinical drug development process, however, with its accumulation of data over time, can be well suited for the use of Bayesian statistical approaches that explicitly incorporate existing data into clinical trial design, analysis and decision-making. Such approaches, if used appropriately, have the potential to substantially reduce the time and cost of bringing innovative medicines to patients, as well as to reduce the exposure of patients in clinical trials to ineffective or unsafe treatment regimens. Nevertheless, despite advances in Bayesian methodology, the availability of the necessary computational power and growing amounts of relevant existing data that could be used, Bayesian methods remain underused in the clinical development and regulatory review of new therapies. Here, we highlight the value of Bayesian methods in drug development, discuss barriers to their application and recommend approaches to address them. Our aim is to engage stakeholders in the process of considering when the use of existing data is appropriate and how Bayesian methods can be implemented more routinely as an effective tool for doing so.

A Useful and Sustainable Role for N-of-1 Trials in the Healthcare Ecosystem.

Clinicians and patients often try a treatment for an initial period to inform longer-term therapeutic decisions. A more rigorous approach involves N-of-1 trials. In these single-patient crossover trials, typically conducted in patients with chronic conditions, individual patients are given candidate treatments in a double-blinded, random sequence of alternating periods to determine the most effective treatment for that patient. However, to date, these trials are rarely done outside of research settings and have not been integrated into general care where they could offer substantial benefit. Designating this classical, N-of-1 trial design as type 1, there also are new and evolving uses of N-of-1 trials that we designate as type 2. In these, rather than focusing on optimizing treatment for chronic diseases when multiple approved choices are available, as is typical of type 1, a type 2 N-of-1 trial tests treatments designed specifically for a patient with a rare disease, to facilitate personalized medicine. While the aims differ, both types face the challenge of collecting individual-patient evidence using standard, trusted, widely accepted methods. To fulfill their potential for producing both clinical and research benefits, and to be available for wide use, N-of-1 trials will have to fit into the current healthcare ecosystem. This will require generalizable and accepted processes, platforms, methods, and standards. This also will require sustainable value-based arrangements among key stakeholders. In this article, we review opportunities, stakeholders, issues, and possible approaches that could support general use of N-of-1 trials and deliver benefit to patients and the healthcare enterprise. To assess and expand the benefits of N-of-1 trials, we propose multistakeholder meetings, workshops, and the generation of methods, standards, and platforms that would support wider availability and the value of N-of-1 trials.

The "Coming of Age" of Real-World Evidence in Drug Development and Regulation.

Real world evidence (RWE) has the potential to inform drug discovery, development, and regulatory decision making. The emergence and availability of large population-based datasets around the globe have stoked enthusiasm in the research community. However, much work remains to refine research methodologies, evolve the science, and define the place of RWE as a complementary source of evidence to randomized controlled trials.

Demographic diversity of participants in Pfizer sponsored clinical trials in the United States.

The approval of new medicinal agents requires robust efficacy and safety clinical trial data demonstrated to be applicable to population subgroups. Limited data have previously been reported by drug sponsors on the topic of clinical trial diversity. In order to establish a baseline of diversity in our clinical trials that can be used by us and other sponsors, an analysis of clinical trial diversity was conducted covering race, ethnicity, sex, and age. This analysis includes Pfizer interventional clinical trials that initiated enrollment between 2011 through 2020. The data set comprises 213 trials with 103,103 US participants. The analysis demonstrated that overall trial participation of Black or African American individuals was at the US census level (14.3% vs 13.4%), participation of Hispanic or Latino individuals was below US census (15.9% vs 18.5%), and female participation was at US census (51.1% vs 50.8%). The analysis also examined the percentage of trials that achieved racial and ethnic distribution levels at or above census levels. Participant levels above census were achieved in 56.1% of Pfizer trials for Black or African American participants, 51.4% of trials for White participants, 16.0% of trials for Asian participants, 14.2% of trials for Native Hawaiian and Pacific Islander participants, 8.5% of trials for American Indian and Alaska Native participants, and 52.3% of trials for Hispanic or Latino participants. The results presented here provide a baseline upon which we can quantify the impact of our ongoing efforts to improve racial and ethnic diversity in clinical trials.

Use of Real-World Data to Emulate a Clinical Trial and Support Regulatory Decision Making: Assessing the Impact of Temporality, Comparator Choice, and Method of Adjustment.

External controls have been primarily used in the setting of single-arm trials of rare diseases; their use in common diseases has not been readily investigated, nor is there guidance on how to best select comparators. Thus, the objective of this study was to emulate a large cardiovascular outcome trial of type 2 diabetes to compare associations of effectiveness with different comparator groups to those reported in the trial. Using the Liraglutide Effect and Action in Diabetes: Evaluation of Cardiovascular Outcome Results (LEADER) trial, we investigated six comparator groups using three calendar time periods (Early: 1999-2003; Later: 2004-2008, and Contemporaneous: 2009-2013) and two comparators (sulfonylureas and other second-to-third-line antidiabetic drugs). Hazard ratios (HRs) of the three-point composite cardiovascular outcome were estimated using four variations of the propensity score (adjustment, stratification, fine stratification, and matching) and compared with the LEADER trial (HR, 0.87; 95% confidence interval, 0.78-0.97). When comparing users of liraglutide with users of sulfonylureas, the HRs ranged from 0.57 to 1.03, with estimates in the early period most closely reflecting the LEADER trial (HR, 0.57-0.88). In contrast, the HRs ranged from 0.73 to 0.97 when comparing liraglutide users with users of any second-to-third-line antidiabetic drugs, although the later period generated estimates closest to the LEADER trial (HR, 0.77-0.84). Different methods of adjustment led to generally consistent HRs, aside from the fine stratification in the early period. This study highlights the complex interplay between comparator, temporality, and method of adjustment when selecting comparators using real-word data. These design choices must be considered in the design of trial emulation studies.

Regulation and Innovation: Role of Regulatory Science in Facilitating Pharmaceutical Innovation.

Regulatory science is defined as science and research intended to inform decision making in a regulatory framework. This issue of Clinical Pharmacology & Therapeutics and the papers herein deal with the expansive topic of regulatory science and its role in fostering innovation and accelerating access to medicines. Regulatory health authorities, industry, and multiple stakeholders have a shared objective in advancing regulatory science to keep up with the ever-increasing pace of biomedical science.

Efficacy and Effectiveness Too Trials: Clinical Trial Designs to Generate Evidence on Efficacy and on Effectiveness in Wide Practice.

Efficacy trials, designed to gain regulatory marketing approval, evaluate drugs in optimally selected patients under advantageous conditions for relatively short time periods. Effectiveness trials, designed to evaluate use in usual practice, assess treatments among more typical patients in real-world conditions with longer follow-up periods. In "efficacy-to-effectiveness (E2E) trials," if the initial efficacy trial component is positive, the trial seamlessly transitions to an effectiveness trial component to efficiently yield both types of evidence. Yet more time could be saved by simultaneously addressing efficacy and effectiveness in an "efficacy and effectiveness too (EE2) trial." Additionally, hybrids of the E2E and EE2 approaches with differing degrees of overlap of the two components could allow flexibility for specific drug development needs. In planning EE2 trials, each stakeholder's current and future needs, incentives, and perspective must be considered. Although challenging, the ultimate benefits to stakeholders, the health system, and the public should justify this effort.

Increased US prescription trends associated with the CDC Bacillus anthracis antimicrobial postexposure prophylaxis campaign.

PURPOSE: We evaluated national outpatient antimicrobial prescription trends in relation to the first United States case of inhalational anthrax due to the intentional delivery of Bacillus anthracis (B. anthracis) spores. METHODS: We queried IMS HEALTH's National Prescription Audit Plus7 database for two 6-month periods (July-December) in 2001 and 2000 to describe outpatient prescription trends of antimicrobials recommended during the Centers for Disease Control and Prevention's (CDC) postexposure prophylaxis campaign. RESULTS: Overall, antimicrobial utilization for the referent 6-month time frame was greater in 2000 compared to 2001. In contrast, ciprofloxacin utilization was greater in 2001 during October, the month following the index case, increasing by more than 40% over utilization in October 2000. Similarly, doxycycline utilization increased by 30% during October/November. This corresponded to relative increases in US utilization for ciprofloxacin of approximately 160,000 prescriptions for the month of October and for doxycycline of approximately 96,000 prescriptions during October and 120,000 prescriptions for November. CONCLUSIONS: We conclude more widespread prescribing of ciprofloxacin and doxycycline occurred in response to the first US bioterrorist-associated anthrax attacks than was warranted based upon confirmed or suspected B. anthracis exposure alone.

Improving the quality of adverse drug reaction reporting by 4th-year medical students.

PURPOSE: Evaluate whether a 15-minute lecture intervention will improve adverse drug reaction reporting quality on standard MedWatch forms. METHODS: Seventy-eight 4th-year medical students were randomized to intervention 'Group-A' or non-intervention 'Group-B' on the first day of a required five-day clinical pharmacology rotation. Group-A participants attended a 15-minute lecture on completing a MedWatch form with quality information considered by the Food and Drug Administration as critical to adequate adverse drug reaction reporting. Group-B participants did not attend this lecture. Both groups then watched a standardized patient interview of a recognizable adverse drug reaction and completed MedWatch forms. Four Safety Evaluators from the Food and Drug Administration (FDA) rated student responses in a blinded fashion for the primary efficacy variable of Overall Impression and six informational domins using a standardized data quality analysis form that was developed within the Office of Postmarketing Drug Risk Assessment of the FDA. RESULTS: Seventy-eight MedWatch forms were evaluated (Group-A = 40, Group B = 38). Overall MedWatch information quality scores for the intervention group were significantly higher than the non-intervention group (p < 0.004). CONCLUSIONS: As little as a 15-minute intervention can significantly improve the quality of adverse drug reaction reporting by 4th-year medical students. Academic medical centers should consider incorporating adverse drug reaction reporting curriculum into the clinical training of medical students.