A digital teleneuropsychology platform for the diagnosis of mild cognitive impairment: from concept to certification as a medical device.

BACKGROUND: Innovative digital solutions are shaping a new concept of dementia care, opening additional venues for prevention, diagnosis, monitoring and treatment. Hereby, we report the development of a tablet-based teleneuropsychology platform (Tenèpsia®), from concept to certification as Medical Device (MD) Class IIA, as per new MD regulation 745/2017. METHODS: The platform was designed for the remote cognitive evaluation and created thanks to the effort of a collaborative working group including experts from three Italian scientific societies and Biogen Italia S.r.l. (hereafter "Biogen"), and developers from Xenia Reply and Inside AI. The development strategy was guided by converting traditional paper-and-pencil tests into digital versions while maintaining comparable neuropsychological features and optimizing patient accessibility and user experience. The experts focused on the choice and adaptation of traditional neuropsychology measures for a 45-min teleneuropsychology assessment. RESULTS: The developers created a web and a mobile interface, respectively, for the professional (neuropsychologist) and non-professional (patient and caregiver) use. Recording of voice, drawing and typing information was enabled. Instant dashboards provide a quick overview of the patient's condition. Simulation activities were performed to obtain MD certification, valid across Europe. CONCLUSION: Neuropsychology services will benefit from the implementation in clinics of harmonized digital tools with adequate scientific and technological standards. The use of digital cognitive testing for the diagnosis of mild cognitive impairment is expected to enhance patient and clinician outcomes through simplified, digital objective data collection, sparing of time and resources, with a positive impact on healthcare costs and access to treatments, reducing inequalities and delays in diagnosis and cure.

Decisions are not all equal-Introducing a utility metric based on case-wise raters' perceptions.

Background and Objective Evaluation of AI-based decision support systems (AI-DSS) is of critical importance in practical applications, nonetheless common evaluation metrics fail to properly consider relevant and contextual information. In this article we discuss a novel utility metric, the weighted Utility (wU), for the evaluation of AI-DSS, which is based on the raters' perceptions of their annotation hesitation and of the relevance of the training cases. Methods We discuss the relationship between the proposed metric and other previous proposals; and we describe the application of the proposed metric for both model evaluation and optimization, through three realistic case studies. Results We show that our metric generalizes the well-known Net Benefit, as well as other common error-based and utility-based metrics. Through the empirical studies, we show that our metric can provide a more flexible tool for the evaluation of AI models. We also show that, compared to other optimization metrics, model optimization based on the wU can provide significantly better performance (AUC 0.862 vs 0.895, p-value <0.05), especially on cases judged to be more complex by the human annotators (AUC 0.85 vs 0.92, p-value <0.05). Conclusions We make the point for having utility as a primary concern in the evaluation and optimization of machine learning models in critical domains, like the medical one; and for the importance of a human-centred approach to assess the potential impact of AI models on human decision making also on the basis of further information that can be collected during the ground-truthing process.

A Proposal For COVID-19 Applications Enabling Extensive Epidemiological Studies.

During the next phase of COVID-19 outbreak, mobile applications could be the most used and proposed technical solution for monitoring and tracking, by acquiring data from subgroups of the population. A possible problem could be data fragmentation, which could lead to three harmful effects: i) data could not cover the minimum percentage of the people for monitoring efficacy, ii) it could be heavily biased due to different data collection policies, and iii) the app could not monitor subjects moving across different zones or countries. A common approach could solve these problems, defining requirements for the selection of observed data and technical specifications for the complete interoperability between different solutions. This work aims to integrate the international framework of requirements in order to mitigate the known issues and to suggest a method for clinical data collection that ensures to researchers and public health institution significant and reliable data. First, we propose to identify which data is relevant for COVID-19 monitoring through literature and guidelines review. Then we analysed how the currently available guidelines for COVID-19 monitoring applications drafted by European Union and World Health Organization face the issues listed before. Eventually we proposed the first draft of integration of current guidelines.

A Novel Method for Validating Multi-Classifiers. A Case Study for ICF-Based Health Status Classification.

In this paper, we propose a novel method for the validation of a multi-classification model according to the intended use and aim of a device for health status classification and the clinical needs of the practitioners involved.

Comprehensive Review on Current and Future Regulatory Requirements on Wearable Sensors in Preclinical and Clinical Testing.

Medical devices are designed, tested, and placed on the market in a highly regulated environment. Wearable sensors are crucial components of various medical devices: design and validation of wearable sensors, if managed according to international standards, can foster innovation while respecting regulatory requirements. The purpose of this paper is to review the upcoming European Union (EU) Medical Device Regulations 2017/745 and 2017/746, the current and future International Electrotechnical Commission (IEC) and International Organization for Standardization (ISO) standards that set methods for design and validation of medical devices, with a focus on wearable sensors. Risk classification according to the regulation is described. The international standards IEC 62304, IEC 60601, ISO 14971, and ISO 13485 are reviewed to define regulatory restrictions during design, pre-clinical validation and clinical validation of devices that include wearable sensors as crucial components. This paper is not about any specific innovation but it is a toolbox for interpreting current and future regulatory restrictions; an integrated method for design planning, validation and clinical testing is proposed. Application of this method to design wearable sensors should be evaluated in the future in order to assess its potentially positive impact to fostering innovation and to ensure timely development.