Your browser doesn't support javascript.
loading
High accurate and explainable multi-pill detection framework with graph neural network-assisted multimodal data fusion.
Nguyen, Anh Duy; Pham, Huy Hieu; Trung, Huynh Thanh; Nguyen, Quoc Viet Hung; Truong, Thao Nguyen; Nguyen, Phi Le.
Afiliação
  • Nguyen AD; School of Information and Communication Technology, Hanoi University of Science and Technology, Hanoi, Vietnam.
  • Pham HH; VinUni-Illinois Smart Health Center, VinUniversity, Hanoi, Vietnam.
  • Trung HT; College of Engineering & Computer Science, VinUniversity, Hanoi, Vietnam.
  • Nguyen QVH; VinUni-Illinois Smart Health Center, VinUniversity, Hanoi, Vietnam.
  • Truong TN; École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland.
  • Nguyen PL; Griffith University, Southport, Queensland, Australia.
PLoS One ; 18(9): e0291865, 2023.
Article em En | MEDLINE | ID: mdl-37768910
Due to the significant resemblance in visual appearance, pill misuse is prevalent and has become a critical issue, responsible for one-third of all deaths worldwide. Pill identification, thus, is a crucial concern that needs to be investigated thoroughly. Recently, several attempts have been made to exploit deep learning to tackle the pill identification problem. However, most published works consider only single-pill identification and fail to distinguish hard samples with identical appearances. Also, most existing pill image datasets only feature single pill images captured in carefully controlled environments under ideal lighting conditions and clean backgrounds. In this work, we are the first to tackle the multi-pill detection problem in real-world settings, aiming at localizing and identifying pills captured by users during pill intake. Moreover, we also introduce a multi-pill image dataset taken in unconstrained conditions. To handle hard samples, we propose a novel method for constructing heterogeneous a priori graphs incorporating three forms of inter-pill relationships, including co-occurrence likelihood, relative size, and visual semantic correlation. We then offer a framework for integrating a priori with pills' visual features to enhance detection accuracy. Our experimental results have proved the robustness, reliability, and explainability of the proposed framework. Experimentally, it outperforms all detection benchmarks in terms of all evaluation metrics. Specifically, our proposed framework improves COCO mAP metrics by 9.4% over Faster R-CNN and 12.0% compared to vanilla YOLOv5. Our study opens up new opportunities for protecting patients from medication errors using an AI-based pill identification solution.
Assuntos

Texto completo: 1 Base de dados: MEDLINE Assunto principal: Benchmarking / Ambiente Controlado Tipo de estudo: Diagnostic_studies / Prognostic_studies Limite: Humans Idioma: En Ano de publicação: 2023 Tipo de documento: Article

Texto completo: 1 Base de dados: MEDLINE Assunto principal: Benchmarking / Ambiente Controlado Tipo de estudo: Diagnostic_studies / Prognostic_studies Limite: Humans Idioma: En Ano de publicação: 2023 Tipo de documento: Article