A Novel Evaluation Framework for Medical LLMs: Combining Fuzzy Logic and MCDM for Medical Relation and Clinical Concept Extraction.

Artificial intelligence (AI) has become a crucial element of modern technology, especially in the healthcare sector, which is apparent given the continuous development of large language models (LLMs), which are utilized in various domains, including medical beings. However, when it comes to using these LLMs for the medical domain, there's a need for an evaluation platform to determine their suitability and drive future development efforts. Towards that end, this study aims to address this concern by developing a comprehensive Multi-Criteria Decision Making (MCDM) approach that is specifically designed to evaluate medical LLMs. The success of AI, particularly LLMs, in the healthcare domain, depends on their efficacy, safety, and ethical compliance. Therefore, it is essential to have a robust evaluation framework for their integration into medical contexts. This study proposes using the Fuzzy-Weighted Zero-InConsistency (FWZIC) method extended to p, q-quasirung orthopair fuzzy set (p, q-QROFS) for weighing evaluation criteria. This extension enables the handling of uncertainties inherent in medical decision-making processes. The approach accommodates the imprecise and multifaceted nature of real-world medical data and criteria by incorporating fuzzy logic principles. The MultiAtributive Ideal-Real Comparative Analysis (MAIRCA) method is employed for the assessment of medical LLMs utilized in the case study of this research. The results of this research revealed that "Medical Relation Extraction" criteria with its sub-levels had more importance with (0.504) than "Clinical Concept Extraction" with (0.495). For the LLMs evaluated, out of 6 alternatives, ( A 4 ) "GatorTron S 10B" had the 1st rank as compared to ( A 1 ) "GatorTron 90B" had the 6th rank. The implications of this study extend beyond academic discourse, directly impacting healthcare practices and patient outcomes. The proposed framework can help healthcare professionals make more informed decisions regarding the adoption and utilization of LLMs in medical settings.

Systematic review of MCDM approach applied to the medical case studies of COVID-19: trends, bibliographic analysis, challenges, motivations, recommendations, and future directions.

When COVID-19 spread in China in December 2019, thousands of studies have focused on this pandemic. Each presents a unique perspective that reflects the pandemic's main scientific disciplines. For example, social scientists are concerned with reducing the psychological impact on the human mental state especially during lockdown periods. Computer scientists focus on establishing fast and accurate computerized tools to assist in diagnosing, preventing, and recovering from the disease. Medical scientists and doctors, or the frontliners, are the main heroes who received, treated, and worked with the millions of cases at the expense of their own health. Some of them have continued to work even at the expense of their lives. All these studies enforce the multidisciplinary work where scientists from different academic disciplines (social, environmental, technological, etc.) join forces to produce research for beneficial outcomes during the crisis. One of the many branches is computer science along with its various technologies, including artificial intelligence, Internet of Things, big data, decision support systems (DSS), and many more. Among the most notable DSS utilization is those related to multicriterion decision making (MCDM), which is applied in various applications and across many contexts, including business, social, technological and medical. Owing to its importance in developing proper decision regimens and prevention strategies with precise judgment, it is deemed a noteworthy topic of extensive exploration, especially in the context of COVID-19-related medical applications. The present study is a comprehensive review of COVID-19-related medical case studies with MCDM using a systematic review protocol. PRISMA methodology is utilized to obtain a final set of (n = 35) articles from four major scientific databases (ScienceDirect, IEEE Xplore, Scopus, and Web of Science). The final set of articles is categorized into taxonomy comprising five groups: (1) diagnosis (n = 6), (2) safety (n = 11), (3) hospital (n = 8), (4) treatment (n = 4), and (5) review (n = 3). A bibliographic analysis is also presented on the basis of annual scientific production, country scientific production, co-occurrence, and co-authorship. A comprehensive discussion is also presented to discuss the main challenges, motivations, and recommendations in using MCDM research in COVID-19-related medial case studies. Lastly, we identify critical research gaps with their corresponding solutions and detailed methodologies to serve as a guide for future directions. In conclusion, MCDM can be utilized in the medical field effectively to optimize the resources and make the best choices particularly during pandemics and natural disasters.

A Comprehensive Review on the Behaviour of Motorcyclists: Motivations, Issues, Challenges, Substantial Analysis and Recommendations.

With the continuous emergence of new technologies and the adaptation of smart systems in transportation, motorcyclist driving behaviour plays an important role in the transition towards intelligent transportation systems (ITS). Studying motorcyclist driving behaviour requires accurate models with accurate and complete datasets for better road safety and traffic management. As accuracy is needed in modelling, motorcyclist driving behaviour analyses can be performed using sensors that collect driving behaviour characteristics during real-time experiments. This review article systematically investigates the literature on motorcyclist driving behaviour to present many findings related to the issues, problems, challenges, and research gaps that have existed over the last 10 years (2011-2021). A number of digital databases (i.e., IEEE Xplore®, ScienceDirect, Scopus, and Web of Science) were searched and explored to collect reliable peer-reviewed articles. Out of the 2214 collected articles, only 174 articles formed the final set of articles used in the analysis of the motorcyclist research area. The filtration process consisted of two stages that were implemented on the collected articles. Inclusion criteria were the core of the first stage of the filtration process keeping articles only if they were a study or review written in English or were articles that mainly incorporated the driving style of motorcyclists. The second phase of the filtration process is based on more rules for article inclusion. The criteria of inclusion for the second phase of filtration examined the deployment of motorcyclist driver behaviour characterisation procedures using a real-time-based data acquisition system (DAS) or a questionnaire. The final number of articles was divided into three main groups: reviews (7/174), experimental studies (41/174), and social studies-based articles (126/174). This taxonomy of the literature was developed to group the literature into articles with similar types of experimental conditions. Recommendation topics are also presented to enable and enhance the pace of the development in this research area. Research gaps are presented by implementing a substantial analysis of the previously proposed methodologies. The analysis mainly identified the gaps in the development of data acquisition systems, model accuracy, and data types incorporated in the proposed models. Finally, research directions towards ITS are provided by exploring key topics necessary in the advancement of this research area.

Integration of fuzzy-weighted zero-inconsistency and fuzzy decision by opinion score methods under a q-rung orthopair environment: A distribution case study of COVID-19 vaccine doses.

Owing to the limitations of Pythagorean fuzzy and intuitionistic fuzzy sets, scientists have developed a distinct and successive fuzzy set called the q-rung orthopair fuzzy set (q-ROFS), which eliminates restrictions encountered by decision-makers in multicriteria decision making (MCDM) methods and facilitates the representation of complex uncertain information in real-world circumstances. Given its advantages and flexibility, this study has extended two considerable MCDM methods the fuzzy-weighted zero-inconsistency (FWZIC) method and fuzzy decision by opinion score method (FDOSM) under the fuzzy environment of q-ROFS. The extensions were called q-rung orthopair fuzzy-weighted zero-inconsistency (q-ROFWZIC) method and q-rung orthopair fuzzy decision by opinion score method (q-ROFDOSM). The methodology formulated had two phases. The first phase 'development' presented the sequential steps of each method thoroughly.The q-ROFWZIC method was formulated and used in determining the weights of evaluation criteria and then integrated into the q-ROFDOSM for the prioritisation of alternatives on the basis of the weighted criteria. In the second phase, a case study regarding the MCDM problem of coronavirus disease 2019 (COVID-19) vaccine distribution was performed. The purpose was to provide fair allocation of COVID-19 vaccine doses. A decision matrix based on an intersection of 'recipients list' and 'COVID-19 distribution criteria' was adopted. The proposed methods were evaluated according to systematic ranking assessment and sensitivity analysis, which revealed that the ranking was subject to a systematic ranking that is supported by high correlation results over different scenarios with variations in the weights of criteria.

Multi-perspectives systematic review on the applications of sentiment analysis for vaccine hesitancy.

A substantial impediment to widespread Coronavirus disease (COVID-19) vaccination is vaccine hesitancy. Many researchers across scientific disciplines have presented countless studies in favor of COVID-19 vaccination, but misinformation on social media could hinder vaccination efforts and increase vaccine hesitancy. Nevertheless, studying people's perceptions on social media to understand their sentiment presents a powerful medium for researchers to identify the causes of vaccine hesitancy and therefore develop appropriate public health messages and interventions. To the best of the authors' knowledge, previous studies have presented vaccine hesitancy in specific cases or within one scientific discipline (i.e., social, medical, and technological). No previous study has presented findings via sentiment analysis for multiple scientific disciplines as follows: (1) social, (2) medical, public health, and (3) technology sciences. Therefore, this research aimed to review and analyze articles related to different vaccine hesitancy cases in the last 11 years and understand the application of sentiment analysis on the most important literature findings. Articles were systematically searched in Web of Science, Scopus, PubMed, IEEEXplore, ScienceDirect, and Ovid from January 1, 2010, to July 2021. A total of 30 articles were selected on the basis of inclusion and exclusion criteria. These articles were formed into a taxonomy of literature, along with challenges, motivations, and recommendations for social, medical, and public health and technology sciences. Significant patterns were identified, and opportunities were promoted towards the understanding of this phenomenon.

Telehealth utilization during the Covid-19 pandemic: A systematic review.

During the coronavirus disease (COVID-19) pandemic, different technologies, including telehealth, are maximised to mitigate the risks and consequences of the disease. Telehealth has been widely utilised because of its usability and safety in providing healthcare services during the COVID-19 pandemic. However, a systematic literature review which provides extensive evidence on the impact of COVID-19 through telehealth and which covers multiple directions in a large-scale research remains lacking. This study aims to review telehealth literature comprehensively since the pandemic started. It also aims to map the research landscape into a coherent taxonomy and characterise this emerging field in terms of motivations, open challenges and recommendations. Articles related to telehealth during the COVID-19 pandemic were systematically searched in the WOS, IEEE, Science Direct, Springer and Scopus databases. The final set included (n = 86) articles discussing telehealth applications with respect to (i) control (n = 25), (ii) technology (n = 14) and (iii) medical procedure (n = 47). Since the beginning of the pandemic, telehealth has been presented in diverse cases. However, it still warrants further attention. Regardless of category, the articles focused on the challenges which hinder the maximisation of telehealth in such times and how to address them. With the rapid increase in the utilization of telehealth in different specialised hospitals and clinics, a potential framework which reflects the authors' implications of the future application and opportunities of telehealth has been established. This article improves our understanding and reveals the full potential of telehealth during these difficult times and beyond.