Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI.

Generative Artificial Intelligence is set to revolutionize healthcare delivery by transforming traditional patient care into a more personalized, efficient, and proactive process. Chatbots, serving as interactive conversational models, will probably drive this patient-centered transformation in healthcare. Through the provision of various services, including diagnosis, personalized lifestyle recommendations, dynamic scheduling of follow-ups, and mental health support, the objective is to substantially augment patient health outcomes, all the while mitigating the workload burden on healthcare providers. The life-critical nature of healthcare applications necessitates establishing a unified and comprehensive set of evaluation metrics for conversational models. Existing evaluation metrics proposed for various generic large language models (LLMs) demonstrate a lack of comprehension regarding medical and health concepts and their significance in promoting patients' well-being. Moreover, these metrics neglect pivotal user-centered aspects, including trust-building, ethics, personalization, empathy, user comprehension, and emotional support. The purpose of this paper is to explore state-of-the-art LLM-based evaluation metrics that are specifically applicable to the assessment of interactive conversational models in healthcare. Subsequently, we present a comprehensive set of evaluation metrics designed to thoroughly assess the performance of healthcare chatbots from an end-user perspective. These metrics encompass an evaluation of language processing abilities, impact on real-world clinical tasks, and effectiveness in user-interactive conversations. Finally, we engage in a discussion concerning the challenges associated with defining and implementing these metrics, with particular emphasis on confounding factors such as the target audience, evaluation methods, and prompt techniques involved in the evaluation process.

USE-Evaluator: Performance metrics for medical image segmentation models supervised by uncertain, small or empty reference annotations in neuroimaging.

Performance metrics for medical image segmentation models are used to measure the agreement between the reference annotation and the predicted segmentation. Usually, overlap metrics, such as the Dice, are used as a metric to evaluate the performance of these models in order for results to be comparable. However, there is a mismatch between the distributions of cases and the difficulty level of segmentation tasks in public data sets compared to clinical practice. Common metrics used to assess performance fail to capture the impact of this mismatch, particularly when dealing with datasets in clinical settings that involve challenging segmentation tasks, pathologies with low signal, and reference annotations that are uncertain, small, or empty. Limitations of common metrics may result in ineffective machine learning research in designing and optimizing models. To effectively evaluate the clinical value of such models, it is essential to consider factors such as the uncertainty associated with reference annotations, the ability to accurately measure performance regardless of the size of the reference annotation volume, and the classification of cases where reference annotations are empty. We study how uncertain, small, and empty reference annotations influence the value of metrics on a stroke in-house data set regardless of the model. We examine metrics behavior on the predictions of a standard deep learning framework in order to identify suitable metrics in such a setting. We compare our results to the BRATS 2019 and Spinal Cord public data sets. We show how uncertain, small, or empty reference annotations require a rethinking of the evaluation. The evaluation code was released to encourage further analysis of this topic https://github.com/SophieOstmeier/UncertainSmallEmpty.git.

Non-inferiority of deep learning ischemic stroke segmentation on non-contrast CT within 16-hours compared to expert neuroradiologists.

We determined if a convolutional neural network (CNN) deep learning model can accurately segment acute ischemic changes on non-contrast CT compared to neuroradiologists. Non-contrast CT (NCCT) examinations from 232 acute ischemic stroke patients who were enrolled in the DEFUSE 3 trial were included in this study. Three experienced neuroradiologists independently segmented hypodensity that reflected the ischemic core on each scan. The neuroradiologist with the most experience (expert A) served as the ground truth for deep learning model training. Two additional neuroradiologists' (experts B and C) segmentations were used for data testing. The 232 studies were randomly split into training and test sets. The training set was further randomly divided into 5 folds with training and validation sets. A 3-dimensional CNN architecture was trained and optimized to predict the segmentations of expert A from NCCT. The performance of the model was assessed using a set of volume, overlap, and distance metrics using non-inferiority thresholds of 20%, 3 ml, and 3 mm, respectively. The optimized model trained on expert A was compared to test experts B and C. We used a one-sided Wilcoxon signed-rank test to test for the non-inferiority of the model-expert compared to the inter-expert agreement. The final model performance for the ischemic core segmentation task reached a performance of 0.46 ± 0.09 Surface Dice at Tolerance 5mm and 0.47 ± 0.13 Dice when trained on expert A. Compared to the two test neuroradiologists the model-expert agreement was non-inferior to the inter-expert agreement, [Formula: see text]. The before, CNN accurately delineates the hypodense ischemic core on NCCT in acute ischemic stroke patients with an accuracy comparable to neuroradiologists.

Focal Visual-Text Attention for Memex Question Answering.

Recent insights on language and vision with neural networks have been successfully applied to simple single-image visual question answering. However, to tackle real-life question answering problems on multimedia collections such as personal photo albums, we have to look at whole collections with sequences of photos. This paper proposes a new multimodal MemexQA task: given a sequence of photos from a user, the goal is to automatically answer questions that help users recover their memory about an event captured in these photos. In addition to a text answer, a few grounding photos are also given to justify the answer. The grounding photos are necessary as they help users quickly verifying the answer. Towards solving the task, we 1) present the MemexQA dataset, the first publicly available multimodal question answering dataset consisting of real personal photo albums; 2) propose an end-to-end trainable network that makes use of a hierarchical process to dynamically determine what media and what time to focus on in the sequential data to answer the question. Experimental results on the MemexQA dataset demonstrate that our model outperforms strong baselines and yields the most relevant grounding photos on this challenging task.

Theoretical analysis and design of hydro-hammer with a jet actuator: An engineering application to improve the penetration rate of directional well drilling in hard rock formations.

Rapid horizontal directional well drilling in hard or fractured formations requires efficient drilling technology. The penetration rate of conventional hard rock drilling technology in horizontal directional well excavations is relatively low, resulting in multiple overgrinding of drill cuttings in bottom boreholes. Conventional drilling techniques with reamer or diamond drill bit face difficulties due to the long construction periods, low penetration rates, and high engineering costs in the directional well drilling of hard rock. To improve the impact energy and penetration rate of directional well drilling in hard formations, a new drilling system with a percussive and rotary drilling technology has been proposed, and a hydro-hammer with a jet actuator has also been theoretically designed on the basis of the impulse hydro-turbine pressure model. In addition, the performance parameters of the hydro-hammer with a jet actuator have been numerically and experimentally analyzed, and the influence of impact stroke and pumped flow rate on the motion velocity and impact energy of the hydro-hammer has been obtained. Moreover, the designed hydro-hammer with a jet actuator has been applied to hard rock drilling in a trenchless drilling program. The motion velocity of the hydro-hammer ranges from 1.2 m/s to 3.19 m/s with diverse flow rates and impact strokes, and the motion frequency ranges from 10 Hz to 22 Hz. Moreover, the maximum impact energy of the hydro-hammer is 407 J, and the pumped flow rate is 2.3 m3/min. Thus, the average penetration rate of the optimized hydro-hammer improves by over 30% compared to conventional directional drilling in hard rock formations.

Two birds, one stone: dual targeting of the cancer cell surface and subcellular mitochondria by the galectin-3-binding peptide G3-C12.

Active tumor-targeting approaches using specific ligands have drawn considerable attention over the years. However, a single ligand often fails to simultaneously target the cancer cell surface and subcellular organelles, which limits the maximum therapeutic efficacy of delivered drugs. We describe a polymeric delivery system modified with the G3-C12 peptide for sequential dual targeting. In this study, galectin-3-targeted G3-C12 peptide was conjugated onto the N-(2-hydroxypropyl) methacrylamide (HPMA) copolymer for the delivery of D(KLAKLAK)2 (KLA) peptide. G3-C12-HPMA-KLA exhibited increased receptor-mediated internalization into galectin-3-overexpressing PC-3 cells. Furthermore, G3-C12 peptide also directed HPMA-KLA conjugates to mitochondria. This occurred because the apoptosis signal triggered the accumulation of galectin-3 in mitochondria, and the G3-C12 peptide that specifically bound to galectin-3 was trafficked along with its receptor intracellularly. As a result, G3-C12-HPMA-KLA disrupted the mitochondrial membrane, increased the generation of reactive oxygen species (ROS) and induced cytochrome c release, which ultimately resulted in enhanced cytotoxicity. An in vivo study revealed that the G3-C12 peptide significantly enhanced the tumor accumulation of the KLA conjugate. In addition, G3-C12-HPMA-KLA exhibited the best therapeutic efficacy and greatly improved the animal survival rate. Our work demonstrates that G3-C12 is a promising ligand with dual-targeting functionality.

Infection with equine infectious anemia virus vaccine strain EIAVDLV121 causes no visible histopathological lesions in target organs in association with restricted viral replication and unique cytokine response.

The live equine infectious anemia virus (EIAV) vaccine strain EIAVDLV121 was developed by in vitro attenuation of a virulent strain, EIAVLN40, in the 1970s, and it has been demonstrated to induce protective immunity under laboratory and natural EIAV infection conditions. The detailed biological features of this attenuated virus remain to be further investigated. Experimental inoculation with EIAVDLV121 did not result in clinical symptoms even with immunosuppressive treatment in our previous studies. Here, we further investigated whether the replication of the vaccine strain EIAVDLV121 in experimentally infected horses causes histopathological lesions to develop in the targeted organs. Both the lungs and the spleen have been demonstrated to support EIAV replication. By evaluating the gross macroscopic and histological changes, we found that EIAVDLV121 did not cause detectable histopathological lesions and that it replicated several hundred times more slowly than its parental virulent strain, EIAVLN40, in tissues. Immunochemical assays of these tissues indicated that the primary target cells of EIAVDLV121 were monocytes/macrophages, but that EIAVLN40 also infected alveolar epithelial cells and vascular endothelial cells. In addition, both of these viral strains promoted the up- and down-regulation of the expression of various cytokines and chemokines, implicating the potential involvement of these cellular factors in the pathological outcomes of EIAV infection and host immune responses. Taken together, these results demonstrate that the EIAV vaccine strain does not cause obvious histopathological lesions or clinical symptoms and that it induces a unique cytokine response profile. These features are considered essential for EIAVDLV121 to function as an effective live vaccine.

[The exploration and practice of Genetic Engineering teaching].

Genetic Engineering is an important specialized basic course for the students majoring in life sciences. The quality of teaching is directly related to the students' professional quality and innovation ability. In order to improve the teaching quatity and train advanced biotechnical students, we made some reforms to the contents and teaching methods of Genetic Engineering according to the experience accumulated in recent years.

[Application of atomic force microscopy in chromosome research].

Atomic force microscopy (AFM) is an effective apparatus for examination of the surface structure of specimens with a higher resolution. It can be used for real-time observation in vacuum, air, and liquid conditions. This review presented the basic principle of AFM and its significant advantages in biological sample research relative to other types of microscopes and summarized its application in chromosome research.

[Comparative analysis of rDNA distribution in metaphase chromosomes of Cucurbitaceae species].

Fluorescence in situ hybridization (FISH) and double FISH experiments were carried out to ascertain the chromosomal distribution patterns of the 45S and 5S ribosomal DNAs in the three species of Cucurbitaceae. Five pairs of 45S rDNA loci and two pairs of 5S rDNA signals were detected on chromosomes of Cucurbita moschata Duch. Luffa cylindrical Roem. contained five pairs of 45S rDNA loci and one pair of 5S rDNA loci. In Benincasa hispida Cogn., two pairs of 45S rDNA sites and one pair of 5S rDNA site were detected. In this species, 5S rDNA and one pair of the 45S loci were collocated closely in chromosome 7S. 45S rDNA chromosomal distribution patterns were highly conserved among the three species, althoufh their number varied markedly. The 5S rDNA sites on chromosomes among the three species were highly polymorphic. We further discussed differentially evolutionary processes of 45S and 5S rDNA in plant genomes.

[Cytogenetic maps and their applications in plants].

Integrated cytogenetic maps encompass the information from both genetic maps and cytological maps. It is possible for cytogenetic maps to simultaneously report the cytological and genetic position of a maker. To constructure cytogenetic maps it is necessary to relate the markers mapped across linkage groups to cytological position on chromosomes. Cytogenetic maps have been constructured primarily in two ways. The first general strategy is to utilize the chromosome breakpoints to determine the location of genetically mapped markers on the chromosomes. A second way is by the direct hybridization of genetically mapped sequences onto chromosomes by FISH. In addition, a novel approach is to use RN-cM maps to predict the physical position of genetic markers on the chromosomes. Cytogenetic maps suggest that both the density of genes and the frequency of recombination increase towards the distal regions of chromosome arms, and they play significant roles in revealing gene colinearity between two species, exploring the evolution relationship between both of them and in map-based gene isolation.

[Flow cytometric analysis and sorting of plant chromosomes.].

Flow cytometry refers to the technique where the measurement of the physical and chemical characteristics of particles including chromosomes, nuclei and cells is made as the particles pass in a fluid stream through a measuring point surrounded by an array of detectors. FCM (flow cytometry) analysis has played an important role in human genome projects. This technique is now increasingly used to establish flow karyotypes, sort chromosomes, map genes and construct DNA libraries in plants. This paper reviewed progress in flow cytometric analysis of plant genomes.

Expressions of human p53 and c-myc gene homologues during caryopsis development in maize.

Tumor suppressor gene p53 and proto-oncogene c-myc have been proved to be highly conserved and participate in many PCD processes in animals. In maize, proteins and RNAs related to p53 and c-myc have already been reported and the sequences homologous to these two genes have also been localized onto maize chromosomes by FISH. In this study, using immunohistochemistry we investigated the expression patterns of maize genes homologous to human p53 and c-myc during caryopsis development stages in maize. In a giving stage after pollination, p53 homologue showed high levels in the antipodal cells, integument, immature endosperm, ovary wall, tracheary elements, and aleurone layer, while c-myc homologue showed low levels in these tissues, only before pollination showed high expression in polar nucleus. The results of TUNEL assay demonstrated that TUNEL positive signals were detected where p53 homologue showed high expression levels. In animal cells, p53 shows reverse function with that of c-myc, so does it in maize basically. These results demonstrated that p53 and c-myc homologues might play some important roles in PCD during caryopsis development in maize. There may be conserved mechanisms for PCD in animals and in high plants.