Smartphone video imaging: A versatile, low-cost technology for food authentication.

This study presents a low-cost smartphone-based imaging technique called smartphone video imaging (SVI) to capture short videos of samples that are illuminated by a colour-changing screen. Assisted by artificial intelligence, the study develops new capabilities to make SVI a versatile imaging technique such as the hyperspectral imaging (HSI). SVI enables classification of samples with heterogeneous contents, spatial representation of analyte contents and reconstruction of hyperspectral images from videos. When integrated with a residual neural network, SVI outperforms traditional computer vision methods for ginseng classification. Moreover, the technique effectively maps the spatial distribution of saffron purity in powder mixtures with predictive performance that is comparable to that of HSI. In addition, SVI combined with the U-Net deep learning module can produce high-quality images that closely resemble the target images acquired by HSI. These results suggest that SVI can serve as a consumer-oriented solution for food authentication.

A dental intraoral image dataset of gingivitis for image captioning.

One of the most striking topics in Artificial Intelligence (AI) is Image captioning that aims to integrate computer vision and natural language processing to create descriptions for each image. In this paper, we propose a new dataset designed specifically for image captioning in gingivitis diagnosis using deep learning. It includes 1,096 high-resolution intraoral images of 12 anterior teeth and surrounding gingival tissue that were collected under controlled conditions with professional-grade photography equipment. Each image features detailed labels and descriptive captions. The labeling process involved three periodontists with over ten years of experience who assigned Modified Gingival Index (MGI) scores to each tooth in the images, achieving high inter-rater reliability through a rigorous calibration process. Captions were then created by the same periodontists, offering diverse descriptions of gingivitis severity and locations. The dataset is systematically organized into training, validation, and testing subsets for systematic accessibility. This dataset supports the development of advanced image captioning algorithms and is a valuable educational resource for integrating real-world data into dental research and curriculum.

Deep neural networks for automated damage classification in image-based visual data of reinforced concrete structures.

Significant strides in deep learning for image recognition have expanded the potential of visual data in assessing damage to reinforced concrete (RC) structures. Our study proposes an automated technique, merging convolutional neural networks (CNNs) and fully convolutional networks (FCNs), to detect, classify, and segment building damage. These deep networks extract RC damage-related features from high-resolution smartphone images (3264 × 2448 pixels), categorized into two groups: damage (exposed reinforcement and spalled concrete) and undamaged area. With a labeled dataset of 2000 images, fine-tuning of network architecture and hyperparameters ensures effective training and testing. Remarkably, we achieve 98.75 % accuracy in damage classification and 95.98 % in segmentation, without overfitting. Both CNNs and FCNs play crucial roles in extracting features, showcasing the adaptability of deep learning. Our promising results validate the potential of these techniques for inspectors, providing an effective means to assess the severity of identified damage in image-based evaluations.

IG-Net: An Instrument-guided real-time semantic segmentation framework for prostate dissection during surgery for low rectal cancer.

BACKGROUND AND OBJECTIVE: Accurate prostate dissection is crucial in transanal surgery for patients with low rectal cancer. Improper dissection can lead to adverse events such as urethral injury, severely affecting the patient's postoperative recovery. However, unclear boundaries, irregular shape of the prostate, and obstructive factors such as smoke present significant challenges for surgeons. METHODS: Our innovative contribution lies in the introduction of a novel video semantic segmentation framework, IG-Net, which incorporates prior surgical instrument features for real-time and precise prostate segmentation. Specifically, we designed an instrument-guided module that calculates the surgeon's region of attention based on instrument features, performs local segmentation, and integrates it with global segmentation to enhance performance. Additionally, we proposed a keyframe selection module that calculates the temporal correlations between consecutive frames based on instrument features. This module adaptively selects non-keyframe for feature fusion segmentation, reducing noise and optimizing speed. RESULTS: To evaluate the performance of IG-Net, we constructed the most extensive dataset known to date, comprising 106 video clips and 6153 images. The experimental results reveal that this method achieves favorable performance, with 72.70% IoU, 82.02% Dice, and 35 FPS. CONCLUSIONS: For the task of prostate segmentation based on surgical videos, our proposed IG-Net surpasses all previous methods across multiple metrics. IG-Net balances segmentation accuracy and speed, demonstrating strong robustness against adverse factors.

Machine learning-enabled computer vision for plant phenotyping: a primer on AI/ML and case study on stomatal patterning.

Artificial intelligence and machine learning (AI/ML) can be used to automatically analyze large image datasets. One valuable application of this approach is estimation of plant trait data contained within images. Here we review 39 papers that describe the development and/or application of such models for estimation of stomatal traits from epidermal micrographs. In doing so, we hope to provide plant biologists with a foundational understanding of AI/ML and summarize the current capabilities and limitations of published tools. While most models show human-level performance for stomatal density (SD) quantification at superhuman speed, they are often likely to be limited in how broadly they can be applied across phenotypic diversity associated with genetic, environmental or developmental variation. Other models can make predictions across greater phenotypic diversity and/or additional stomatal/epidermal traits, but require significantly greater time investment to generate ground-truth data. We discuss the challenges and opportunities presented by AI/ML-enabled computer vision analysis, and make recommendations for future work to advance accelerated stomatal phenotyping.

Hyperspectral imaging and artificial intelligence enhance remote phenotyping of grapevine rootstock influence on whole vine photosynthesis.

Rootstocks are gaining importance in viticulture as a strategy to combat abiotic challenges, as well as enhancing scion physiology. Photosynthetic parameters such as maximum rate of carboxylation of RuBP (Vcmax) and the maximum rate of electron transport driving RuBP regeneration (Jmax) have been identified as ideal targets for potential influence by rootstock and breeding. However, leaf specific direct measurement of these photosynthetic parameters is time consuming, limiting the information scope and the number of individuals that can be screened. This study aims to overcome these limitations by employing hyperspectral imaging combined with artificial intelligence (AI) to predict these key photosynthetic traits at the canopy level. Hyperspectral imaging captures detailed optical properties across a broad range of wavelengths (400 to 1000 nm), enabling use of all wavelengths in a comprehensive analysis of the entire vine's photosynthetic performance (Vcmax and Jmax). Artificial intelligence-based prediction models that blend the strength of deep learning and machine learning were developed using two growing seasons data measured post-solstice at 15 h, 14 h, 13 h and 12 h daylengths for Vitis hybrid 'Marquette' grafted to five commercial rootstocks and 'Marquette' grafted to 'Marquette'. Significant differences in photosynthetic efficiency (Vcmax and Jmax) were noted for both direct and indirect measurements for the six rootstocks, indicating that rootstock genotype and daylength have a significant influence on scion photosynthesis. Evaluation of multiple feature-extraction algorithms indicated the proposed Vitis base model incorporating a 1D-Convolutional neural Network (CNN) had the best prediction performance with a R2 of 0.60 for Vcmax and Jmax. Inclusion of weather and chlorophyll parameters slightly improved model performance for both photosynthetic parameters. Integrating AI with hyperspectral remote phenotyping provides potential for high-throughput whole vine assessment of photosynthetic performance and selection of rootstock genotypes that confer improved photosynthetic performance potential in the scion.

Computer vision and deep transfer learning for automatic gauge reading detection.

This manuscript proposes an automatic reading detection system for an analogue gauge using a combination of deep learning, machine learning, and image processing. The study suggests image-processing techniques in manual analogue gauge reading that include generating readings for the image to provide supervised data to address difficulties in unsupervised data in gauges and to achieve better accuracy using DenseNet 169 compared to other approaches. The model uses artificial intelligence to automate reading detection using deep transfer learning models like DenseNet 169, InceptionNet V3, and VGG19. The models were trained using 1011 labeled pictures, 9 classes, and readings from 0 to 8. The VGG19 model exhibits a high training precision of 97.00% but a comparatively lower testing precision of 75.00%, indicating the possibility of overfitting. On the other hand, InceptionNet V3 demonstrates consistent precision across both datasets, but DenseNet 169 surpasses other models in terms of precision and generalization capabilities.

Multi-category sorting of plastic waste using Swin Transformer: A vision-based approach.

Sorting out plastic waste (PW) from municipal solid waste (MSW) by material type is crucial for reutilization and pollution reduction. However, current automatic separation methods are costly and inefficient, necessitating an advanced sorting process to ensure high feedstock purity. This study introduces a Swin Transformer-based model for effectively detecting PW in real-world MSW streams, leveraging both morphological and material properties. And, a dataset comprising 3560 optical images and infrared spectra data was created to support this task. This vision-based system can localize and classify PW into five categories: polypropylene (PP), polyethylene (PE), polyethylene terephthalate (PET), polyvinyl chloride (PVC), and polystyrene (PS). Performance evaluations reveal an accuracy rate of 99.75% and a mean Average Precision (mAP50) exceeding 91%. Compared to popular convolutional neural network (CNN)-based models, this well-trained Swin Transformer-based model offers enhanced convenience and performance in five-category PW detection task, maintaining a mAP50 over 80% in the real-life deployment. The model's effectiveness is further supported by visualization of detection results on MSW streams and principal component analysis of classification scores. These results demonstrate the system's significant effectiveness in both lab-scale and real-life conditions, aligning with global regulations and strategies that promote innovative technologies for plastic recycling, thereby contributing to the development of a sustainable circular economy.

Neuroscientific insights about computer vision models: a concise review.

The development of biologically-inspired computational models has been the focus of study ever since the artificial neuron was introduced by McCulloch and Pitts in 1943. However, a scrutiny of literature reveals that most attempts to replicate the highly efficient and complex biological visual system have been futile or have met with limited success. The recent state-of the-art computer vision models, such as pre-trained deep neural networks and vision transformers, may not be biologically inspired per se. Nevertheless, certain aspects of biological vision are still found embedded, knowingly or unknowingly, in the architecture and functioning of these models. This paper explores several principles related to visual neuroscience and the biological visual pathway that resonate, in some manner, in the architectural design and functioning of contemporary computer vision models. The findings of this survey can provide useful insights for building futuristic bio-inspired computer vision models. The survey is conducted from a historical perspective, tracing the biological connections of computer vision models starting with the basic artificial neuron to modern technologies such as deep convolutional neural network (CNN) and spiking neural networks (SNN). One spotlight of the survey is a discussion on biologically plausible neural networks and bio-inspired unsupervised learning mechanisms adapted for computer vision tasks in recent times.

Fall risk prediction using temporal gait features and machine learning approaches.

Introduction: Falls have been acknowledged as a major public health issue around the world. Early detection of fall risk is pivotal for preventive measures. Traditional clinical assessments, although reliable, are resource-intensive and may not always be feasible. Methods: This study explores the efficacy of artificial intelligence (AI) in predicting fall risk, leveraging gait analysis through computer vision and machine learning techniques. Data was collected using the Timed Up and Go (TUG) test and JHFRAT assessment from MMU collaborators and augmented with a public dataset from Mendeley involving older adults. The study introduces a robust approach for extracting and analyzing gait features, such as stride time, step time, cadence, and stance time, to distinguish between fallers and non-fallers. Results: Two experimental setups were investigated: one considering separate gait features for each foot and another analyzing averaged features for both feet. Ultimately, the proposed solutions produce promising outcomes, greatly enhancing the model's ability to achieve high levels of accuracy. In particular, the LightGBM demonstrates a superior accuracy of 96% in the prediction task. Discussion: The findings demonstrate that simple machine learning models can successfully identify individuals at higher fall risk based on gait characteristics, with promising results that could potentially streamline fall risk assessment processes. However, several limitations were discovered throughout the experiment, including an insufficient dataset and data variation, limiting the model's generalizability. These issues are raised for future work consideration. Overall, this research contributes to the growing body of knowledge on fall risk prediction and underscores the potential of AI in enhancing public health strategies through the early identification of at-risk individuals.

WormCNN-Assisted Establishment and Analysis of Glycation Stress Models in C. elegans: Insights into Disease and Healthy Aging.

Glycation Stress (GS), induced by advanced glycation end-products (AGEs), significantly impacts aging processes. This study introduces a new model of GS of Caenorhabditis elegans by feeding them Escherichia coli OP50 cultured in a glucose-enriched medium, which better simulates human dietary glycation compared to previous single protein-glucose cross-linking methods. Utilizing WormCNN, a deep learning model, we assessed the health status and calculated the Healthy Aging Index (HAI) of worms with or without GS. Our results demonstrated accelerated aging in the GS group, evidenced by increased autofluorescence and altered gene expression of key aging regulators, daf-2 and daf-16. Additionally, we observed elevated pharyngeal pumping rates in AGEs-fed worms, suggesting an addictive response similar to human dietary patterns. This study highlights the profound effects of GS on worm aging and underscores the critical role of computer vision in accurately assessing health status and aiding in the establishment of disease models. The findings provide insights into glycation-induced aging and offer a comprehensive approach to studying the effects of dietary glycation on aging processes.

Three-Dimensional Posture Estimation of Vehicle Occupants Using Depth and Infrared Images.

Pose estimation is crucial for ensuring passenger safety and better user experiences in semi- and fully autonomous vehicles. Traditional methods relying on pose estimation from regular color images face significant challenges due to a lack of three-dimensional (3D) information and the sensitivity to occlusion and lighting conditions. Depth images, which are invariant to lighting issues and provide 3D information about the scene, offer a promising alternative. However, there is a lack of strong work in 3D pose estimation from such images due to the time-consuming process of annotating depth images with 3D postures. In this paper, we present a novel approach to 3D human posture estimation using depth and infrared (IR) images. Our method leverages a three-stage fine-tuning process involving simulation data, approximated data, and a limited set of manually annotated samples. This approach allows us to effectively train a model capable of accurate 3D pose estimation with a median error of under 10 cm across all joints, using fewer than 100 manually annotated samples. To the best of our knowledge, this is the first work focusing on vehicle occupant posture detection utilizing only depth and IR data. Our results demonstrate the feasibility and efficacy of this approach, paving the way for enhanced passenger safety in autonomous vehicle systems.

Leakage Identification of Underground Structures Using Classification Deep Neural Networks and Transfer Learning.

Water leakage defects often occur in underground structures, leading to accelerated structural aging and threatening structural safety. Leakage identification can detect early diseases of underground structures and provide important guidance for reinforcement and maintenance. Deep learning-based computer vision methods have been rapidly developed and widely used in many fields. However, establishing a deep learning model for underground structure leakage identification usually requires a lot of training data on leakage defects, which is very expensive. To overcome the data shortage, a deep neural network method for leakage identification is developed based on transfer learning in this paper. For comparison, four famous classification models, including VGG16, AlexNet, SqueezeNet, and ResNet18, are constructed. To train the classification models, a transfer learning strategy is developed, and a dataset of underground structure leakage is created. Finally, the classification performance on the leakage dataset of different deep learning models is comparatively studied under different sizes of training data. The results showed that the VGG16, AlexNet, and SqueezeNet models with transfer learning can overall provide higher and more stable classification performance on the leakage dataset than those without transfer learning. The ResNet18 model with transfer learning can overall provide a similar value of classification performance on the leakage dataset than that without transfer learning, but its classification performance is more stable than that without transfer learning. In addition, the SqueezeNet model obtains an overall higher and more stable performance than the comparative models on the leakage dataset for all classification metrics.

Smart Ship Draft Reading by Dual-Flow Deep Learning Architecture and Multispectral Information.

In maritime transportation, a ship's draft survey serves as a primary method for weighing bulk cargo. The accuracy of the ship's draft reading determines the fairness of bulk cargo transactions. Human visual-based draft reading methods face issues such as safety concerns, high labor costs, and subjective interpretation. Therefore, some image processing methods are utilized to achieve automatic draft reading. However, due to the limitations in the spectral characteristics of RGB images, existing image processing methods are susceptible to water surface environmental interference, such as reflections. To solve this issue, we obtained and annotated 524 multispectral images of a ship's draft as the research dataset, marking the first application of integrating NIR information and RGB images for automatic draft reading tasks. Additionally, a dual-branch backbone named BIF is proposed to extract and combine spectral information from RGB and NIR images. The backbone network can be combined with the existing segmentation head and detection head to perform waterline segmentation and draft detection. By replacing the original ResNet-50 backbone of YOLOv8, we reached a mAP of 99.2% in the draft detection task. Similarly, combining UPerNet with our dual-branch backbone, the mIoU of the waterline segmentation task was improved from 98.9% to 99.3%. The inaccuracy of the draft reading is less than ±0.01 m, confirming the efficacy of our method for automatic draft reading tasks.

A Review of Vision-Based Pothole Detection Methods Using Computer Vision and Machine Learning.

Potholes and other road surface damages pose significant risks to vehicles and traffic safety. The current methods of in situ visual inspection for potholes or cracks are inefficient, costly, and hazardous. Therefore, there is a pressing need to develop automated systems for assessing road surface conditions, aiming to efficiently and accurately reconstruct, recognize, and locate potholes. In recent years, various methods utilizing (a) computer vision, (b) three-dimensional (3D) point clouds, or (c) smartphone data have been employed to map road surface quality conditions. Machine learning and deep learning techniques have increasingly enhanced the performance of these methods. This review aims to provide a comprehensive overview of cutting-edge computer vision and machine learning algorithms for pothole detection. It covers topics such as sensing systems for acquiring two-dimensional (2D) and 3D road data, classical algorithms based on 2D image processing, segmentation-based algorithms using 3D point cloud modeling, machine learning, deep learning algorithms, and hybrid approaches. The review highlights that hybrid methods combining traditional image processing and advanced machine learning techniques offer the highest accuracy in pothole detection. Machine learning approaches, particularly deep learning, demonstrate superior adaptability and detection rates, while traditional 2D and 3D methods provide valuable baseline techniques. By reviewing and evaluating existing vision-based methods, this paper clarifies the current landscape of pothole detection technologies and identifies opportunities for future research and development. Additionally, insights provided by this review can inform the design and implementation of more robust and effective systems for automated road surface condition assessment, thereby contributing to enhanced roadway safety and infrastructure management.

Computer-Simulated Virtual Image Datasets to Train Machine Learning Models for Non-Invasive Fish Detection in Recirculating Aquaculture.

Artificial Intelligence (AI) and Machine Learning (ML) can assist producers to better manage recirculating aquaculture systems (RASs). ML is a data-intensive process, and model performance primarily depends on the quality of training data. Relatively higher fish density and water turbidity in intensive RAS culture produce major challenges in acquiring high-quality underwater image data. Additionally, the manual image annotation involved in model training can be subjective, time-consuming, and labor-intensive. Therefore, the presented study aimed to simulate fish schooling behavior for RAS conditions and investigate the feasibility of using computer-simulated virtual images to train a robust fish detection model. Additionally, to expedite the model training and automate the virtual image annotation, a process flow was developed. The 'virtual model' performances were compared with models trained on real-world images and combinations of real and virtual images. The results of the study indicate that the virtual model trained solely with computer-simulated images could not perform satisfactorily (mAP = 62.8%, F1 score = 0.61) to detect fish in a real RAS environment; however, replacing a small number of the virtual images with real images in the training dataset significantly improved the model's performance. The M6 mixed model trained with 630 virtual and 70 real images (virtual-to-real image ratio: 90:10) achieved mAP and F1 scores of 91.8% and 0.87, respectively. Furthermore, the training time cost for the M6 model was seven times shorter than that for the 'real model'. Overall, the virtual simulation approach exhibited great promise in rapidly training a reliable fish detection model for RAS operations.

A Study on the 3D Reconstruction Strategy of a Sheep Body Based on a Kinect v2 Depth Camera Array.

Non-contact measurement based on the 3D reconstruction of sheep bodies can alleviate the stress response in sheep during manual measurement of body dimensions. However, data collection is easily affected by environmental factors and noise, which is not conducive to practical production needs. To address this issue, this study proposes a non-contact data acquisition system and a 3D point cloud reconstruction method for sheep bodies. The collected sheep body data can provide reference data for sheep breeding and fattening. The acquisition system consists of a Kinect v2 depth camera group, a sheep passage, and a restraining pen, synchronously collecting data from three perspectives. The 3D point cloud reconstruction method for sheep bodies is implemented based on C++ language and the Point Cloud Library (PCL). It processes noise through pass-through filtering, statistical filtering, and random sample consensus (RANSAC). A conditional voxel filtering box is proposed to downsample and simplify the point cloud data. Combined with the RANSAC and Iterative Closest Point (ICP) algorithms, coarse and fine registration are performed to improve registration accuracy and robustness, achieving 3D reconstruction of sheep bodies. In the base, 135 sets of point cloud data were collected from 20 sheep. After 3D reconstruction, the reconstruction error of body length compared to the actual values was 0.79%, indicating that this method can provide reliable reference data for 3D point cloud reconstruction research of sheep bodies.

Mosquito species identification accuracy of early deployed algorithms in IDX, A vector identification tool.

Mosquito-borne diseases continue to pose a great threat to global public health systems due to increased insecticide resistance and climate change. Accurate vector identification is crucial for effective control, yet it presents significant challenges. IDX - an automated computer vision-based device capable of capturing mosquito images and outputting mosquito species ID has been deployed globally resulting in algorithms currently capable of identifying 53 mosquito species. In this study, we evaluate deployed performance of the IDX mosquito species identification algorithms using data from partners in the Southeastern United States (SE US) and Papua New Guinea (PNG) in 2023 and 2024. This preliminary assessment indicates continued improvement of the IDX mosquito species identification algorithms over the study period for individual species as well as average regional accuracy with macro average recall improving from 55.3 % [Confidence Interval (CI) 48.9, 61.7] to 80.2 % [CI 77.3, 84.9] for SE US, and 84.1 % [CI 75.1, 93.1] to 93.6 % [CI 91.6, 95.6] for PNG using a CI of 90 %. This study underscores the importance of algorithm refinement and dataset expansion covering more species and regions to enhance identification systems thereby reducing the workload for human experts, addressing taxonomic expertise gaps, and improving vector control efforts.

Computer vision and statistical insights into cycling near miss dynamics.

Across the globe, many transport bodies are advocating for increased cycling due to its health and environmental benefits. Yet, the real and perceived dangers of urban cycling remain obstacles. While serious injuries and fatalities in cycling are infrequent, "near misses"-events where a person on a bike is forced to avoid a potential crash or is unsettled by a close vehicle-are more prevalent. To understand these occurrences, researchers have turned to naturalistic studies, attaching various sensors like video cameras to bikes or cyclists. This sensor data holds the potential to unravel the risks cyclists face. Still, the sheer amount of video data often demands manual processing, limiting the scope of such studies. In this paper, we unveil a cutting-edge computer vision framework tailored for automated near-miss video analysis and for detecting various associated risk factors. Additionally, the framework can understand the statistical significance of various risk factors, providing a comprehensive understanding of the issues faced by cyclists. We shed light on the pronounced effects of factors like glare, vehicle and pedestrian presence, examining their roles in near misses through Granger causality with varied time lags. This framework enables the automated detection of multiple factors and understanding their significant weight, thus enhancing the efficiency and scope of naturalistic cycling studies. As future work, this research opens the possibility of integrating this AI framework into edge sensors through embedded AI, enabling real-time analysis.

Performance evaluation of semi-supervised learning frameworks for multi-class weed detection.

Precision weed management (PWM), driven by machine vision and deep learning (DL) advancements, not only enhances agricultural product quality and optimizes crop yield but also provides a sustainable alternative to herbicide use. However, existing DL-based algorithms on weed detection are mainly developed based on supervised learning approaches, typically demanding large-scale datasets with manual-labeled annotations, which can be time-consuming and labor-intensive. As such, label-efficient learning methods, especially semi-supervised learning, have gained increased attention in the broader domain of computer vision and have demonstrated promising performance. These methods aim to utilize a small number of labeled data samples along with a great number of unlabeled samples to develop high-performing models comparable to the supervised learning counterpart trained on a large amount of labeled data samples. In this study, we assess the effectiveness of a semi-supervised learning framework for multi-class weed detection, employing two well-known object detection frameworks, namely FCOS (Fully Convolutional One-Stage Object Detection) and Faster-RCNN (Faster Region-based Convolutional Networks). Specifically, we evaluate a generalized student-teacher framework with an improved pseudo-label generation module to produce reliable pseudo-labels for the unlabeled data. To enhance generalization, an ensemble student network is employed to facilitate the training process. Experimental results show that the proposed approach is able to achieve approximately 76% and 96% detection accuracy as the supervised methods with only 10% of labeled data in CottonWeedDet3 and CottonWeedDet12, respectively. We offer access to the source code (https://github.com/JiajiaLi04/SemiWeeds), contributing a valuable resource for ongoing semi-supervised learning research in weed detection and beyond.