Real-Time Detection of Strawberry Ripeness Using Augmented Reality and Deep Learning.

Currently, strawberry harvesting relies heavily on human labour and subjective assessments of ripeness, resulting in inconsistent post-harvest quality. Therefore, the aim of this work is to automate this process and provide a more accurate and efficient way of assessing ripeness. We explored a unique combination of YOLOv7 object detection and augmented reality technology to detect and visualise the ripeness of strawberries. Our results showed that the proposed YOLOv7 object detection model, which employed transfer learning, fine-tuning and multi-scale training, accurately identified the level of ripeness of each strawberry with an mAP of 0.89 and an F1 score of 0.92. The tiny models have an average detection time of 18 ms per frame at a resolution of 1280 × 720 using a high-performance computer, thereby enabling real-time detection in the field. Our findings distinctly establish the superior performance of YOLOv7 when compared to other cutting-edge methodologies. We also suggest using Microsoft HoloLens 2 to overlay predicted ripeness labels onto each strawberry in the real world, providing a visual representation of the ripeness level. Despite some challenges, this work highlights the potential of augmented reality to assist farmers in harvesting support, which could have significant implications for current agricultural practices.

Trends of Augmented Reality for Agri-Food Applications.

Recent years have witnessed an increasing interest in deploying state-of-the-art augmented reality (AR) head-mounted displays (HMDs) for agri-food applications. The benefits of AR HMDs to agri-food industry stakeholders (e.g., food suppliers, retail/food service) have received growing attention and recognition. AR HMDs enable users to make healthier dietary choices, experience novel changes in their perception of taste, enhance the cooking and food shopping experience, improve productivity at work and enhance the implementation of precision farming. Therefore, although development costs are still high, the case for integration of AR in food chains appears to be compelling. This review will present the most recent developments of AR HMDs for agri-food relevant applications. The summarized applications can be clustered into different themes: (1) dietary and food nutrition assessment; (2) food sensory science; (3) changing the eating environment; (4) retail food chain applications; (5) enhancing the cooking experience; (6) food-related training and learning; and (7) food production and precision farming. Limitations of current practices will be highlighted, along with some proposed applications.