Guaranteed Coverage Prediction Intervals with Gaussian Process Regression.

Gaussian Process Regression (GPR) is a popular regression method, which unlike most Machine Learning techniques, provides estimates of uncertainty for its predictions. These uncertainty estimates however, are based on the assumption that the model is well-specified, an assumption that is violated in most practical applications, since the required knowledge is rarely available. As a result, the produced uncertainty estimates can become very misleading; for example the prediction intervals (PIs) produced for the 95% confidence level may cover much less than 95% of the true labels. To address this issue, this paper introduces an extension of GPR based on a Machine Learning framework called, Conformal Prediction (CP). This extension guarantees the production of PIs with the required coverage even when the model is completely misspecified. The proposed approach combines the advantages of GPR with the valid coverage guarantee of CP, while the performed experimental results demonstrate its superiority over existing methods.

Real-time construction demolition waste detection using state-of-the-art deep learning methods; single-stage vs two-stage detectors.

Central to the development of a successful waste sorting robot lies an accurate and fast object detection system. This study assesses the performance of the most representative deep-learning models for the real-time localisation and classification of Construction and Demolition Waste (CDW). For the investigation, both single-stage (SSD, YOLO) and two-stage (Faster-RCNN) detector architectures coupled with various backbone feature extractors (ResNet, MobileNetV2, efficientDet) were considered. A total of 18 models of variable depth were trained and tested on the first openly accessible CDW dataset developed by the authors of this study. This dataset consists of images of 6600 samples of CDW belonging to three object categories: brick, concrete, and tile. For an in-depth examination of the performance of the developed models under working conditions, two testing datasets containing normally and heavily stacked and adhered samples of CDW were developed. A comprehensive comparison between the different models yields that the latest version of the YOLO series (YoloV7) attains the best accuracy (mAP50:95 ≈ 70%) at the highest inference speed (<30 ms), while also exhibiting enough precision to deal with severely stacked and adhered samples of CDW. Additionally, it was observed that despite the rising popularity of single-stage detectors, apart from YoloV7, Faster-RCNN models remain the most robust in terms of exhibiting the least mAP fluctuations over the testing datasets considered.

Reliable prediction intervals with regression neural networks.

This paper proposes an extension to conventional regression neural networks (NNs) for replacing the point predictions they produce with prediction intervals that satisfy a required level of confidence. Our approach follows a novel machine learning framework, called Conformal Prediction (CP), for assigning reliable confidence measures to predictions without assuming anything more than that the data are independent and identically distributed (i.i.d.). We evaluate the proposed method on four benchmark datasets and on the problem of predicting Total Electron Content (TEC), which is an important parameter in trans-ionospheric links; for the latter we use a dataset of more than 60000 TEC measurements collected over a period of 11 years. Our experimental results show that the prediction intervals produced by our method are both well calibrated and tight enough to be useful in practice.

Reliable confidence measures for medical diagnosis with evolutionary algorithms.

Conformal Predictors (CPs) are machine learning algorithms that can provide predictions complemented with valid confidence measures. In medical diagnosis, such measures are highly desirable, as medical experts can gain additional information for each machine diagnosis. A risk assessment in each prediction can play an important role for medical decision making, in which the outcome can be critical for the patients. Several classical machine learning methods can be incorporated into the CP framework. In this paper, we propose a CP that makes use of evolved rule sets generated by a genetic algorithm (GA). The rule-based GA has the advantage of being human readable. We apply our method on two real-world datasets for medical diagnosis, one dataset for breast cancer diagnosis, which contains data gathered from fine needle aspirate of breast mass; and one dataset for ovarian cancer diagnosis, which contains proteomic patterns identified in serum. Our results on both datasets show that the proposed method is as accurate as the classical techniques, while it provides reliable and useful confidence measures.