Should artificial intelligence have lower acceptable error rates than humans?

The first patient was misclassified in the diagnostic conclusion according to a local clinical expert opinion in a new clinical implementation of a knee osteoarthritis artificial intelligence (AI) algorithm at Bispebjerg-Frederiksberg University Hospital, Copenhagen, Denmark. In preparation for the evaluation of the AI algorithm, the implementation team collaborated with internal and external partners to plan workflows, and the algorithm was externally validated. After the misclassification, the team was left wondering: what is an acceptable error rate for a low-risk AI diagnostic algorithm? A survey among employees at the Department of Radiology showed significantly lower acceptable error rates for AI (6.8 %) than humans (11.3 %). A general mistrust of AI could cause the discrepancy in acceptable errors. AI may have the disadvantage of limited social capital and likeability compared to human co-workers, and therefore, less potential for forgiveness. Future AI development and implementation require further investigation of the fear of AI's unknown errors to enhance the trustworthiness of perceiving AI as a co-worker. Benchmark tools, transparency, and explainability are also needed to evaluate AI algorithms in clinical implementations to ensure acceptable performance.

Approximate entropy based on attempted steady isometric contractions with the ankle dorsal- and plantarflexors: reliability and optimal sampling frequency.

The aim of this study was to (1) examine the test-retest reliability of approximate entropy (ApEn) calculated for torque time-series from attempted steady isometric contractions performed at two different days, and (2) examine the significance of the sampling frequency for the ApEn values. Eighteen healthy young subjects (13+/-3 years, mean+/-1 S.D.) performed attempted steady isometric submaximal contractions with the ankle dorsal- and plantarflexors at two different days. Relative (ICC(3.1)) and absolute (standard error of measurement [S.E.M.], and S.E.M.%) test-retest reliability was assessed for the ApEn values calculated for torque time-series down-sampled to 30 and 100Hz, respectively. The relative reliability was generally moderate (0.360< or =ICC(3.1)< or =0.897), with an absolute reliability (S.E.M.%) of 6-14%. The mean ApEn values varied considerably depending on the applied down-sampling frequency (5-200Hz). When ApEn was used to quantify structure in the torque time-series, the relative and absolute reliability of steady isometric contractions with the ankle proved to be good in healthy young subjects. We propose that an optimal time-series down-sampling frequency exists for ApEn calculations, which will increase the sensitivity for biological system-changes, reduce adverse effects of random noise, and ensure that biological information in the signal is preserved. We recommend estimating this frequency using a variable high-pass filter-method for frequency analysis. Based on this method, the optimized time-series down-sampling frequency was around 30Hz for the isometric contractions performed with the ankle in the present study.

Assessment of torque-steadiness reliability at the ankle level in healthy young subjects: implications for cerebral palsy.

It was the primary objective of this study to investigate whether quantifying fluctuations in dorsi and plantarflexor torque during submaximal isometric contractions is a reliable measurement in young healthy subjects. A secondary objective was to investigate the reliability of the associated muscle activity (EMG) data. Eighteen young subjects (12.8 +/- 3.1 years, mean +/- 1 SD) were examined twice. At each visit, fluctuations in exerted torque (torque steadiness) and muscle activity from the tibialis anterior, gastrocnemius and soleus muscles were determined during submaximal isometric dorsi and plantarflexions. The relative reliability of the torque steadiness variables was substantial (0.80 < ICC(3.1) < 0.92), with an absolute reliability (average coefficient of variation) of 13-17%. The relative reliability of the muscle activity data was generally moderate (0.51 < ICC(3.1) < 0.90), with an absolute reliability of 6-26%. The reliability of dorsi and plantarflexion torque-steadiness measurements proved to be good in young healthy subjects.