Autonomous artificial intelligence in pediatric radiology: the use and perception of BoneXpert for bone age assessment.

BACKGROUND: The autonomous artificial intelligence (AI) system for bone age rating (BoneXpert) was designed to be used in clinical radiology practice as an AI-replace tool, replacing the radiologist completely. OBJECTIVE: The aim of this study was to investigate how the tool is used in clinical practice. Are radiologists more inclined to use BoneXpert to assist rather than replace themselves, and how much time is saved? MATERIALS AND METHODS: We sent a survey consisting of eight multiple-choice questions to 282 radiologists in departments in Europe already using the software. RESULTS: The 97 (34%) respondents came from 18 countries. Their answers revealed that before installing the automated method, 83 (86%) of the respondents took more than 2 min per bone age rating; this fell to 20 (21%) respondents after installation. Only 17/97 (18%) respondents used BoneXpert to completely replace the radiologist; the rest used it to assist radiologists to varying degrees. For instance, 39/97 (40%) never overruled the automated reading, while 9/97 (9%) overruled more than 5% of the automated ratings. The majority 58/97 (60%) of respondents checked the radiographs themselves to exclude features of underlying disease. CONCLUSION: BoneXpert significantly reduces reporting times for bone age determination. However, radiographic analysis involves more than just determining bone age. It also involves identification of abnormalities, and for this reason, radiologists cannot be completely replaced. AI systems originally developed to replace the radiologist might be more suitable as AI assist tools, particularly if they have not been validated to work autonomously, including the ability to omit ratings when the image is outside the range of validity.

Automated determination of bone age from hand X-rays at the end of puberty and its applicability for age estimation.

The BoneXpert method for automated determination of bone age from hand X-rays was introduced in 2009, covering the Greulich-Pyle bone age ranges up to 17 years for boys and 15 years for girls. This paper presents an extension of the method up to bone age 19 years for boys and 18 years for girls. The extension was developed based on images from the First Zurich Longitudinal Study of 231 healthy children born in 1954-1956 and followed with annual X-rays of both hands until adulthood. The method was validated on two cross-sectional studies of healthy children from Rotterdam and Los Angeles. We found root mean square deviations from manual rating of 0.69 and 0.45 years in these two studies for boys in the bone age range 17-19 years. For girls, the deviations were 0.75 and 0.59 years, respectively, in the bone age range 15-18 years. It is shown how the automated bone age method can be applied to infer the age probability distribution for healthy Caucasian European males. Considering a population with age 15.0-21.0 years, the method can be used to decide whether the subject is above 18 years with a false positive rate (children classified as adults) of 10 % (95% confidence interval = 7-13%) and a false negative rate of 30 % (adults classified as children). To apply this method in other ethnicities will require a study of the average of "bone age - age" at the end of puberty, i.e. how much this population is shifted relative to the Greulich-Pyle standard.

Metacarpal bone loss in patients with rheumatoid arthritis estimated by a new Digital X-ray Radiogrammetry method - initial results.

BACKGROUND: The Digital X-ray Radiogrammetry (DXR) method measures the cortical bone thickness in the shafts of the metacarpals and has demonstrated its relevance in the assessment of hand bone loss caused by rheumatoid arthritis (RA). The aim of this study was to validate a novel approach of the DXR method in comparison with the original version considering patients with RA. METHOD: The study includes 49 patients with verified RA. The new version is an extension of the BoneXpert method commonly used in pediatrics which has these characteristics: (1) It introduces a new technique to analyze the images which automatically validates the results for most images, and (2) it defines the measurement region relative to the ends of the metacarpals. The BoneXpert method measures the Metacarpal Index (MCI) at the metacarpal bone (II to IV). Additionally, the MCI is quantified by the DXR X-posure System. RESULTS: The new version correctly analyzed all 49 images, and 45 were automatically validated. The standard deviation between the MCI results of the two versions was 2.9% of the mean MCI. The average Larsen score was 2.6 with a standard deviation of 1.3. The correlation of MCI to Larsen score was -0.81 in both versions, and there was no significant difference in their ability to detect erosions. CONCLUSION: The new DXR version (BoneXpert) validated 92% of the cases automatically, while the same good correlation to RA severity could be presented compared to the old version.

Validation of adult height prediction based on automated bone age determination in the Paris Longitudinal Study of healthy children.

BACKGROUND: An adult height prediction model based on automated determination of bone age was developed and validated in two studies from Zurich, Switzerland. Varied living conditions and genetic backgrounds might make the model less accurate. OBJECTIVE: To validate the adult height prediction model on children from another geographical location. MATERIALS AND METHODS: We included 51 boys and 58 girls from the Paris Longitudinal Study of children born 1953 to 1958. Radiographs were obtained once or twice a year in these children from birth to age 18. Bone age was determined using the BoneXpert method. Radiographs in children with bone age greater than 6 years were considered, in total 1,124 images. RESULTS: The root mean square deviation between the predicted and the observed adult height was 2.8 cm for boys in the bone age range 6-15 years and 3.1 cm for girls in the bone age range 6-13 years. The bias (the average signed difference) was zero, except for girls below bone age 12, where the predictions were 0.8 cm too low. CONCLUSION: The accuracy of the BoneXpert method in terms of root mean square error was as predicted by the model, i.e. in line with what was observed in the Zurich studies.

Standardization of the Tanner-Whitehouse bone age method in the context of automated image analysis.

BACKGROUND/AIMS: The Tanner-Whitehouse (TW) method for bone age determination has been the basis for many population studies and it is used in many clinics. However, TW bone age raters can differ systematically from each other. The aim of the study was to present a new standard version of TW bone age rating implemented by the automated BoneXpert method and calibrated on the manual TW stage ratings of the First Zurich Longitudinal Study. SUBJECTS: Hand radiographs of 231 children born in 1954-1956 were recorded annually from an average age of 5-20 years. For validation, 76 X-rays of Tanner's original Gold Series from eight boys were used. RESULTS: The root mean square deviation between manual and automated TW ratings in the Zurich data was 0.67 years for boys in the TW bone age range 5-15 years and 0.63 years for girls, 5-14 years. The new standard TW rating differs systematically from two previous TW versions of the automated method, based on different raters. CONCLUSION: The new automated TW ratings show good accuracy relative to the manual ratings of the Zurich data and the Gold Series. There are significant differences between manual TW raters, an effect which is eliminated with the automated method.

Validation of automatic bone age rating in children with precocious and early puberty.

BACKGROUND AND AIMS: Manual bone age (BA) rating in precocious puberty (PP) is associated with considerable rater variability. The aim was to evaluate a new method for automated Greulich and Pyle (GP) BA determination in children with PP. METHODS: Seven hundred forty-one archived X-rays from 13 boys and 103 girls with PP or early puberty of various etiologies (age range at time of X-ray, 0.3-14.8 years; mean BA advancement, 2.3 years) were rated. Automatic rating (BoneXpert BA, or BXBA) was compared with the original manual GP rating (manual BA, or ManBA). X-rays where BXBA deviated from ManBA by more than 1.5 years were rerated by three raters, and the average was formed (ReferenceBA). RESULTS: All 741 X-rays, except nine (three images had poor quality and six were from children with a chronological age younger than 1.5 years), were analyzed automatically. The mean difference of BXBA-ManBA was -0.19 years; the SD of the differences was 0.76 years (95% confidence interval 0.72-0.80). ReferenceBA was determined for 41 images. A discrepancy from ReferenceBA greater 1.5 years was found in four images against BXBA and in 10 images against ManBA. CONCLUSION: Automated BA is efficient and reliable in children with PP.

Automatic determination of Greulich and Pyle bone age in healthy Dutch children.

BACKGROUND: Bone age (BA) assessment is a routine procedure in paediatric radiology, for which the Greulich and Pyle (GP) atlas is mostly used. There is rater variability, but the advent of automatic BA determination eliminates this. OBJECTIVE: To validate the BoneXpert method for automatic determination of skeletal maturity of healthy children against manual GP BA ratings. MATERIALS AND METHODS: Two observers determined GP BA with knowledge of the chronological age (CA). A total of 226 boys with a BA of 3-17 years and 179 girls with a BA of 3-15 years were included in the study. BoneXpert's estimate of GP BA was calibrated to agree on average with the manual ratings based on several studies, including the present study. RESULTS: Seven subjects showed a deviation between manual and automatic BA in excess of 1.9 years. They were re-rated blindly by two raters. After correcting these seven ratings, the root mean square error between manual and automatic rating in the 405 subjects was 0.71 years (range 0.66-0.76 years, 95% CI). BoneXpert's GP BA is on average 0.28 and 0.20 years behind the CA for boys and girls, respectively. CONCLUSION: BoneXpert is a robust method for automatic determination of BA.

Clinical application of automated Greulich-Pyle bone age determination in children with short stature.

BACKGROUND: Bone age (BA) rating is time consuming and highly rater dependent. OBJECTIVE: To adjust the fully automated BoneXpert method to agree with the manual Greulich and Pyle BA (GP BA) ratings of five raters and to validate the accuracy for short children. MATERIALS AND METHODS: A total of 1,097 left hand radiographs from 188 children with short stature, including growth hormone deficiency (44%) and Turner syndrome (29%) were evaluated. RESULTS: BoneXpert rejected 14 of the 1,097 radiographs, and deviated by more than 1.9 years from the operator BA for 27 radiographs. These were rerated blindly by four operators. Of the 27 new ratings, 26 were within 1.9 years of the automatic BA values. The root mean square deviation between manual and automatic rating was 0.72 years (95% CI 0.69-0.75). CONCLUSION: BoneXpert's ability to process 99% of images automatically without errors, and to obtain good agreement with an operator suggests that the method is efficient and reliable for short children.

Improving Automated Pediatric Bone Age Estimation Using Ensembles of Models from the 2017 RSNA Machine Learning Challenge.

PURPOSE: To investigate improvements in performance for automatic bone age estimation that can be gained through model ensembling. MATERIALS AND METHODS: A total of 48 submissions from the 2017 RSNA Pediatric Bone Age Machine Learning Challenge were used. Participants were provided with 12 611 pediatric hand radiographs with bone ages determined by a pediatric radiologist to develop models for bone age determination. The final results were determined using a test set of 200 radiographs labeled with the weighted average of six ratings. The mean pairwise model correlation and performance of all possible model combinations for ensembles of up to 10 models using the mean absolute deviation (MAD) were evaluated. A bootstrap analysis using the 200 test radiographs was conducted to estimate the true generalization MAD. RESULTS: The estimated generalization MAD of a single model was 4.55 months. The best-performing ensemble consisted of four models with an MAD of 3.79 months. The mean pairwise correlation of models within this ensemble was 0.47. In comparison, the lowest achievable MAD by combining the highest-ranking models based on individual scores was 3.93 months using eight models with a mean pairwise model correlation of 0.67. CONCLUSION: Combining less-correlated, high-performing models resulted in better performance than naively combining the top-performing models. Machine learning competitions within radiology should be encouraged to spur development of heterogeneous models whose predictions can be combined to achieve optimal performance.© RSNA, 2019 Supplemental material is available for this article. See also the commentary by Siegel in this issue.

A new implementation of digital X-ray radiogrammetry and reference curves of four indices of cortical bone for healthy European adults.

UNLABELLED: Digital X-ray radiogrammetry performs measurements on a hand radiograph in digital form. We present an improved implementation of the method and provide reference curves for four indices for the amount of bone. We collected 1662 hand radiographs of healthy subjects of age 9-100 years. PURPOSE: The digital X-ray radiogrammetry (DXR) method has been shown to be efficient for diagnosis of osteoporosis and for assessment of progression of rheumatoid arthritis. The aim of this work is to present a new DXR implementation and reference curves of four indices of cortical bone and to compare their relative SDs in healthy subjects at fixed age and gender. MATERIALS AND METHODS: A total of 1662 hand radiographs of healthy subjects of age 9-100 years were collected in Jena in 2001-2005. We also used a longitudinal study of 116 Danish children born in 1952 with on average 11 images taken over the age range 7 to 40 years. The new DXR method reconstructs the whole metacarpal contour so that the metacarpal lengths can be measured and used in two of the indices. The new DXR method automatically validates 97 % of the images and is implemented as a local server for PACS users. RESULTS: The Danish bone health index (BHI) data are consistent with the Jena data and also with the published BHI reference for healthy children. BHI is found to have smaller relative SD than the other three indices in the Jena cohort over the age range 20-80 years. CONCLUSION: The new DXR method is an extension of the existing BoneXpert method for children, which allows patients to be followed from childhood into adulthood with the same method. By making all four indices of cortical bone available within the same medical device, it becomes possible to decide which index has the best relation to fracture risk in future studies.

Automated determination of bone age in a modern chinese population.

Rationale and Objective. Large studies have previously been performed to set up a Chinese bone age reference, but it has been difficult to compare the maturation of Chinese children with populations elsewhere due to the potential variability between raters in different parts of the world. We re-analysed the radiographs from a large study of normal Chinese children using an automated bone age rating method to establish a Chinese bone age reference, and to compare the tempo of maturation in the Chinese with other populations. Materials and Methods. X-rays from 2883 boys and 3143 girls aged 2-20 years from five Chinese cities, taken in 2005, were evaluated using the BoneXpert automated method. Results. Chinese children reached full maturity at the same age as previously studied Asian children from Los Angeles, but 0.6 years earlier than Caucasian children in Los Angeles. The Greulich-Pyle bone age method was adapted to the Chinese population creating a new bone age scale BX-China05. The standard deviation between BX-China05 and chronologic age was 1.01 years in boys aged 8-14, and 1.08 years in girls aged 7-12. Conclusion. By eliminating rater variability, the automated method provides a reliable and efficient standard for bone age determination in China.

Automation of bone age reading and a new prediction model improve adult height prediction in children with short stature.

AIM: A new method (BX AHP) for adult height prediction (AHP), based on automated bone age (BoneXpert®, here called autBA) assessment, has been developed and validated. The aim of this study was to evaluate the performance of autBA and BX AHP in comparison with manual Greulich-Pyle bone age (manBA) and Bayley-Pinneau AHP (BP AHP) in children with untreated idiopathic short stature (ISS; including familial short stature and constitutional delay of growth and puberty). MATERIALS AND METHODS: We acquired the adult height of 190 patients (123 boys and 67 girls) with ISS and 448 (303 male and 135 female) X-rays of their left hand. RESULTS: Mean adult height was 168 ± 6 cm for boys and 155 ± 7 cm for girls. The root mean square error of AHP using BP AHP was 6.35 cm for boys and 4.55 cm for girls. BX AHP using autBA achieved 4.71 cm for boys (p = 0.0013) and 3.72 cm for girls (0.04); including parent's height improved its root mean square error to 4.46 cm (0.0001) for boys and 3.35 cm (0.02) for girls. CONCLUSION: The new AHP model performed significantly better than the BP AHP model. Including parental height further improved the performance of AHP.

The use of bone age in clinical practice - part 1.

This review examines the role of skeletal maturity ('bone age', BA) assessment in clinical practice. BA is mainly used in children with the following conditions: short stature (addressed in part 1 of this review), tall stature, early or late puberty, and congenital adrenal hyperplasia (all addressed in part 2). Various manual and automatic methods of BA assessment have been developed. Healthy tall children tend to have advanced BA and healthy short children tend to have delayed BA in comparison to chronological age. Growth hormone (GH) treatment of children with GH deficiency leads to a catch-up in BA that is usually appropriate for the height of the child. Response to GH is dependent on BA delay in young children with idiopathic short stature, and GH dosage appears to affect BA acceleration. In chronic renal failure, BA is delayed until puberty but then increases due to increased sensitivity of the growth plate to sex steroids, thus further impairing adult height. The assessment of BA provides an important contribution to the diagnostic workup and management of children with short stature.

The use of bone age in clinical practice - part 2.

If height-limiting treatment is being considered for a child with tall stature, skeletal maturity is invaluable in the selection of appropriate patients for treatment, determining appropriate age of treatment commencement, monitoring progress of treatment, and determining the expected treatment effect on adult height. In precocious puberty, bone maturation can be usefully assessed at initial diagnosis and start of treatment and at regular intervals thereafter during treatment monitoring. Together with height, bone maturation is an essential parameter for long-term treatment monitoring in congenital adrenal hyperplasia. Bone age (BA) determination in children with skeletal dysplasia is only feasible in a few disorders and estimations should be treated with caution. Radiographs of the left hand and wrist are, however, essential in the diagnosis of many skeletal disorders. Bone mineralization and measures of bone lengths, width, thickness and cortical thickness should always be evaluated in relation to a child's height and BA, especially around puberty. The use of skeletal maturity, assessed on a radiograph alone to estimate chronological age for immigration authorities or criminal courts is not recommended.

Validation and reference values of automated bone age determination for four ethnicities.

RATIONALE AND OBJECTIVES: Bone age (BA) rating is associated with a considerable rater variability, which would be eliminated with an automated computerized method. The aim of the study was to validate the BoneXpert method, an automated determination of BA, in American children of four ethnicities. MATERIALS AND METHODS: The study is based on a publicly available database of hand x-rays of healthy children, established in a previous, National Institutes of Health-funded study. Radiographs of the left hand were recorded between 1993 and 2006 in Los Angeles, including 1100 images with two independent manual BA ratings and 280 additional images for which the manual ratings were not used. Images were evenly split between Caucasian, African American, Hispanic, and Asian children, and the age range was 0-18.99 years. RESULTS: The automated method analyzed all images with BA >2.5 years for boys and >2 years for girls. The root-mean-square (RMS) error between the two manual ratings was 0.63 years, whereas the RMS deviation between the automated BA and the average of the two manual ratings was 0.61 years. The mean BA minus age was computed versus age for each sex and ethnicity. The largest deviation from zero was seen for Hispanic and Asian children older than 12 years, who were about 1 year advanced relative to the Greulich and Pyle standard. CONCLUSION: The automated method can analyze images of all ethnicities within a BA range of 2.5-17 years for boys and 2-15 years for girls, and can therefore eliminate the problem with rater variability in BA rating.

Validation of a new method for automated determination of bone age in Japanese children.

BoneXpert, an automated method for analysis of hand radiographs of children, has recently been developed and validated in European children. It determines Tanner-Whitehouse (TW) and Greulich Pyle (GP) bone ages (BA). The purpose of this work is to validate BoneXpert BA in Japanese children and determine the following two properties of the method: (1) The accuracy of the BA, i.e. the standard deviation from an experienced Japanese TW BA rater. (2) The precision of the BA, i.e. BoneXpert's ability to yield the same BA value on a repeated radiograph. The data consist of two studies: 185 radiographs of 22 normal children followed longitudinally from approximately 7 years to full maturity, and 284 radiographs of 22 patients with growth hormone deficiency treated with growth hormone and gonadotropin-releasing hormone analogue followed from an age of 4-11 years to almost full maturity. All radiographs were rated manually according to the TW-Japan system. BoneXpert processed all images, and the accuracy (SD) of TW-Japan BA was 0.72 years (95% CI 0.68-0.76). The precision error (SD) on a single determination of GP BA was 0.17 years (95% CI 0.15-0.19). It is concluded that BoneXpert performs as well in Japanese children as it does in Caucasian children. This study accomplishes a calibration of BoneXpert to the TW-Japan standard, which performs well for the entire BA range from 4 years up to full maturity.

Automatic determination of left- and right-hand bone age in the First Zurich Longitudinal Study.

BACKGROUND/AIMS: A more advanced bone age (BA) has been reported for the left hand relative to the right hand, while another study has found no such effect. The aim was to study the average difference of automated BoneXpert BA determination (left- vs. right-hand) for normal children, examine the precision of automatic BA and provide a BA reference for normal Caucasian children. METHODS: Radiographs of both hands (age range: 2-20 years) were digitised and analysed automatically to determine Greulich-Pyle BA, producing analysis results for 3,374 left-hand and 2,752 right-hand images. RESULTS: Comparison of left- and right-hand BA showed no average difference (<0.07 years, 95% confidence). The SD of the differences between left and right sides was 0.25 years for boys as well as girls, implying the precision of automated Greulich-Pyle BA determination was 0.18 years or better. Greulich-Pyle BA for boys and girls were on average 0.10 and 0.21 years below the chronological age. CONCLUSION: The left and right hand give the same BA on average and the SD between the sides is 0.25 years, indicating an excellent precision of the automated method.

Validation of bone age methods by their ability to predict adult height.

AIM: Several bone age (BA) methods are in use today. The aim of this study was to introduce a framework for assessing the validity of a BA method by its ability to predict adult height (H) and to apply it to manual ratings based on Greulich-Pyle (GP) and Tanner-Whitehouse 3 (TW) and to the fully automated BoneXpert method. MATERIAL: The study used X-rays of 232 children from the First Zurich Longitudinal Study recorded close to each anniversary. METHOD: For each height measurement (h), we calculated the growth potential (gp), defined as gp = (H-h)/H. The standard deviation of the gp prediction error for children of the same age was taken as a measure of the validity of the BA method and averaged over the age range 10-18 years for boys and 8-16 years for girls to obtain the overall gp prediction error (GPPE). RESULTS: Manual TW yielded GPPE = 1.32% [95% CI 1.28-1.36], and was significantly outperformed by manual GP with GPPE = 1.26% [1.22-1.30]. The automated rating obtained GPPE = 1.23%, and omitting radius and ulna yielded GPPE = 1.22%. CONCLUSION: Manual GP rating is better than manual TW rating in predicting adult height, and the fully automated method works as well as manual GP rating.

Clinical review: An automated method for determination of bone age.

CONTEXT: Bone age rating is associated with a considerable rater variability, which limits its usefulness in modern pediatric endocrinology. An automated computerized method would theoretically solve this problem but has been surprisingly difficult to establish. EVIDENCE ACQUISITION: We review the development of automated bone age assessment and describe how the conceptual understanding of bone age rating shifted from a rule-based theory to a more intuitive and experience-based approach. The role of the CASAS system from 1992 is described. The BoneXpert system from 2008 employs deformable models of each bone to locate the bones and extracts the component of the bone appearance related to maturity in a holistic, statistical manner. Two clinical studies have been published on its accuracy, defined as the root mean square deviation from manual rating. Other studies addressed the precision of the method, defined as its ability to give the same result on a repeated x-ray, expressed as the sd on a single measurement. EVIDENCE SYNTHESIS: The accuracy of the automated bone age determination was 0.71-0.72 yr, and the precision was 0.17-0.18 yr. More than 98.6% of the images could be analyzed. The system was validated on children with various diagnoses of short stature in the bone age range 2.5-17 yr for boys and 2-15 yr for girls. CONCLUSION: The reviewed validation studies suggest that this automated bone age determination system has adequate accuracy, precision, and efficiency to be clinically useful.