Is real-time ultrasonic bladder volume estimation reliable and valid? A systematic overview.

To assess the reliability and validity of real-time ultrasonic estimation of bladder volume we conducted an overview of the published literature identified using MEDLINE search (1966-96) and scanning of the bibliographies of known primary and review articles. Short-listed papers were classified into reliability (observer agreement) and validity (comparison of ultrasound estimation with actual bladder volume) studies. Study selection and data extraction were performed independently in duplicate. There were 81 subjects enrolled in 3 reliability studies and 504 subjects in 16 validity studies. Where reported, the index of concordance for reliability ranged from 0.923 to 1.00, while for validity it ranged from 0.914 to 0.983. However, there were several inadequacies in the design, conduct and analysis of these studies, leaving some doubt about the trustworthiness of the high levels of reliability and validity reported in the literature.

Evaluating measurement variability in clinical investigations: the case of ultrasonic estimation of urinary bladder volume.

OBJECTIVE: To evaluate the quality of studies seeking to establish measurement properties (reliability and validity) of ultrasonic estimation of urinary bladder volume. DESIGN: Online searching of the MEDLINE database between 1966 and 1995, and scanning of bibliography of known studies on ultrasonic bladder volume estimation. Study selection and study quality assessment were performed independently by two reviewers. Each article was evaluated for suitability of the reference standard, adequacy of reported blinding of the observers and appropriateness of the statistical index of concordance. The last two of these guidelines were applied to reliability studies (evaluating the relation among observed ultrasonic estimations), and all three guidelines were applied to validity studies (evaluating the relation of ultrasonic estimation with a definitive measurement). POPULATION: One hundred and twenty-five participants enrolled in the five reliability studies and 769 participants in the 27 validity studies selected for appraisal of their quality. MAIN OUTCOME MEASURE: Rate of study compliance with preset criteria for high quality. RESULTS: None of the studies complied with all of the criteria for high methodologic quality. In the five reliability studies, investigators did not report adequate blinding of observers in three (60%) and an appropriate index of reliability was not used in any. Among the 27 validity studies, there was a lack of a suitable reference standard in 6 (22%), an inadequate blinding in 25 (93%), and an inappropriate index of validity in all (100%). CONCLUSION: Based on our guidelines for quality assessment, a large proportion of studies on measurement properties was found to have inadequate methods, raising concern about the credibility of the reliability and validity estimates reported. These deficiencies highlight the lack of rigour employed in the design, conduct and analysis of reliability and validity studies, which has the potential for leading to patient mismanagement due to biases in the assessment of measurement variability in clinical investigations.