How accurately does the VIVO Harvester reflect actual Clinical and Translational Sciences Award-affiliated faculty member publications?

OBJECTIVE: The research tested the accuracy of the VIVO Harvester software in identifying publications authored by faculty members affiliated with a National Institutes of Health Clinical and Translational Sciences Award (CTSA) site. METHODS: Health sciences librarians created "gold standard" lists of references for the years 2001 to 2011 from PubMed for twenty-five randomly selected investigators from one CTSA site. These gold standard lists were compared to the same twenty-five investigators' reference lists produced by VIVO Harvester. The authors subjected the discrepancies between the lists to sensitivity and specificity analyses. RESULTS: The VIVO Harvester correctly identified only about 65% of the total eligible PubMed references for the years 2001-2011 for the CTSA-affiliated investigators. The identified references produced by VIVO Harvester were precise yet incomplete. The sensitivity rate was 0.65, and the specificity rate was 1.00. CONCLUSION: While the references produced by VIVO Harvester could be confirmed in PubMed, the VIVO Harvester retrieved only two-thirds of the required references from PubMed. National Institutes of Health CTSA sites will need to supplement VIVO Harvester-produced references with the expert searching skills of health sciences librarians. IMPLICATIONS: Health sciences librarians with searching skills need to alert their CTSA sites about these deficiencies and offer their skills to advance their sites' missions.