The NYU Data Catalog: a modular, flexible infrastructure for data discovery.

OBJECTIVE: Researchers at New York University (NYU) Grossman School of Medicine contacted the Health Sciences Library for help with locating large datasets for reuse. In response, the library developed and maintained the NYU Data Catalog, a public-facing data catalog that has supported not only faculty acquisition of data but also the dissemination of the products of their research in various ways. MATERIALS AND METHODS: The current NYU Data Catalog is built upon the Symfony framework with a tailored metadata schema reflecting the scope of faculty research areas. The project team curates new resources, including datasets and supporting software code, and conducts quarterly and annual evaluations to assess user interactions with the NYU Data Catalog and opportunities for growth. RESULTS: Since its launch in 2015, the NYU Data Catalog underwent a number of changes prompted by an increase in the disciplines represented by faculty contributors. The catalog has also utilized faculty feedback to enhance support of data reuse and researcher collaboration through alterations to its schema, layout, and visibility of records. DISCUSSION: These findings demonstrate the flexibility of data catalogs as a platform for enabling the discovery of disparate sources of data. While not a repository, the NYU Data Catalog is well-positioned to support mandates for data sharing from study sponsors and publishers. CONCLUSION: The NYU Data Catalog makes the most of the data that researchers share and can be harnessed as a modular and adaptable platform to promote data sharing as a cultural practice.

Clinical and Translational Science Personas: Expansion and use cases.

Twelve evidence-based profiles of roles across the Clinical and Translational Science (CTS) workforce and two patient profiles were developed by CTS Personas collaborators in 2019 as part of the CTSA Program National Center for Data to Health (CD2H). Based on feedback received from the community, CTS Personas team members collaborated to produce five additional Personas to broaden representation of the CTS workforce and enhance the existing portfolio. This paper presents the rationale and methodology used in the latest CTS Personas initiative. This work also includes an implementation scenario incorporating multiple Personas. Using the new National Institutes of Health's (NIH) Data Management and Sharing Policy as an example, we demonstrate how administrators, researchers, support staff, and all CTS collaborators can use the Personas to respond to this new policy while considering the needs of service providers and users, CTS employees with short- and long-term needs, and interdisciplinary perspectives.

Rigor and reproducibility instruction in academic medical libraries.

Background: Concerns over scientific reproducibility have grown in recent years, leading the National Institutes of Health (NIH) to require researchers to address these issues in research grant applications. Starting in 2020, training grants were required to provide a plan for educating trainees in rigor and reproducibility. Academic medical centers have responded with different solutions to fill this educational need. As experienced instructors with expertise in topics relating to reproducibility, librarians can play a prominent role in providing trainings, classes, and events to educate investigators and trainees, and bolstering reproducibility in their communities. Case Presentations: This special report summarizes efforts at five institutions to provide education in reproducibility to biomedical and life sciences researchers. Our goal is to expand awareness of the range of approaches in providing reproducibility services in libraries. Conclusions: Reproducibility education by medical librarians can take many forms. These specific programs in reproducibility education build upon libraries' existing collaborations, with funder mandates providing a major impetus. Collaborator needs shaped the exact type of educational or other reproducibility support and combined with each library's strengths to yield a diversity of offerings based on capacity and interest. As demand for and complexity of reproducibility education increases due to new institutional and funder mandates, reproducibility education will merit special attention.

Piloting a long-term evaluation of library data workshops.

BACKGROUND: Over four years of hosting library data workshops, we conducted post-workshop evaluation of attendees' satisfaction with the workshops but not longer-term follow-up. To best allocate library resources and most effectively serve the needs of our users, we sought to determine whether our data workshops were impactful and useful to our community. This paper describes a pilot project to evaluate the impact of data workshops at our academic health sciences library. CASE PRESENTATION: We surveyed individuals who signed up for data workshops between 2016 and 2019. Surveys included open-ended and multiple-choice questions, with the goal of having participants describe their motivations for taking the workshop(s) and how they ultimately used what they learned. An analysis of responses using the Applied Thematic Analysis model indicated that the workshops had an impact on the respondents, although the strength of our conclusions is limited by a relatively low response rate. CONCLUSIONS: Survey results indicated that our workshops impacted how researchers at our medical center collect and analyze data, supporting the conclusion that we should concentrate our educational efforts on providing skills-based workshops. The low response rate and time-consuming nature of the analysis point toward several improvements for future evaluation efforts, including better tracking of workshop attendees, a shorter survey with fewer open-ended questions, and survey implementation within one year of the workshop date.

A model for initiating research data management services at academic libraries.

BACKGROUND: Librarians developed a pilot program to provide training, resources, strategies, and support for medical libraries seeking to establish research data management (RDM) services. Participants were required to complete eight educational modules to provide the necessary background in RDM. Each participating institution was then required to use two of the following three elements: (1) a template and strategies for data interviews, (2) the Teaching Toolkit to teach an introductory RDM class, or (3) strategies for hosting a data class series. CASE PRESENTATION: Six libraries participated in the pilot, with between two and eight librarians participating from each institution. Librarians from each institution completed the online training modules. Each institution conducted between six and fifteen data interviews, which helped build connections with researchers, and taught between one and five introductory RDM classes. All classes received very positive evaluations from attendees. Two libraries conducted a data series, with one bringing in instructors from outside the library. CONCLUSION: The pilot program proved successful in helping participating librarians learn about and engage with their research communities, jump-start their teaching of RDM, and develop institutional partnerships around RDM services. The practical, hands-on approach of this pilot proved to be successful in helping libraries with different environments establish RDM services. The success of this pilot provides a proven path forward for libraries that are developing data services at their own institutions.

A two-tiered curriculum to improve data management practices for researchers.

BACKGROUND: Better research data management (RDM) provides the means to analyze data in new ways, effectively build on another researcher's results, and reproduce the results of an experiment. Librarians are recognized by many as a potential resource for assisting researchers in this area, however this potential has not been fully realized in the biomedical research community. While librarians possess the broad skill set needed to support RDM, they often lack specific knowledge and time to develop an appropriate curriculum for their research community. The goal of this project was to develop and pilot educational modules for librarians to learn RDM and a curriculum for them to subsequently use to train their own research communities. MATERIALS AND METHODS: We created online modules for librarians that address RDM best practices, resources and regulations, as well as the culture and practice of biomedical research. Data was collected from librarians through questions embedded in the online modules on their self-reported changes in understanding of and comfort level with RDM using a retrospective pre-post design. We also developed a Teaching Toolkit which consists of slides, a script, and an evaluation form for librarians to use to teach an introductory RDM class to researchers at their own institutions. Researchers' satisfaction with the class and intent to use the material they had learned was collected. Actual changes in RDM practices by researchers who attended was assessed with a follow-up survey administered seven months after the class. RESULTS AND DISCUSSION: The online curriculum increased librarians' self-reported understanding of and comfort level with RDM. The Teaching Toolkit, when employed by librarians to teach researchers in person, resulted in improved RDM practices. This two-tiered curriculum provides concise training and a ready-made curriculum that allows working librarians to quickly gain an understanding of RDM, and translate this knowledge to researchers through training at their own institutions.

The relative citation ratio: what is it and why should medical librarians care?

Bibliometrics is becoming increasingly prominent in the world of medical libraries. The number of presentations related to research impact at the Medical Library Association (MLA) annual meeting has been increasing in past years. Medical centers have been using institutional dashboards to track clinical performance for over a decade, and more recently, these institutional dashboards have included measures of academic performance. This commentary reviews current practices and considers the role for a newer metric, the relative citation ratio.

Utility of General and Specific Word Embeddings for Classifying Translational Stages of Research.

Conventional text classification models make a bag-of-words assumption reducing text into word occurrence counts per document. Recent algorithms such as word2vec are capable of learning semantic meaning and similarity between words in an entirely unsupervised manner using a contextual window and doing so much faster than previous methods. Each word is projected into vector space such that similar meaning words such as "strong" and "powerful" are projected into the same general Euclidean space. Open questions about these embeddings include their utility across classification tasks and the optimal properties and source of documents to construct broadly functional embeddings. In this work, we demonstrate the usefulness of pre-trained embeddings for classification in our task and demonstrate that custom word embeddings, built in the domain and for the tasks, can improve performance over word embeddings learnt on more general data including news articles or Wikipedia.

Improving data collection, documentation, and workflow in a dementia screening study.

BACKGROUND: A clinical study team performing three multicultural dementia screening studies identified the need to improve data management practices and facilitate data sharing. A collaboration was initiated with librarians as part of the National Library of Medicine (NLM) informationist supplement program. The librarians identified areas for improvement in the studies' data collection, entry, and processing workflows. CASE PRESENTATION: The librarians' role in this project was to meet needs expressed by the study team around improving data collection and processing workflows to increase study efficiency and ensure data quality. The librarians addressed the data collection, entry, and processing weaknesses through standardizing and renaming variables, creating an electronic data capture system using REDCap, and developing well-documented, reproducible data processing workflows. CONCLUSIONS: NLM informationist supplements provide librarians with valuable experience in collaborating with study teams to address their data needs. For this project, the librarians gained skills in project management, REDCap, and understanding of the challenges and specifics of a clinical research study. However, the time and effort required to provide targeted and intensive support for one study team was not scalable to the library's broader user community.

Data Day to Day: building a community of expertise to address data skills gaps in an academic medical center.

BACKGROUND: The New York University Health Sciences Library data services team had developed educational material for research data management and data visualization and had been offering classes at the request of departments, research groups, and training programs, but many members of the medical center were unaware of these library data services. There were also indications of data skills gaps in these subject areas and other data-related topics. CASE PRESENTATION: The data services team enlisted instructors from across the medical center with data expertise to teach in a series of classes hosted by the library. We hosted eight classes branded as a series called "Data Day to Day." Seven instructors from four units in the medical center, including the library, taught the classes. A multipronged outreach approach resulted in high turnout. Evaluations indicated that attendees were very satisfied with the instruction, would use the skills learned, and were interested in future classes. CONCLUSIONS: Data Day to Day met previously unaddressed data skills gaps. Collaborating with outside instructors allowed the library to serve as a hub for a broad range of data instruction and to raise awareness of library services. We plan to offer the series three times in the coming year with an expanding roster of classes.

Classifying publications from the clinical and translational science award program along the translational research spectrum: a machine learning approach.

BACKGROUND: Translational research is a key area of focus of the National Institutes of Health (NIH), as demonstrated by the substantial investment in the Clinical and Translational Science Award (CTSA) program. The goal of the CTSA program is to accelerate the translation of discoveries from the bench to the bedside and into communities. Different classification systems have been used to capture the spectrum of basic to clinical to population health research, with substantial differences in the number of categories and their definitions. Evaluation of the effectiveness of the CTSA program and of translational research in general is hampered by the lack of rigor in these definitions and their application. This study adds rigor to the classification process by creating a checklist to evaluate publications across the translational spectrum and operationalizes these classifications by building machine learning-based text classifiers to categorize these publications. METHODS: Based on collaboratively developed definitions, we created a detailed checklist for categories along the translational spectrum from T0 to T4. We applied the checklist to CTSA-linked publications to construct a set of coded publications for use in training machine learning-based text classifiers to classify publications within these categories. The training sets combined T1/T2 and T3/T4 categories due to low frequency of these publication types compared to the frequency of T0 publications. We then compared classifier performance across different algorithms and feature sets and applied the classifiers to all publications in PubMed indexed to CTSA grants. To validate the algorithm, we manually classified the articles with the top 100 scores from each classifier. RESULTS: The definitions and checklist facilitated classification and resulted in good inter-rater reliability for coding publications for the training set. Very good performance was achieved for the classifiers as represented by the area under the receiver operating curves (AUC), with an AUC of 0.94 for the T0 classifier, 0.84 for T1/T2, and 0.92 for T3/T4. CONCLUSIONS: The combination of definitions agreed upon by five CTSA hubs, a checklist that facilitates more uniform definition interpretation, and algorithms that perform well in classifying publications along the translational spectrum provide a basis for establishing and applying uniform definitions of translational research categories. The classification algorithms allow publication analyses that would not be feasible with manual classification, such as assessing the distribution and trends of publications across the CTSA network and comparing the categories of publications and their citations to assess knowledge transfer across the translational research spectrum.

Starting the data conversation: informing data services at an academic health sciences library.

OBJECTIVE: The research obtained information to plan data-related products and services. METHODS: Biomedical researchers in an academic medical center were selected using purposive sampling and interviewed using open-ended questions based on a literature review. Interviews were conducted until saturation was achieved. RESULTS: Interview responses informed library planners about researchers' key data issues. CONCLUSIONS: This approach proved valuable for planning data management products and services and raising library visibility among clients in the research data realm.

Task shifting interventions for cardiovascular risk reduction in low-income and middle-income countries: a systematic review of randomised controlled trials.

OBJECTIVE: To evaluate evidence from published randomised controlled trials (RCTs) for the use of task-shifting strategies for cardiovascular disease (CVD) risk reduction in low-income and middle-income countries (LMICs). DESIGN: Systematic review of RCTs that utilised a task-shifting strategy in the management of CVD in LMICs. DATA SOURCES: We searched the following databases for relevant RCTs: PubMed from the 1940s, EMBASE from 1974, Global Health from 1910, Ovid Health Star from 1966, Web of Knowledge from 1900, Scopus from 1823, CINAHL from 1937 and RCTs from ClinicalTrials.gov. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: We focused on RCTs published in English, but without publication year. We included RCTs in which the intervention used task shifting (non-physician healthcare workers involved in prescribing of medications, treatment and/or medical testing) and non-physician healthcare providers in the management of CV risk factors and diseases (hypertension, diabetes, hyperlipidaemia, stroke, coronary artery disease or heart failure), as well as RCTs that were conducted in LMICs. We excluded studies that are not RCTs. RESULTS: Of the 2771 articles identified, only three met the predefined criteria. All three trials were conducted in practice-based settings among patients with hypertension (2 studies) and diabetes (1 study), with one study also incorporating home visits. The duration of the studies ranged from 3 to 12 months, and the task-shifting strategies included provision of medication prescriptions by nurses, community health workers and pharmacists and telephone follow-up posthospital discharge. Both hypertension studies reported a significant mean blood pressure reduction (2/1 mm Hg and 30/15 mm Hg), and the diabetes trial reported a reduction in the glycated haemoglobin levels of 1.87%. CONCLUSIONS: There is a dearth of evidence on the implementation of task-shifting strategies to reduce the burden of CVD in LMICs. Effective task-shifting interventions targeted at reducing the global CVD epidemic in LMICs are urgently needed.

Leveraging technology and staffing in developing a new liaison program.

With nearly all library resources and services delivered digitally, librarians working for the New York University Health Sciences Library struggled with maintaining awareness of changing user needs, understanding barriers faced in using library resources and services, and determining knowledge management challenges across the organization. A liaison program was created to provide opportunities for librarians to meaningfully engage with users. The program was directed toward a subset of high-priority user groups to provide focused engagement with these users. Responsibility for providing routine reference service was reduced for liaison librarians to provide maximum time to engage with their assigned user communities.