Development and testing of a resuscitation-specific measure of organizational culture for resuscitation teams.

Background: The organizational culture (shared beliefs, perceptions, and values) of teams informs their behaviours and practices. Little is known about organizational culture for resuscitation teams. Our objective was to develop a reliable and valid resuscitation-specific organizational culture instrument (ROCI) with the goal of improving team performance. Methods: Using Neonatal Resuscitation Program principles, literature review, and discussion of existing culture measures with experts, we identified organizational culture components for resuscitation and adapted existing measures to resuscitation. We developed a ROCI with five subscales (role clarity, shared-mental models, closed-loop communication, team adaptability, and psychological safety) and administered it to neonatal resuscitation team members across a hospital network. Survey psychometric assessment included reliability analyses (Cronbach's α, Pearson correlation coefficients) and validity testing (confirmatory factor analysis [CFA] and regression models examining the association of culture with implementation outcomes: climate and perceived success). Results: Across 11 hospitals there were 318 complete responses (41 % response rate). Of the 22-items tested, 18 were retained after iterative psychometric assessment. The ROCI had excellent overall reliability (Cronbach's α = 0.994) and very good subscale reliability (Cronbach's α = 0.789-0.867). The CFA goodness-of-fit statistics confirmed five constructs (subscales). At the individual-level, the ROCI and all subscales were associated with both implementation outcomes. At the hospital-level, the ROCI overall and three subscales were associated with perceived success. Conclusion: The ROCI is a reliable and valid measure of the organizational culture of resuscitation teams. Future ROCI assessments may provide a foundation to inform culture change initiatives to improve resuscitation quality and outcomes across populations and contexts.

Development of a Hospice Perceptions Instrument for Diverse Patients and Families: Establishing Content and Face Validity.

CONTEXT: For many, the perception of "hospice" is synonymous with "death." Even clinicians struggle to have conversations that distinguish between hospice and palliative care for fear that discussing hospice may diminish hope. To date, there are no existing measurement tools to evaluate patient and family perceptions of hospice care. OBJECTIVE: This research aimed to develop a Hospice Perceptions Instrument (HPI) to capture these perceptions among diverse patients and families. METHODS: Building on previous studies and literature, 79 potential items were drafted for the instrument. Our interprofessional team independently and collectively evaluated these, resulting in 36 items rated on a 5-point Likert scale. Overarching domains include (1) hospice philosophy and definitions; (2) hospice services; (3) values; and (4) counter-perceptions. Sixteen national subject matter experts from various professions and roles were invited to participate in the content-validity index and five hospice caregivers were invited to participate in face validity. RESULTS: Fourteen experts responded, with ten meeting inclusion criteria: one physician, four nurses, three social workers, and two chaplains. Six of the ten identified as Black. Three items were removed (I-CVI ranged from 0.5-06), and nine items were revised (I-CVI ranged from 0.6-07). The overall Content Validity Index (CVI) was 0.83, indicating excellent content validity. After revisions, five hospice caregivers assessed face validity and no changes were made based on feedback. CONCLUSION: Results reveal a disconnect between professional expertise and patient/family voices related to hospice perceptions. Development of this instrument invites a better understanding of perceptions leading to new opportunities for patient/family engagement.

Use of Cognitive Interviews in the Development of a Survey Assessing American Indian and Alaska Native Adult Perspectives on Genetics and Biological Specimens.

The cognitive interview process is a method to validate a survey instrument's face validity and enhance confidence in item interpretation, as well as a method to engage communities in the research process. Trained American Indian and Alaska Native (AIAN) interviewers conducted retrospective cognitive interviews at three AIAN communities to assess the item quality of a 131-item survey item that measures AIAN knowledge and attitudes on genetics and biological specimens. A cognitive interview process was used to assess cultural consonance, thought processes used when considering survey instructions, items and responses, and language preference of survey items in the development of a survey to assess public knowledge and attitudes on genetics. Content analysis was used to analyze interview data. Survey instructions, items and scales generated no cognitive difficulties. The participants noted being unfamiliar with terminology used to describe genetic and biological specimens. In several cases, the participants' written response in the survey and verbal response in the interview did not align. A resultant 52-item survey for use in AIAN communities was finalized. Cognitive interviewing is resource-intensive; however, ignoring community engagement during survey development results in inappropriate interpretations about culturally diverse populations such as AIAN peoples.

Evaluating cancer patient-reported experience measures against health literacy best practices.

PURPOSE: Positive patient experiences can lead to better adherence to cancer treatment and greater patient health outcomes. The primary aim of this descriptive study was to determine whether commonly used cancer PREMs have been developed according to health literacy best practices. The secondary and third aims were to examine the development of PREMs and to assess their comprehensiveness against principles of patient-centered care. METHODS: To assess adherence to best practice literacy principles regarding readability and understandability of commonly used cancer PREMs, three validated readability calculators and a validated instrument were utilized. To better understand how PREMs were developed, data about survey items, patient involvement, and expert consultation were collected. Finally, the Picker framework was used to evaluate the comprehensiveness of PREMs against principles of patient-centered care. RESULTS: Thirty-five PREMs studies met inclusion criteria for the study. The mean reading grade level of cancer PREMs was 9.7 (SD = 0.75, range = 8.2-11.2) with best practice recommendation being a grade 6 reading grade level. Twenty-eight PREMs were rated on understandability, with a mean score of 74% (SD = 10.6, range = 46-93%, with optimal score of greater than 80%). The mean number of items across PREMs was 49 (SD = 31, range = 13-136). Recommendations for the number of items to include in a questionnaire is 25-30 items. Most PREMs (n = 33, 94.3%) asked ≥ 1 double-barreled question. All PREMs addressed ≥ 2 patient-centered care principles. CONCLUSION: Cancer PREMs included in this study did not meet evidence-informed thresholds for readability and understandability. As such, it is possible that there may be gaps in how we understand the care experiences of low health literacy populations. Future development of PREMs should engage patients with low health literacy to ensure their perspectives are accurately captured and that PREMs are designed to meet the needs of all patients.

Creating a Culturally Safe Online Data Collection Instrument to Measure Vaccine Confidence Among Indigenous Youth: Indigenous Consensus Method.

BACKGROUND:  Participating in surveys can shape the perception of participants related to the study topic. Administering a vaccine hesitancy questionnaire can have negative impacts on participants' vaccine confidence. This is particularly true for online and cross-cultural data collection because culturally safe health education to correct misinformation is typically not provided after the administration of an electronic survey. OBJECTIVE:  To create a culturally safe, online, COVID-19 vaccine confidence survey for Indigenous youth designed to collect authentic, culturally relevant data of their vaccine experiences, with a low risk of contributing to further vaccine confusion among participants. METHODS:  Using the Aboriginal Telehealth Knowledge Circle consensus method, a team of academics, health care providers, policy makers, and community partners reviewed COVID-19 vaccine hesitancy surveys used in public health research, analyzed potential risks, and created a framework for electronic Indigenous vaccine confidence surveys as well as survey items. RESULTS:  The framework for safer online survey items is based on 2 principles, a first do-no-harm approach and applying a strengths-based lens. Relevant survey domains identified in the process include sociodemographic information, participants' connection to their community, preferred sources for health information, vaccination uptake among family members and peers, as well as personal attitudes toward vaccines. A total of 44 survey items were developed, including 5 open-ended items to improve the authenticity of the data and the analysis of the experiences of Indigenous youth. CONCLUSIONS:  Using an Indigenous consensus method, we have developed an online COVID-19 vaccine confidence survey with culturally relevant domains and reduced the risk of amplifying misinformation and negative impacts on vaccine confidence among Indigenous participants. Our approach can be adapted to other online survey development in collaboration with Indigenous communities.

A Model for Engaging Citizen Scientists: A Community-Partnered Research Collaboration to Address Inequities for Black Birthing People.

PURPOSE: Co-creation of a citizen-science research initiative with a collaborative team of community members and university-based scientists to address regional disparities in maternal and fetal health outcomes for Black birthing people. DESCRIPTION: Citizen scientist-led projects, where community members actively contribute to each discovery step, from setting a research agenda to collecting data and disseminating results, can extend community participatory research initiatives and help reconceptualize traditional research processes. The Pregnancy Collaborative is a citizen-science research initiative and one of nine scientific committees of The Pittsburgh Study-a longitudinal, community-partnered study designed to bring together collaborators to improve child thriving. ASSESSMENT: Ten community members and five university-based scientists participated during all phases of developing a citizen-scientist collaboration over an initial two-and-a-half-year period. Phases include forming the Pregnancy Collaborative and group research ethics training; co-creating a research agenda grounded in shared principles; and community-partnered data collection, analysis, and dissemination. These phases produced three key co-designed products: (1) a mission and vision statement of the Pregnancy Collaborative, (2) a Collaborative-endorsed research agenda, and (3) a citizen-scientist-executed research survey. CONCLUSION: Lessons learned from the formation of the Pregnancy Collaborative highlight the importance of equitable power distribution through bidirectional knowledge sharing and by centering intellectual effort, lived experience, and tools and resources of those affected by health inequities. Using a citizen science approach to co-designing and executing research helps us move maternal health inequity work from "research on" to "research with."

Development and assessment of analytic methods to improve the measurement of cognition in longitudinal studies of aging through the use of substudies with comprehensive neuropsychological testing.

INTRODUCTION: The Health and Retirement Study International Partner Surveys (HRS IPS) have rich longitudinal data, but the brevity of cognitive batteries is a limitation. METHODS: We used data from a substudy of the English Longitudinal Study of Ageing (ELSA) administering detailed cognitive assessments with the Harmonized Cognitive Assessment Protocol (ELSA-HCAP) (N = 1273) to inform approaches for estimating cognition in ELSA (N = 11,213). We compared two novel approaches: confirmatory factor analysis (CFA)- and regression-based prediction. RESULTS: Compared to estimates from the full HCAP battery, estimated cognitive functioning derived using regression models or CFA had high correlations (regression: r = 0.85 [95% confidence interval [CI]: 0.83 to 0.87]; CFA: r = 0.83 [95% CI: 0.81 to 0.85]) and reasonable mean squared error (regression: 0.25 [0.22 to 0.27]; CFA: 0.29 [0.26 to 0.32]) in held-out data. The use of additional items from waves 7 to 9 improved performance. DISCUSSION: Both approaches are recommended for future research; the similarity in approaches may be due to the brevity of available cognitive assessments in ELSA. HIGHLIGHTS: Estimates of cognitive functioning informed by English Longitudinal Study of Ageing-Harmonized Cognitive Assessment Protocol (ELSA-HCAP) data had an adequate performance. Standard errors were smaller for associations with example risks when using measures informed by ELSA-HCAP. Performance was better when including additional cognitive measures available in waves 7 to 9. Conceptual advantages to the confirmatory factor analysis (CFA) approach were not important in practice due to the brevity of the ELSA cognitive battery.

Development of a Reliable and Valid Survey Instrument to Measure Nurses' Health Behaviors Towards an Infectious Disease.

Background and Purpose: The purpose of this study is to describe the process for developing a reliable and valid survey instrument guided by the protection motivation theory (PMT) to evaluate nurses' health behaviors toward an infectious disease such as Ebola. Methods: The instrument was developed and tested through a systematic process that included a literature review, focus group, validity testing, and reliability testing. Results: The outcome variable, protection motivation, contained two elements, determined by principal component analysis. The instrument's internal consistency had a Cronbach's alpha of .80 or greater. Conclusion: The development and testing of an instrument based on PMT constructs as the theoretical framework have demonstrated a relationship between the perceived threat toward the disease and the proposed coping process needed to address the disease.

Developing a Measure of Key Adult Outcomes in Adults with Developmental Disabilities: Conceptual Model and Item Generation of the REALS (Relationships, Employment, Autonomy, and Life Satisfaction).

Employment, social relationships, and autonomy are priorities to people with intellectual and developmental disabilities (IDDs). However, few validated measures exist to systematically assess these key adult outcomes in this population. This research includes first steps to develop self- and proxy report measures of life outcomes for adults with IDDs-the Relationships, Employment, Autonomy, and Life Satisfaction (REALS). A literature search identified existing adult outcome measures, and comparison of their domains informed initial conceptual model development. External consultants revised the model, and items were generated. Autistic adults (n = 15), adults with other IDDs (n = 7), caregivers of autistic adults (n = 13), and caregivers of adults with other IDDs (n = 10) completed in-depth cognitive interviews to assess comprehension of items and response categories, factors influencing how participants respond to items, and the inclusiveness of the item pool. A final conceptual model was generated with three subdomains (social relationships, employment, and autonomy), including assessment of life satisfaction within each domain. Cognitive interviews revealed that response set restructuring and item-level revisions were needed to capture the complexity of adult life and make the measure more accessible across a range of abilities. This study developed a conceptual model of relationships, work, and autonomy specific to adults with IDDs. Future work will involve collecting data from 800 + self-reporters with IDDs and 800 + caregivers of adults with IDDs to conduct psychometric analyses. Improving measurement in this area is critical to better understanding the needs of adults with IDDs and improving services available to them.

Child Care as a Barrier to Perinatal Health Care in Illinois.

PURPOSE: Within a multi-state Collaborative Improvement and Innovation Network addressing the social determinants of health during 2017-2020, the Illinois Department of Public Health led an exploratory project to understand how the availability of child care affects maternal health care utilization. The project assessed whether lack of child care was a barrier to perinatal health care utilization and gathered information on health facility practices, resources, and policies related to child care DESCRIPTION: TWe surveyed (1) birthing hospitals (n = 98), (2) federally qualified health centers (FQHCs) (n = 40), and (3) a convenience sample of postpartum persons (n = 60). ASSESSMENT: Each group reported that child care concerns negatively affect health care utilization (66% of birthing hospitals, 50% of FQHCs, and 32% of postpartum persons). Among postpartum persons, the most common reported reason for missing a visit due to child care issues was "not feeling comfortable leaving my child(ren) in the care of others" (22%). The most common child care resource reported by facilities was "staff watching children" (53% of birthing hospitals, 75% of FQHCs); however, most did not have formal child care policies or dedicated space for children. Fewer than half of FQHCs (43%) discussed child care at the first prenatal visit. CONCLUSION: The project prompted the Illinois Title V program to add a child care-related strategy to their 2021-2025 Action Plan, providing opportunity for further examination of practices and policies that could be implemented to reduce child care barriers to perinatal care. Systematically addressing child care in health care settings may improve health care utilization among birthing/postpartum persons.

Development of the Ready to Care Survey for VA Women's Health Primary Care Provider.

INTRODUCTION: Women Veterans are at increased risk for poor pregnancy outcomes and are increasingly using Veteran Affairs (VA) for maternity benefits. VA Women's Health Primary Care Providers (WH-PCPs) are well positioned to improve maternal outcomes for women Veterans, yet little is understood about their experience and comfort with perinatal care. The objective of this study was to develop and validate a survey that could be utilized to assess WH-PCPs' experience, comfort, and attitudes towards perinatal care. METHODS: After a review of the literature, we adapted a previously published survey to address four content areas including clinical experience, comfort level, and attitudes towards perinatal care and knowledge of VA specific maternity services. This survey was piloted with five WH-PCPs before undergoing two rounds of content validation with content experts. Content validity indexes (CVI) were calculated based on the content experts' ratings. Qualitative feedback from the content experts were summarized and reviewed by the research team. The CVI and qualitative responses were utilized to guide the decision to revise, refine, or delete survey questions. RESULTS: After the first round of content validation, we deleted three questions, revised three questions, and add three questions to the content areas of clinical experience and comfort. In the domain of attitudes towards perinatal care, we deleted one question and revised two questions and three questions were added to the knowledge of VA specific maternity services domain. After the second round of content validation, only one question was deleted from the attitudes domain. DISCUSSION: We developed and validated the Ready to Care Survey for VA WH-PCP using two rounds of content validation. The final survey had face and content validity. This survey tool can be used to assess VA WH-PCP's knowledge and readiness in caring for Veterans of child-bearing age for operational and research needs.

Development of a novel instrument to characterize telemedicine programs in primary care.

BACKGROUND: Given the rapid deployment of telemedicine at the onset of the COVID - 19 pandemic, updated assessment methods are needed to study and characterize telemedicine programs. We developed a novel semi - structured survey instrument to systematically describe the characteristics and implementation processes of telemedicine programs in primary care. METHODS: In the context of a larger study aiming to describe telemedicine programs in primary care, a survey was developed in 3 iterative steps: 1) literature review to obtain a list of telemedicine features, facilitators, and barriers; 2) application of three evaluation frameworks; and 3) stakeholder engagement through a 2-stage feedback process. During survey refinement, items were tested against the evaluation frameworks while ensuring it could be completed within 20-25 min. Data reduction techniques were applied to explore opportunity for condensed variables/items. RESULTS: Sixty initially identified telemedicine features were reduced to 32 items / questions after stakeholder feedback. Per the life cycle framework, respondents are asked to report a month in which their telemedicine program reached a steady state, i.e., "maturation". Subsequent questions on telemedicine features are then stratified by telemedicine services offered at the pandemic onset and the reported point of maturation. Several open - ended questions allow for additional telemedicine experiences to be captured. Data reduction techniques revealed no indication for data reduction. CONCLUSION: This 32-item semi-structured survey standardizes the description of primary care telemedicine programs in terms of features as well as maturation process. This tool will facilitate evaluation of and comparisons between telemedicine programs across the United States, particularly those that were deployed at the pandemic onset.

Validation and reliability for the updated REAP-S dietary screener, (Rapid Eating Assessment of Participants, Short Version, v.2).

BACKGROUND: The American Heart Association (AHA) chose the REAP-S dietary screener in 2020 as one of three US dietary screeners recommended for integrating dietary assessment into clinical care. The REAP-S v.2 is an updated version that is aligned with the 2020-2025 US Dietary Guidelines and is easily incorporated into electronic medical records and taught to medical students. METHODS: The University of New England, Institutional Review Board, approved the study protocol. We evaluated the reliability and validity of the REAP-S v.2 scale by having first-year medical students (n = 167) complete both the REAP-S v.2 and a three-day food record and then analyzing their data with the following statistical techniques: Internal consistency was measured using Cronbach's alpha. Construct validity was assessed with exploratory factor analysis. Criterion validity was evaluated using analysis of variance (ANOVA) that explored the associations between REAP-S v.2 scale item responses and selected nutrient estimates from the food record analyses. The hierarchical cluster analysis classified healthy and unhealthy diet grouping under each subscale. Further using these groupings, cut points for "good" and "bad" diets for each of the three main REAP-S v.2 subscales (Food Sufficiency/Food Insufficiency; Healthy Eating Pattern and Low Nutrient Density Foods) were calculated using receiver operating characteristics (ROC) analysis. Students analyzed their three-day food intake records using an online USDA application called SuperTracker. RESULTS: The Cronbach's alpha measuring internal consistency was acceptable for the overall scale at 0.71. The exploratory factor analysis extracted three factors that roughly paralleled the three main subscales, suggesting construct validity. Most selected food record-derived nutrient values were significantly associated with scale items confirming criterion validity. The score cut points suggest that dietary counseling might be needed at ≤ 8, ≤ 10, and ≤ 16 for the above subscales. CONCLUSION: The REAP-S v.2 is intended for clinicians to use as a brief dietary screener with their patients. Tested in a population of first-year medical students, the REAP-S v.2 brief dietary screener showed acceptable internal consistency, criterion, and construct validity. It is easily scored and incorporated into the electronic medical record.

Evaluating cannabis exposure in survey items: Insights, strategies, and remaining challenges identified from cognitive interviewing.

Background: The diversity in characteristics of cannabis products and behavior patterns make evaluation of cannabis exposure in population-based, self-report surveys challenging. Accurate identification of cannabis exposure and related outcomes necessitates a thorough understanding of participants' interpretations of survey questions assessing cannabis consumption behaviors. Objectives: The current study utilized cognitive interviewing to gain insight on participants' interpretation of items in a self-reported survey instrument used to estimate the quantity of THC consumed in population samples. Methods: Cognitive interviewing was used to evaluate survey items assessing cannabis use frequency, routes of administration, quantity, potency, and perceived "typical patterns" of use. Ten participants ≥18 years (n = 4 cisgender-men; n = 3 cisgender-women; n = 3 non-binary/transgender) who had used cannabis plant material or concentrates in the past week were recruited to take a self-administered questionnaire and subsequently answer a series of scripted probes regarding survey items. Results: While most items presented no issues with comprehension, participants identified several areas of ambiguity in question or response item wording or in visual cues included in the survey. Generally, participants with irregular use patterns (i.e., non-daily use) reported more difficulty recalling the time or quantity of cannabis use. Findings resulted in several changes to the updated survey, including updated reference images and new quantity/frequency of use items specific to the route of administration. Conclusion: Incorporating cognitive interviewing into cannabis measurement development among a sample of knowledgeable cannabis consumers led to improvements in assessing cannabis exposure in population surveys, which may otherwise have been missed.

Developing conceptually sound items for a clinical courage questionnaire.

INTRODUCTION: Clinical courage can be described as a rural doctor's adaptability and willingness to undertake clinical work at the limits of their training and experience to meet the needs of their patients. This article describes the in-house development of survey items to include in a quantitative measure of clinical courage. METHODS: The questionnaire development involved two key concepts: a second-order latent factor model structure and a nominal group technique, used to develop consensus among the research team members. RESULTS: The steps taken to develop a sound clinical courage questionnaire are described in detail. The resulting initial questionnaire is presented, ready for testing with rural clinicians and refinement. CONCLUSION: This article outlines the psychometric process of questionnaire design and presents the resultant clinical courage questionnaire.

Development of a Survey Tool: Understanding the Patient Experience With Personalized 3D Models in Surgical Patient Education.

BACKGROUND: Three-dimensional (3D) printing has been increasingly utilized in the healthcare sector for many applications including guiding surgical procedures, creating medical devices, and producing custom prosthetics. As personalized medicine becomes more accessible and desired, 3D printed models emerge as a potential tool in providing patient-specific education. These personalized 3D models are at the intersection of technological innovation and medical education. Our study group utilized a modified Delphi process to create a comprehensive survey tool assessing patient experience with personalized 3D models in preoperative education. METHODS: A rigorous literature review was conducted of prior patient education survey tools in surgical cases across specialties involving personalized 3D printed models. Through categorization and mapping, a core study team reviewed individual questions, removed duplicates, and edited them into generalizable form. A modified Delphi process was then used to solicit feedback on question clarity and relevance from both 3D printing healthcare experts and patients to create a final survey.  Results: 173 survey questions from the literature were evaluated by the core study team, yielding 31 unique questions for further review. After multiple rounds of feedback, a final survey containing 18 questions was developed.  Conclusion: 3D printed models have the potential to be helpful tools in surgical patient education, and there exists a need to standardize the assessment of patient experience with these models. This survey provides a standardized, generalizable way to investigate the patient experience with personalized 3D-printed models.

Use of a focus group-based cognitive interview methodology to validate a cooking behavior survey among African-American adults.

Disparities in diet-related diseases persist among African-Americans despite advances in risk factor identification and evidence-based management strategies. Cooking is a dietary behavior linked to improved dietary quality and cardiometabolic health outcomes. However, epidemiologic studies suggest that African-American adults report a lower frequency of cooking at home when compared to other racial groups, despite reporting on average cooking time. To better understand cooking behavior among African-Americans and reported disparities in behavior, we sought to develop a survey instrument using focus group-based cognitive interviews, a pretesting method that provides insights into a survey respondent's interpretation and mental processing of survey questions. A comprised survey instrument was developed based on input from a community advisory board, a literature review, and a content review by cooking behavior experts. The cognitive interview pretesting of the instrument involved African-American adults (n = 11) at risk for cardiovascular disease who were recruited from a community-based participatory research study in Washington, D.C., to participate in a focus group-based cognitive interview. Cognitive interview methodologies included the verbal think-aloud protocol and the use of retrospective probes. Thematic analysis and evaluation of verbalized cognitive processes were conducted using verbatim transcripts. Five thematic themes related to the survey were generated: (1) Clarity and relevancy of question items; (2) influence of participants' perspectives and gender roles; (3) participant social desirability response to questions; (4) concern regarding question intent. Eleven survey items were determined as difficult by participants. Cooking topics for these items were: cooking practices, cooking skills, cooking perception (how one defines cooking), food shopping skills, and socialization around cooking. Question comprehension and interpreting response selections were the most common problems identified. Cognitive interviews are useful for cooking research as they can evaluate survey questions to determine if the meaning of the question as intended by the researcher is communicated to the respondents-specific implications from the results that apply to cooking research include revising questions on cooking practice and skills. Focus-group-based cognitive interviews may provide a feasible method to develop culturally grounded survey instruments to help understand disparities in behavior for culturally relevant diet behaviors such as cooking.

Standardizing evaluation of patient-specific 3D printed models in surgical planning: development of a cross-disciplinary survey tool for physician and trainee feedback.

BACKGROUND: 3D printed models are becoming increasingly popular in healthcare as visual and tactile tools to enhance understanding of anatomy and pathology in medical trainee education, provide procedural simulation training, and guide surgical procedures. Patient-specific 3D models are currently being used preoperatively for trainee medical education in planning surgical approaches and intraoperatively to guide decision-making in several specialties. Our study group utilized a modified Delphi process to create a standardized assessment for trainees using patient-specific 3D models as a tool in medical education during pre-surgical planning. METHODS: A literature review was conducted to identify survey questions administered to clinicians in published surgical planning studies regarding the use of patient-specific 3D models. A core study team reviewed these questions, removed duplicates, categorized them, mapped them to overarching themes, and, where applicable, modified individual questions into a form generalizable across surgical specialties. The core study panel included a physician, physician-scientist, social scientist, engineer/medical student, and 3D printing lab manager. A modified Delphi process was then used to solicit feedback on the clarity and relevance of the individual questions from an expert panel consisting of 12 physicians from specialties including anesthesiology, emergency medicine, radiology, urology, otolaryngology, and obstetrics/gynecology. When the Radiological Society of North America (RSNA)/American College of Radiology (ACR) 3D Printing Registry Data Dictionary was released, additional survey questions were reviewed. A final cross-disciplinary survey of the utility of 3D printed models in surgical planning medical education was developed. RESULTS: The literature review identified 100 questions previously published in surveys assessing patient-specific 3D models for surgical planning. Following the review, generalization, and mapping of survey questions from these studies, a list of 24 questions was generated for review by the expert study team. Five additional questions were identified in the RSNA/ACR 3D Printing Registry Data Dictionary and included for review. A final questionnaire consisting of 20 questions was developed. CONCLUSIONS: As 3D printed models become more common in medical education, the need for standardized assessment is increasingly essential. The standardized questionnaire developed in this study reflects the interests of a variety of stakeholders in patient-specific 3D models across disciplines.