Using ethnographic approaches to document, evaluate, and facilitate virtual community-engaged implementation research.

BACKGROUND: Community Advisory Boards (CABs) have been frequently used to engage diverse partners to inform research projects. Yet, evaluating the quality of engagement has not been routine. We describe a multi-method ethnographic approach documenting and assessing partner engagement in two "virtual" CABs, for which we conducted all meetings remotely. METHODS: Two research projects for increasing equitable COVID-19 testing, vaccination, and clinical trial participation for underserved communities involved remote CAB meetings. Thirty-three partners representing 17 community groups participated in 15 sessions across the two CABs facilitated by a social change organization. We developed ethnographic documentation forms to assess multiple aspects of CAB member engagement (e.g., time spent speaking, modality used, types of interactions). Documenters were trained to observe CAB sub-groups via virtual sessions. Debriefing with the documentation team after CAB meetings supported quality assurance and process refinement. CAB members completed a brief validated survey after each meeting to assess the quality and frequency of engagement. Content and rapid thematic analysis were used to analyze documentation data. Quantitative data were summarized as frequencies and means. Qualitative and quantitative findings were triangulated. RESULTS: A total of 4,540 interactions were identified across 15 meetings. The most frequent interaction was providing information (44%), followed by responding (37-38%). The quality and frequency of stakeholder engagement were rated favorably (average 4.7 of 5). Most CAB members (96%) reported good/excellent engagement. Specific comments included appreciation for the diversity of perspectives represented by the CAB members and suggestions for improved live interpretation. Debriefing sessions led to several methodological refinements for the documentation process and forms. CONCLUSION: We highlight key strategies for documenting and assessing community engagement. Our methods allowed for rich ethnographic data collection that refined our work with community partners. We recommend ongoing trainings, including debriefing sessions and routinely reviewed assessment of data to strengthen meaningful community engagement.