Sharing of Individual Patient-Level Data by Trialists of Randomized Clinical Trials of Pharmacological Treatments for COVID-19.

This study describes access to individual patient-level data from randomized clinical trials during the COVID-19 pandemic to determine whether the intent to share what was reported in the registry, publication, or preprint was consistent with actual data access.

Transparency and reporting characteristics of COVID-19 randomized controlled trials.

BACKGROUND: In the context of the COVID-19 pandemic, randomized controlled trials (RCTs) are essential to support clinical decision-making. We aimed (1) to assess and compare the reporting characteristics of RCTs between preprints and peer-reviewed publications and (2) to assess whether reporting improves after the peer review process for all preprints subsequently published in peer-reviewed journals. METHODS: We searched the Cochrane COVID-19 Study Register and L·OVE COVID-19 platform to identify all reports of RCTs assessing pharmacological treatments of COVID-19, up to May 2021. We extracted indicators of transparency (e.g., trial registration, data sharing intentions) and assessed the completeness of reporting (i.e., some important CONSORT items, conflict of interest, ethical approval) using a standardized data extraction form. We also identified paired reports published in preprint and peer-reviewed publications. RESULTS: We identified 251 trial reports: 121 (48%) were first published in peer-reviewed journals, and 130 (52%) were first published as preprints. Transparency was poor. About half of trials were prospectively registered (n = 140, 56%); 38% (n = 95) made their full protocols available, and 29% (n = 72) provided access to their statistical analysis plan report. A data sharing statement was reported in 68% (n = 170) of the reports of which 91% stated their willingness to share. Completeness of reporting was low: only 32% (n = 81) of trials completely defined the pre-specified primary outcome measures; 57% (n = 143) reported the process of allocation concealment. Overall, 51% (n = 127) adequately reported the results for the primary outcomes while only 14% (n = 36) of trials adequately described harms. Primary outcome(s) reported in trial registries and published reports were inconsistent in 49% (n = 104) of trials; of them, only 15% (n = 16) disclosed outcome switching in the report. There were no major differences between preprints and peer-reviewed publications. Of the 130 RCTs published as preprints, 78 were subsequently published in a peer-reviewed journal. There was no major improvement after the journal peer review process for most items. CONCLUSIONS: Transparency, completeness, and consistency of reporting of COVID-19 clinical trials were insufficient both in preprints and peer-reviewed publications. A comparison of paired reports published in preprint and peer-reviewed publication did not indicate major improvement.

Improving Comparative Effectiveness Research of Complex Health Interventions: Standards from the Patient-Centered Outcomes Research Institute (PCORI).

INTRODUCTION: Complex health interventions (CHIs) are increasingly studied in comparative effectiveness research (CER), and there is a need for improvements in CHI research practices. The Patient-Centered Outcomes Research Institute (PCORI) Methodology Committee (MC) launched an effort in 2016 to develop formal guidance on this topic. OBJECTIVE: To develop a set of minimal standards for scientifically valid, transparent, and reproducible CER studies of CHIs. The standards are intended to apply to research examining a broad range of healthcare interventions including delivery system, behavior change, and other non-pharmacological interventions. METHODS: We conducted a literature review, reviewed existing methods guidance, and developed standards through an iterative process involving the MC, two panels of external research methods experts, and a 60-day public comment period. The final standards were approved by the PCORI MC and adopted by the PCORI Board of Governors on April 30, 2018. RESULTS: The final standards include the following: (1) fully describe the intervention and comparator and define their core functions, (2) specify the hypothesized causal pathways and their theoretical basis, (3) specify how adaptations to the form of the intervention and comparator will be allowed and recorded, (4) plan and describe a process evaluation, and (5) select patient outcomes informed by the causal pathway. DISCUSSION: The new standards offer three major contributions to research: (1) they provide a simple framework to help investigators address the major methodological features of a CHI study, (2) they emphasize the importance of the causal model and the need to understand how a CHI achieves its effects rather than simply measuring these effects, and (3) they require description of a CHI using the concepts of core functions and forms. While these standards apply formally to PCORI-funded CER studies, they have broad applicability.

Researchers, patients, and other stakeholders' perspectives on challenges to and strategies for engagement.

BACKGROUND: There is growing interest in patient and stakeholder engagement in research, yet limited evidence about effective methods. Since 2012, the Patient-Centered Outcomes Research Institute (PCORI) has funded patient-centered comparative effectiveness research with a requirement for engaging patients and other stakeholders as research partners in study planning, conduct, and dissemination. This requirement, unique among large healthcare research funders in the US, provides an opportunity to learn about challenges encountered and specific strategies used by PCORI-funded study teams. The primary objective of this study is to describe -- from the perspective of PCORI investigators and research partners-the most common engagement challenges encountered in the first two years of the projects and promising strategies to prevent and overcome these challenges. METHODS: Descriptive information about investigators, partners, and their engagement was collected from investigators via annual (N = 235) and mid-year (N = 40) project progress reporting to PCORI, and from their partners (N = 260) via voluntary survey. Qualitative data were analyzed using content and thematic analyses. RESULTS: Investigators and partners most often described engagement challenges in three domains: (1) infrastructure to support engagement, (2) building relationships, and (3) maintaining relationships. Infrastructure challenges related to financial and human resources, including funding support and dedicated staff, identifying diverse groups of partners, and partners' logistical needs. Challenges for both building and maintaining relationships encompass a variety of aspects of authentic, positive interactions that facilitate mutual understanding, full participation, and genuine influence on the projects. Strategies to prevent or mitigate engagement challenges also corresponded overall to the same three domains. Both groups typically described strategies more generally, with applicability to a range of challenges rather than specific actions to address only particular challenges. CONCLUSION: Meaningful engagement of patients and other stakeholders comes with challenges, as does any innovation in the research process. The challenges and promising practices identified by these investigators and partners, related to engagement infrastructure and the building and maintenance of relationships, reveal actionable areas to improve engagement, including organizational policies and resources, training, new engagement models, and supporting engagement by viewing it as an investment in research uptake and impact.

What motivates patients and caregivers to engage in health research and how engagement affects their lives: Qualitative survey findings.

BACKGROUND: US research organizations increasingly are supporting patient and stakeholder engagement in health research with a goal of producing more useful, relevant and patient-centered evidence better aligned with real-world clinical needs. The Patient-Centered Outcomes Research Institute (PCORI) engages patients, family caregivers and other health-care stakeholders, including clinicians, payers and policymakers, as active partners in prioritizing, designing, conducting and disseminating research as a key strategy to produce useful evidence for health-care decision making. OBJECTIVE: To inform effective engagement practices and policies, we sought to understand what motivates patients and caregivers to engage as partners on PCORI-funded research projects and how such engagement changed their lives. METHODS: We conducted thematic analysis of open-ended survey responses from 255 patients, family caregivers and individuals from advocacy and community-based organizations who engaged as partners on 139 PCORI-funded research projects focusing on a range of health conditions. RESULTS: Partners' motivations for engaging in research were oriented primarily towards benefiting others, including a desire to improve patients' lives and to support effective health-care interventions. In addition to feeling they made a positive difference, many partners reported direct benefits from engagement, such as new relationships and improved health habits. DISCUSSION AND CONCLUSIONS: By identifying patient and caregiver motivations for engaging in research partnerships and what they get out of the experience, our study may help research teams and organizations attract partners and foster more satisfying and sustainable partnerships. Our findings also add to evidence that engagement benefits the people involved as partners, strengthening the case for more widespread engagement.

Qualitative Methods in Patient-Centered Outcomes Research.

The Patient-Centered Outcomes Research Institute (PCORI), created to fund research guided by patients, caregivers, and the broader health care community, offers a new research venue. Many (41 of 50) first funded projects involved qualitative research methods. This study was completed to examine the current state of the science of qualitative methodologies used in PCORI-funded research. Principal investigators participated in phenomenological interviews to learn (a) how do researchers using qualitative methods experience seeking funding for, implementing and disseminating their work; and (b) how may qualitative methods advance the quality and relevance of evidence for patients? Results showed the experience of doing qualitative research in the current research climate as "Being a bona fide qualitative researcher: Staying true to research aims while negotiating challenges," with overlapping patterns: (a) researching the elemental, (b) expecting surprise, and (c) pushing boundaries. The nature of qualitative work today was explicitly described and is rendered in this article.

Evaluating patient and stakeholder engagement in research: moving from theory to practice.

Despite the growing demand for research that engages stakeholders, there is limited evidence in the literature to demonstrate its value - or return on investment. This gap indicates a general lack of evaluation of engagement activities. To adequately inform engagement activities, we need to further investigate the dividends of engaged research, and how to evaluate these effects. This paper synthesizes the literature on hypothesized impacts of engagement, shares what has been evaluated and identifies steps needed to reduce the gap between engagement's promises and the underlying evidence supporting its practice. This assessment provides explicit guidance for better alignment of engagement's promised benefits with evaluation efforts and identifies specific areas for development of evaluative measures and better reporting processes.

Value-of-information analysis within a stakeholder-driven research prioritization process in a US setting: an application in cancer genomics.

OBJECTIVE: The objective of this study was to evaluate the feasibility and outcomes of incorporating value-of-information (VOI) analysis into a stakeholder-driven research prioritization process in a US-based setting. METHODS: . Within a program to prioritize comparative effectiveness research areas in cancer genomics, over a period of 7 months, we developed decision-analytic models and calculated upper-bound VOI estimates for 3 previously selected genomic tests. Thirteen stakeholders representing patient advocates, payers, test developers, regulators, policy makers, and community-based oncologists ranked the tests before and after receiving VOI results. The stakeholders were surveyed about the usefulness and impact of the VOI findings. RESULTS: The estimated upper-bound VOI ranged from $33 million to $2.8 billion for the 3 research areas. Seven stakeholders indicated the results modified their rankings, 9 stated VOI data were useful, and all indicated they would support its use in future prioritization processes. Some stakeholders indicated expected value of sampled information might be the preferred choice when evaluating specific STUDY DESIGN: Limitations. Our study was limited by the size and the potential for selection bias in the composition of the external stakeholder group, lack of a randomized design to assess effect of VOI data on rankings, and the use of expected value of perfect information v. expected value of sample information methods. CONCLUSIONS: Value of information analyses may have a meaningful role in research topic prioritization for comparative effectiveness research in the United States, particularly when large differences in VOI across topic areas are identified. Additional research is needed to facilitate the use of more complex value of information analyses in this setting.

Getting our priorities straight: a novel framework for stakeholder-informed prioritization of cancer genomics research.

PURPOSE: Prioritization of translational research on genomic tests is critically important given the rapid pace of innovation in genomics. The goal of this study was to evaluate a stakeholder-informed priority-setting framework in cancer genomics. METHODS: An external stakeholder advisory group including patients/consumers, payers, clinicians, and test developers used a modified Delphi approach to prioritize six candidate cancer genomic technologies during a 1-day meeting. Nine qualitative priority-setting criteria were considered. We used a directed, qualitative content-analysis approach to investigate the themes of the meeting discussion. RESULTS: Stakeholders primarily discussed six of the original nine criteria: clinical benefits, population health impacts, economic impacts, analytical and clinical validity, clinical trial implementation and feasibility, and market factors. Several new priority-setting criteria were identified from the workshop transcript, including "patient-reported outcomes," "clinical trial ethics," and "trial recruitment." The new criteria were incorporated with prespecified criteria to develop a novel priority-setting framework. CONCLUSION: This study highlights key criteria that stakeholders can consider when prioritizing comparative effectiveness research for cancer genomic applications. Applying an explicit priority-setting framework to inform investment in comparative effectiveness research can help to ensure that critical factors are weighed when deciding between many potential research questions and trial designs.

Improving the efficiency and effectiveness of pragmatic clinical trials in older adults in the United States.

Pragmatic clinical trials (PCTs) seek to improve the generalizability and increase the statistical power of traditional explanatory trials. They are a major tenet of comparative effectiveness research. While a powerful study design, PCTs have been limited by high cost, modest efficiency, and limited ability to fill relevant evidence gaps. Based on an American Reinvestment and Recovery Act (ARRA) supported meeting of national stakeholders, we propose several innovations and future research that could improve the efficiency and effectiveness of such studies focused in the U.S. Innovations discussed include optimizing the use of community based practices through partnership with Practice Based Research Networks (PBRNs), using information technology to simplify PCT subject recruitment, consent and randomization processes, and utilizing linkages to large administrative databases, such as Medicare, as a mechanism to capture outcomes and other important PCT variables with lower subject and research team burden. Testing and adaptation of such innovations to PCT are anticipated to improve the public health value of these increasingly important studies.

Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement.

AIMS: Stakeholder engagement is fundamental to comparative effectiveness research (CER), but lacks consistent terminology. This paper aims to define stakeholder engagement and present a conceptual model for involving stakeholders in CER. MATERIALS #ENTITYSTARTX00026; METHODS: The definitions and model were developed from a literature search, expert input and experience with the Center for Comparative Effectiveness Research in Cancer Genomics, a proof-of-concept platform for stakeholder involvement in priority setting and CER study design. RESULTS: Definitions for stakeholder and stakeholder engagement reflect the target constituencies and their role in CER. The 'analytic-deliberative' conceptual model for stakeholder engagement illustrates the inputs, methods and outputs relevant to CER. The model differentiates methods at each stage of the project; depicts the relationship between components; and identifies outcome measures for evaluation of the process. CONCLUSION: While the definitions and model require testing before being broadly adopted, they are an important foundational step and will be useful for investigators, funders and stakeholder groups interested in contributing to CER.

The politics behind the implementation of the WTO Paragraph 6 Decision in Canada to increase global drug access.

BACKGROUND: The reform of pharmaceutical policy can often involve trade-offs between competing social and commercial goals. Canada's Access to Medicines Regime (CAMR), a legislative amendment that permits compulsory licensing for the production and export of medicines to developing countries, aimed to reconcile these goals. Since it was passed in 2004, only two orders of antiretroviral drugs, enough for 21,000 HIV/AIDS patients in Rwanda have been exported. Future use of the regime appears unlikely. This research aimed to examine the politics of CAMR. METHODS: Parliamentary Committee hearing transcripts from CAMR's legislative development (2004) and legislative review (2007) were analysed using a content analysis technique to identify how stakeholders who participated in the debates framed the issues. These findings were subsequently analysed using a framework of framing, institutions and interests to determine how these three dimensions shaped CAMR. RESULTS: In 2004, policy debates in Canada were dominated by two themes: intellectual property rights and the TRIPS Agreement. The right to medicines as a basic human right and CAMR's potential impact on innovation were hardly discussed. With the Departments of Industry Canada and International Trade as the lead institutions, the goals of protecting intellectual property and ensuring good trade relations with the United States appear to have taken priority over encouraging generic competition to achieve drug affordability. The result was a more limited interpretation of patent flexibilities under the WTO Paragraph 6 Decision. The most striking finding is the minimal discussion over the potential barriers developing country beneficiaries might face when attempting to use compulsory licensing, including their reluctance to use TRIPS flexibilities, their desire to pursue technological development and the constraints inherent in the WTO Paragraph 6 Decision. Instead, these issues were raised in 2007, which can be partly accounted for by experience in implementing the legislation and hence a greater representation of the interests of potential beneficiary country governments. CONCLUSIONS: The Canadian Government designed CAMR as a last resort measure. Increased input from the developing country beneficiaries and shifting to institutions where the right to health gets prioritized may lead to policies that better achieves affordable drug access.

Prioritization in comparative effectiveness research: the CANCERGEN Experience.

BACKGROUND: Systematic approaches to stakeholder-informed research prioritization are a central focus of comparative effectiveness research. Genomic testing in cancer is an ideal area to refine such approaches given rapid innovation and potentially significant impacts on patient outcomes. OBJECTIVE: To develop and pilot test a stakeholder-informed approach to prioritizing genomic tests for future study in collaboration with the cancer clinical trials consortium SWOG. METHODS: We conducted a landscape analysis to identify genomic tests in oncology using a systematic search of published and unpublished studies, and expert consultation. Clinically valid tests suitable for evaluation in a comparative study were presented to an external stakeholder group. Domains to guide the prioritization process were identified with stakeholder input, and stakeholders ranked tests using multiple voting rounds. RESULTS: A stakeholder group was created including representatives from patient-advocacy groups, payers, test developers, regulators, policy makers, and community-based oncologists. We identified 9 domains for research prioritization with stakeholder feedback: population impact; current standard of care, strength of association; potential clinical benefits, potential clinical harms, economic impacts, evidence of need, trial feasibility, and market factors. The landscape analysis identified 635 studies; of 9 tests deemed to have sufficient clinical validity, 6 were presented to stakeholders. Two tests in lung cancer (ERCC1 and EGFR) and 1 test in breast cancer (CEA/CA15-3/CA27.29) were identified as top research priorities. CONCLUSIONS: Use of a diverse stakeholder group to inform research prioritization is feasible in a pragmatic and timely manner. Additional research is needed to optimize search strategies, stakeholder group composition, and integration with existing prioritization mechanisms.

Informatics in action: lessons learned in comparative effectiveness research.

Comparative effectiveness research (CER) is meant to provide evidence about the relative risks and benefits of different treatment options. It is gaining visibility as a tool to address the evidence gaps that clinicians struggle with every day; however, CER is particularly challenging in oncology as there is great variability in how individuals respond to interventions, and a wide range of drugs and procedures are available. In order to overcome these obstacles and conduct reliable CER studies, it is critical to create a robust data infrastructure to support it.The Center for Medical Technology Policy held its first annual CER Summit in November 2010, with a particular focus on oncology. A number of critical informatics themes emerged including the need for consistent data standards, registry reform, tools to assist trial accrual, and data to integrate into value deliberations. Addressing the data issues highlighted in this report will provide a significant opportunity to improve the health of our medical system.

Framing access to medicines in developing countries: an analysis of media coverage of Canada's Access to Medicines Regime.

BACKGROUND: In September 2003, the Canadian government committed to developing legislation that would facilitate greater access to affordable medicines for developing countries. Over the course of eight months, the legislation, now known as Canada's Access to Medicines Regime (CAMR), went through a controversial policy development process and the newspaper media was one of the major venues in which the policy debates took place. The purpose of this study was to examine how the media framed CAMR to determine how policy goals were conceptualized, which stakeholder interests controlled the public debate and how these variables related to the public policy process. METHODS: We conducted a qualitative content analysis of newspaper coverage of the CAMR policy and implementation process from 2003-2008. The primary theoretical framework for this study was framing theory. A total of 90 articles from 11 Canadian newspapers were selected for inclusion in our analysis. A team of four researchers coded the articles for themes relating to access to medicines and which stakeholders' voice figured more prominently on each issue. Stakeholders examined included: the research-based industry, the generic industry, civil society, the Canadian government, and developing country representatives. RESULTS: The most frequently mentioned themes across all documents were the issues of drug affordability, intellectual property, trade agreements and obligations, and development. Issues such as human rights, pharmaceutical innovation, and economic competitiveness got little media representation. Civil society dominated the media contents, followed far behind by the Canadian government, the research-based and generic pharmaceutical industries. Developing country representatives were hardly represented in the media. CONCLUSIONS: Media framing obscured the discussion of some of the underlying policy goals in this case and failed to highlight issues which are now significant barriers to the use of the legislation. Using the media to engage the public in more in-depth exploration of the policy issues at stake may contribute to a more informed policy development process. The media can be an effective channel for those stakeholders with a weaker voice in policy deliberations to raise public attention to particular issues; however, the political and institutional context must be taken into account as it may outweigh media framing effects.

The role of supportive supervision on immunization program outcome - a randomized field trial from Georgia.

BACKGROUND: One of the most common barriers to improving immunization coverage rates is human resources and its management. In the Republic of Georgia, a country where widespread health care reforms have taken place over the last decade, an intervention was recently implemented to strengthen performance of immunization programs. A range of measures were taken to ensure that immunization managers carry out their activities effectively through direct, personal contact on a regular basis to guide, support and assist designated health care facility staff to become more competent in their immunization work. The aim of this study was to document the effects of "supportive" supervision on the performance of the immunization program at the district(s) level in Georgia. METHODS: A pre-post experimental research design is used for the quantitative evaluation. Data come from baseline and follow-up surveys of health care providers and immunization managers in 15 intervention and 15 control districts. These data were supplemented by focus group discussions amongst Centre of Public Health and health facility staff. RESULTS: The results of the study suggest that the intervention package resulted in a number of expected improvements. Among immunization managers, the intervention independently contributed to improved knowledge of supportive supervision, and helped remove self-perceived barriers to supportive supervision such as availability of resources to supervisors, lack of a clear format for providing supportive supervision, and lack of recognition among providers of the importance of supportive supervision. The intervention independently contributed to relative improvements in district-level service delivery outcomes such as vaccine wastage factors and the DPT-3 immunization coverage rate. The clear positive improvement in all service delivery outcomes across both the intervention and control districts can be attributed to an overall improvement in the Georgian population's access to health care. CONCLUSION: Provider-based interventions such as supportive supervision can have independent positive effects on immunization program indicators. Thus, it is recommended to implement supportive supervision within the framework of national immunization programs in Georgia and other countries in transition with similar institutional arrangements for health services organization. ABSTRACT IN RUSSIAN: See the full article online for a translation of this abstract in Russian.

Canada's implementation of the Paragraph 6 Decision: is it sustainable public policy?

BACKGROUND: Following the Implementation of Paragraph 6 of the Doha Declaration on TRIPS and Public Health, Canada was among the first countries globally to amend its patent law, which resulted in Canada's Access to Medicines Regime (CAMR). CAMR allows the production and export of generic drugs to developing countries without the requisite manufacturing capacity to undertake a domestic compulsory license. CAMR has been the subject of much criticism lodged at its inability to ensure fast access to urgent medicines for least developing and developing countries in need. Only recently did the Canadian government grant Apotex the compulsory licenses required under CAMR to produce and export antiretroviral therapy to Rwanda's population. METHODS: The objective of this research is to investigate whether the CAMR can feasibly achieve its humanitarian objectives given the political interests embedded in the crafting of the legislation. We used a political economy framework to analyze the effect of varied institutions, political processes, and economic interests on public policy outcomes. In-depth, semi-structured interviews were conducted with nineteen key stakeholders from government, civil society and industry. Qualitative data analysis was performed using open-coding for themes, analyzing by stakeholder group. RESULTS: CAMR is removed from the realities of developing countries and the pharmaceutical market. The legislation needs to include commercial incentives to galvanize the generic drug industry to make use of this legislation. CAMR assumes that developing country governments have the requisite knowledge and human resource capacity to make use of the regime, which is not the case. The legislation does not offer sufficient incentives for countries to turn to Canada when needed drugs may be procured cheaply from countries such as India. In the long term, developing and least developing countries seek sustainable solutions to meet the health needs of their population, including developing their own capacity and local industries. CONCLUSION: CAMR is symbolically meaningful but in practice, limited. The Rwanda case will be noteworthy in terms of the future of the legislation. To meet its intended international health objectives, this legislation needs to be better informed of developing country needs and global pharmaceutical market imperatives. Finally, we contend that serious public policy change cannot strike a balance between all vested interests. Above all, any feasible policy that aims to facilitate compulsory licensing must prioritize public health over trade or economic interests.

Scientific misconduct, the pharmaceutical industry, and the tragedy of institutions.

This paper examines how current legislative and regulatory models do not adequately govern the pharmaceutical industry towards ethical scientific conduct. In the context of a highly profit-driven industry, governments need to ensure ethical and legal standards are not only in place for companies but that they are enforceable. We demonstrate with examples from both industrialized and developing countries how without sufficient controls, there is a risk that corporate behaviour will transgress ethical boundaries. We submit that there is a critical need for urgent drug regulatory reform. There must be robust regulatory structures in place which enforce corporate governance mechanisms to ensure that pharmaceutical companies maintain ethical standards in drug research and development and the marketing of pharmaceuticals. What is also needed is for the pharmaceutical industry to adopt authentic "corporate social responsibility" policies as current policies and practices are insufficient.

Human resource management in the Georgian National Immunization Program: a baseline assessment.

BACKGROUND: Georgia's health care system underwent dramatic reform after gaining independence in 1991. The decentralization of the health care system was one of the core elements of health care reform but reports suggest that human resource management issues were overlooked. The Georgian national immunization program was affected by these reforms and is not functioning at optimum levels. This paper describes the state of human resource management practices within the Georgian national immunization program in late 2004. METHODS: Thirty districts were selected for the study. Within these districts, 392 providers and thirty immunization managers participated in the study. Survey questionnaires were administered through face-to-face interviews to immunization managers and a mail survey was administered to immunization providers. Qualitative data collection involved four focus groups. Analysis of variance (ANOVA) and Chi-square tests were used to test for differences between groups for continuous and categorical variables. Content analysis identified main themes within the focus groups. RESULTS: Weak administrative links exist between the Centres of Public Health (CPH) and Primary Health Care (PHC) health facilities. There is a lack of clear management guidelines and only 49.6% of all health providers had written job descriptions. A common concern among all respondents was the extremely inadequate salary. Managers cited lack of authority and poor knowledge and skills in human resource management. Lack of resources and infrastructure were identified as major barriers to improving immunization. CONCLUSION: Our study found that the National Immunization Program in Georgia was characterized by weak organizational structure and processes and a lack of knowledge and skills in management and supervision, especially at peripheral levels. The development of the skills and processes of a well-managed workforce may help improve immunization rates, facilitate successful implementation of remaining health care reforms and is an overall, wise investment. However, reforms at strategic policy levels and across sectors will be necessary to address the systemic financial and health system constraints impeding the performance of the immunization program and the health care system as a whole.