Machine learning and lean six sigma for targeted patient-specific quality assurance of volumetric modulated arc therapy plans.

Background and purpose: Radiotherapy plans with excessive complexity exhibit higher uncertainties and worse patient-specific quality assurance (PSQA) results, while the workload of measurement-based PSQA can impact the efficiency of the radiotherapy workflow. Machine Learning (ML) and Lean Six Sigma, a process optimization method, were implemented to adopt a targeted PSQA approach, aiming to reduce workload, risk of failures, and monitor complexity. Materials and methods: Lean Six Sigma was applied using DMAIC (define, measure, analyze, improve, and control) steps. Ten complexity metrics were computed for 69,811 volumetric modulated arc therapy (VMAT) arcs from 28,612 plans delivered in our Institute (2013-2021). Outlier complexities were defined as >95th-percentile of the historical distributions, stratified by treatment. An ML model was trained to predict the gamma passing rate (GPR-3 %/1mm) of an arc given its complexity. A decision support system was developed to monitor the complexity and expected GPR. Plans at risk of PSQA failure, either extremely complex or with average GPR <90 %, were identified. The tool's impact was assessed after nine months of clinical use. Results: Among 1722 VMAT plans monitored prospectively, 29 (1.7 %) were found at risk of failure. Planners reacted by performing PSQA measurement and re-optimizing the plan. Occurrences of outlier complexities remained stable within 5 %. The expected GPR increased from a median of 97.4 % to 98.2 % (Mann-Whitney p < 0.05) due to plan re-optimization. Conclusions: ML and Lean Six Sigma have been implemented in clinical practice enabling a targeted measurement-based PSQA approach for plans at risk of failure to improve overall quality and patient safety.

Use of the DMAIC Lean Six Sigma quality improvement framework to improve beta-lactam antibiotic adequacy in the critically ill.

Beta-lactam antibiotics are widely used in the intensive care unit due to their favorable effectiveness and safety profiles. Beta-lactams given to patients with sepsis must be delivered as soon as possible after infection recognition (early), treat the suspected organism (appropriate), and be administered at a dose that eradicates the infection (adequate). Early and appropriate antibiotic delivery occurs in >90% of patients, but less than half of patients with sepsis achieve adequate antibiotic exposure. This project aimed to address this quality gap and improve beta-lactam adequacy using the Define, Measure, Analyze, Improve, and Control Lean Six Sigma quality improvement framework. A multidisciplinary steering committee was formed, which completed a stakeholder analysis to define the gap in practice. An Ishikawa cause and effect (Fishbone) diagram was used to identify the root causes and an impact/effort grid facilitated prioritization of interventions. An intervention that included bundled education with the use of therapeutic drug monitoring (TDM; i.e. drug-level testing) was projected to have the highest impact relative to the amount of effort and selected to address beta-lactam inadequacy in the critically ill. The education and TDM intervention were deployed through a Plan, Do, Study, Act cycle. In the 3 months after "go-live," 54 episodes of beta-lactam TDM occurred in 41 unique intensive care unit patients. The primary quality metric of beta-lactam adequacy was achieved in 94% of individuals after the intervention. Ninety-four percent of clinicians gauged the education provided as sufficient. The primary counterbalance of antimicrobial days of therapy, a core antimicrobial stewardship metric, was unchanged over time (favorable result; P = .73). Application of the Define, Measure, Analyze, Improve, and Control Lean Six Sigma quality improvement framework effectively improved beta-lactam adequacy in critically ill patients. The approach taken in this quality improvement project is widely generalizable to other drugs, drug classes, or settings to increase the adequacy of drug exposure.

Reducing turnaround time for routine outpatient biochemical tests through Lean Six Sigma: A case study in China.

BACKGROUND: Routine clinical biochemistry tests are crucial for clinical diagnostics and play a key role in enhancing outpatient turnover efficiency and patient satisfaction. This study aimed to implement Lean Six Sigma in the biochemistry laboratory of a hospital in China to improve efficiency and quality by reducing turnaround time. METHODS: The study was conducted from January to December 2023, using the DMAIC (Define, Measure, Analyze, Improve, Control) framework, and employed tools such as the voice of the customer, Value Stream Mapping, '5 whys' technique, Nominal Group Technique, and Pareto chart. RESULTS: The turnaround time for outpatient routine clinical biochemistry tests was reduced from 139 min to 58 min (p < 0.05), effectively increasing both patient and physician satisfaction. CONCLUSIONS: Lean Six Sigma aimed to reduce the turnaround time for biochemical tests have significant advantages. This study confirms the effectiveness of Lean Six Sigma in a Chinese clinical laboratory setting and provides guidance for optimizing efficiency in global clinical laboratories with limited implementation experience, constrained technical and equipment resources, and high demand for medical diagnostics.

A case study of lean digital transformation through robotic process automation in healthcare.

Under Taiwan's National Health Insurance (NHI) system, it's crucial for all healthcare providers to accurately submit medical expense claims to the National Health Insurance Administration (NHIA) to avoid incorrect deductions. With changes in healthcare policies and adjustments in hospital management strategies, the complexity of claiming rules has resulted in hospitals expending significant manpower and time on the medical expense claims process. Therefore, this study utilizes the Lean Six Sigma DMAIC (Define, Measure, Analyze, Improve, Control) management approach to identify wasteful and non-value-added steps in the process. Simultaneously, it introduces Robotic Process Automation (RPA) tools to replace manual operations. After implementation, the study effectively reduces the process time by 380 min and enhances Process Cycle Efficiency (PCE) from 69.07 to 95.54%. This research validates a real-world case of Lean digital transformation in healthcare institutions. It enables human resources to be allocated to more valuable and creative tasks while assisting hospitals in providing more comprehensive and patient-centric services.

Contextual factors and mechanisms in the implementation of Value Stream Mapping in breast cancer centres - A multiple case study.

INTRODUCTION: Lean, especially Value Stream Mapping is increasingly used in hospitals to optimize processes. This method, which originated in the automotive industry, enables all staff involved in the process to make it more customer-friendly. Despite the widely reported success of Lean projects, they have failed in some cases. This study investigated the contextual factors and mechanisms that contribute to a successful implementation of Value Stream Mapping. METHODS: Value Stream Mapping was applied to the discharge process in four breast cancer centers. A mixed-method approach was used in two steps. First, to verify the successful implementation, defined as time optimization, time measurement was conducted at three points in time and analyzed using an ANOVA. Second, an analysis of contextual factors was combined with a qualitative content analysis of mechanisms based on normalization process theory, using routine data, meeting protocols, field notes, and interview transcripts as data source. RESULTS: At one of the four breast cancer centers, lead- and waiting time were significantly reduced; at the others, these reductions did not occur. Failure/success cannot be explained by the size of the hospital, the number of cases or staffing levels. The variable project team composition is evident, especially leadership involvement. DISCUSSION: A comparative analysis was conducted to identify the factors that led to success. These factors were: participation of all leaders relevant to the process, in the case of the discharge process including medical and nursing leaders; dissemination of the changes from the project team to colleagues including its sense and possibility to discuss it; joint reflection of the implementation process in regular work team meetings. CONCLUSIONS: These results confirm the important role of leadership in implementation projects. Leadership support enabled the mechanisms found. The used combination of theoretical approaches from management research and implementation science determined the interpretation and should be applied more often in implementation science.

Revolutionising Quality Management in the Oral Pathology Laboratory: A Deep Dive Into the Six Sigma Methodology.

Six Sigma Foundations is a statistical standard that indicates an exceptionally high level of quality, along with a customer satisfaction management approach that intends to lower error rates and boost process efficiency. The Define, Measure, Analyse, Improve, and Control (DMAIC) approach is a fundamental component of Six Sigma and provides an organised framework for process improvement. In contrast to conventional techniques that are more manual-based, Six Sigma emphasises and focuses on making decisions based on facts and evidence. The key to the success of Six Sigma is its reliance on statistical methods. Advanced tools like Pareto charts, histograms, regression analysis, and fishbone diagrams are used ardently for the benefit of customers and to reduce the overall error rate. To support clinical decision-making, a clinical laboratory's primary responsibility is to generate test results that are accurate, repeatable, fast, and appropriately interpreted. Ensuring desired clinical outcomes must be the ultimate objective. To accomplish this goal, laboratories must prioritise cost-effectiveness while establishing and maintaining quality in all laboratory procedures. The concept of the Lean Six Sigma (LSS) methodology, which mainly centres on efficiency by discerning and eradicating actions or operations that do not provide any benefit to the organisation, is combined with the proposition of Six Sigma, which emphasises data-driven analyses and optimization. The integration of these powerful concepts aids in the overall improvement of the organisations adopting these techniques. This review provides a brief overview of the benefits of the LSS methodology and its implementation in the oral pathology laboratory.

Development of a Data-Centric Design Thinking Process for Innovative Care Delivery.

OBJECTIVES: This work aims to improve the quality of care provided to patients by equipping caregivers with comprehensive set of problem-solving tools and competencies. This is achieved through the development of a customized health design process that incorporates both human-centric and data-centric tools. BACKGROUND: To meet the growing complexity of today's clinical practice, caregivers need to be empowered with the tools and competencies necessary to address the multifaceted challenges they encounter. This has emphasized the need to broaden the traditional role of caregivers as evidence-based practitioners to include being healthcare problem-solvers and innovators who utilize their creative and critical thinking skills. METHOD: While design thinking (DT) is a popular methodology that fosters caregivers' empathy and creativity, it does not provide tools for evaluating the quality of obtained solutions. To address this gap, a problem-solving process that combines DT and data-centric tools of the Lean Six Sigma method was developed in this work. RESULTS: The evaluation of this customized design process was based on targeted competencies derived from the six aims of healthcare. The potential benefits are then highlighted through mapping the possible outputs of every phase with the targeted set of caregivers' skills. Additionally, an implementation plan was outlined for a local hospital, showcasing the potential impact this process can have in empowering caregivers with the necessary competencies to create effective and innovative solutions for care delivery. CONCLUSION: Overall, This unique approach has the potential to contribute to the ongoing effort to transform healthcare into an efficient system that meets the needs of both patients and caregivers.

Using Lean Six Sigma in a Private Hospital Setting to Reduce Trauma Orthopedic Patient Waiting Times and Associated Administrative and Consultant Caseload.

In Ireland, the extent of outpatient orthopedic waiting lists results in long waiting times for patients, delays in processing referrals, and variation in the consultant caseload. At the study site, the Define, Measure, Analyze, Improve, and Control (DMAIC) Lean Six Sigma framework was applied to evaluate sources of Non-Value-Added (NVA) activity in the process of registering and triaging patients referred to the trauma orthopedic service from the Emergency Department. A pre- (October-December 2021)/post- (April-August 2022) intervention design was employed, utilizing Gemba, Process Mapping, and the TIMWOODS tool. Embracing a person-centered approach, stakeholder Voice of Customer feedback was sought at each stage of the improvement process. Following data collection and analysis, a co-designed pilot intervention (March 2022) was implemented, consisting of a new triage template, dedicated trauma clinic slots, a consultant triage roster, and a new option to refer directly to physiotherapy services. This resulted in the total wait time of patients for review being reduced by 34%, a 51% reduction in the process steps required for registering, and an increase in orthopedic consultant clinic capacity of 22%. The reduction in NVA activities in the process and the increase in management options for triaging consultants have delivered a more efficient trauma and orthopedic pathway.

A total productive maintenance & reliability framework for an active pharmaceutical ingredient plant utilising design for Lean Six Sigma.

This research investigates the requirement for and relationship and implementation of a total productive maintenance (TPM) and Reliability Centred Maintenance (RCM) strategy within an Active Pharmaceutical Ingredient (API) Plant. This research aimed to study the tools and techniques of TPM and Reliability Engineering and then deploy a designed model to an API plant. A case study involving Design for Lean Six Sigma phases of Define, Measure, Analyse, Design, and Verify was utilised to build an API site TPM pilot program. Data was collected using interviews across Company 'X's local and Global Engineering teams. Process runtime, downtime and plant availability metrics were compiled and a new design for Total Productive Maintenance and Reliability was proposed and verified. A maintenance framework was designed to optimally incorporate Total Productive Maintenance, Reliability and Operational Excellence with an emphasis on Overall Equipment Efficiency (OEE) realizing a 33 % reduction in planned maintenance activities, a 70 % reduction in Corrective Maintenance, Cleaning for Maintenance was reduced by 50 %, the pilot maintenance area of the centrifuge has its OEE increased by 20 % and plant availability increased by two hundred and 6 h. This research highlights the importance of Total Productive Maintenance as a key component of an effective maintenance strategy and its potential to transform maintenance practices. Based on this research and results, TPM is recommended to be applied to any API manufacturing organization. A limitation of the study is that it is a single-site case study. The novelty of this research is based upon the emphasis on Reliability Engineering to remove non-value add Maintenance time from the manufacturing schedule. The Total Productive Maintenance & Reliability model designed and implemented in this research is unique in the literature and can be leveraged by engineering professionals and academics to understand the benefits of TPM.

Training Healthcare Professionals to Mitigate the Climate Challenge - Development of a Lean Six Sigma E-Learning.

This paper presents the preliminary findings of developing and evaluating a Lean Six Sigma (LSS) e-learning aimed at empowering healthcare professionals in different roles to make healthcare more sustainable. The e-learning was developed with experienced trainers and LSS experts by combining traditional LSS and environmental practices. Participants found the training engaging and confirmed they felt motivated and prepared to start applying the acquired skills and knowledge. We are now following the 39 participants to further investigate the effectiveness of LSS for mitigating the climate challenge in healthcare.

Increasing Abnormal Involuntary Movement Scale (AIMS) Screening for Tardive Dyskinesia in an Outpatient Psychiatry Clinic: A Resident-Led Outpatient Lean Six Sigma Initiative.

Objective To increase compliance with Abnormal Involuntary Movement Score (AIMS) documentation for patients taking antipsychotics to recognize and treat tardive dyskinesia in the psychiatry outpatient clinic. Methods The Lean Six Sigma quality improvement (QI) model, utilizing DMAIC steps of define, measure, analyze, improve, control, was followed. Psychiatry attendings and residents were surveyed to assess reasons for AIMS non-documentation, and they ranked their preferred solutions to increase compliance. A random sample of patient charts for individuals on antipsychotic medications was obtained to determine AIMS documentation compliance prior to and following the implementation of improvements. Results The most highly ranked solution was implementing a one-hour AIMS training session. Three months post-intervention, a random sample of 60 patient charts showed that 87% (52/60) of patients had AIMS documented which was a significant increase compared to 3% (1/30) pre-intervention (p<0.001). Conclusion An annual, one-hour AIMS training session for residents improved rates of AIMS documentation.

Improving efficiency and reducing costs in robotic surgery: a Lean Six Sigma approach to optimize turnover time.

Operating room (OR) turnover time (TOT) is the time it takes to prepare an OR for the next surgery after the previous one has been completed. Reducing OR TOT can improve the efficiency of the OR, reduce costs, and improve surgeons' and patients' satisfaction. The objective of this study is to evaluate the effectiveness of an operating room (OR) turnover time (TOT) reduction initiative using the Lean Six Sigma methodology (DMAIC) in the bariatric and thoracic service lines. Performance improvement strategies consist of simplifying steps (surgical tray optimization) and concurrent steps (parallel task execution). We compared 2-month pre-implementation vs. post-implementation. A paired t-test was used to assess whether the difference in the measurements was statistically significant. The study found that TOT was reduced by 15.6% from an average of 35.6 ± 8.1 to minutes 30.09 ± 9.7 min (p < 0.05). Specifically, in the bariatric service line, TOT was reduced by 17.15% and in the thoracic service line, TOT was reduced by 9.6%. No adverse events related to the initiative were reported. The results of this study indicate that the TOT reduction initiative was effective in reducing TOT. The efficient use of operating rooms is crucial in hospital management, as it not only impacts finances but also affects the satisfaction of surgical teams and patients. This study shows the effectiveness of Lean Six Sigma methodology in reducing TOT and improving the efficiency in the OR.

Improving the Hospital Transfer Process for Acute Type A Aortic Dissections.

Patients with acute type A aortic dissection who arrive at hospitals that lack the facilities to treat them must be transferred to a tertiary care facility to receive treatment. The transfer process involves a checkpoint at which the transfer is accepted or denied. Delays in making this decision may lead to suboptimal health outcomes. In light of this, the goal of this project was to devise a way to reduce the time to decision of transfer requests for patients with an acute type A aortic dissection. The project followed the Define-Measure-Analyze-Improve-Control (DMAIC) approach. To better understand the process, data were obtained from the University of Texas Southwestern Medical Center regarding reasons for patient transfer cancellation and the average time until a transfer was denied or accepted. After data analysis, a fishbone diagram was used to display 23 root causes of the delays in time to decision of the transfer request. These were narrowed down to the following four significant causes using a nominal voting technique: (1) no standard on disease-specific information for the handoff, (2) lack of a real-time database, (3) incompatible electronic health record system between facilities, and (4) multiple communication handoffs causing confusion. Solutions to each root cause were evaluated using a solution selection matrix. The final two solutions proposed for implementation were as follows: (1) to establish checklists of required documents and patient transfer criteria and (2) to create a regional database to provide real-time information on hospital capacity.

Increasing Frequency of Hemoglobin A1C Measurements in Hospitalized Patients With Diabetes: A Quality Improvement Project Using Lean Six Sigma.

BACKGROUND: The American Diabetes Association (ADA) recommends measuring A1C in all inpatients with diabetes if not performed in the prior three months. Our objective was to determine the impact of utilizing Lean Six Sigma to increase the frequency of A1C measurements in hospitalized patients. METHODS: We evaluated inpatients with diabetes mellitus consecutively admitted in a community hospital between January 2016 and June 2021, excluding those who had an A1C in the electronic health record (EHR) in the previous three months. Lean Six Sigma was utilized to define the extent of the problem and devise solutions. The intervention bundle delivered between November 2017 and February 2018 included (1) provider education on the utility of A1C, (2) more rapid turnaround of A1C results, and (3) an EHR glucose-management tab and insulin order set that included A1C. Hospital encounter and patient-level data were extracted from the EHR via bulk query. Frequency of A1C measurement was compared before (January 2016-November 2017) and after the intervention (March 2018-June 2021) using χ2 analysis. RESULTS: Demographics did not differ preintervention versus postintervention (mean age [range]: 70.9 [18-104] years, sex: 52.2% male, race: 57.0% white). A1C measurements significantly increased following implementation of the intervention bundle (61.2% vs 74.5%, P < .001). This level was sustained for more than two years following the initial intervention. Patients seen by the diabetes consult service (40.4% vs 51.7%, P < 0.001) and length of stay (mean: 135 hours vs 149 hours, P < 0.001) both increased postintervention. CONCLUSIONS: We demonstrate a novel approach in improving A1C in hospitalized patients. Lean Six Sigma may represent a valuable methodology for community hospitals to improve inpatient diabetes care.

Lean Six Sigma Methodologies to Reduce the Cardiac Troponin Turnaround Time in the Core Laboratory.

OBJECTIVE: Measurement of cardiac biomarker troponin T plays a vital role in the diagnosis of myocardial infarction. A quick turnaround time (TAT) for cardiac troponin T is crucial in managing patients in the emergency department and critical care unit. The goal of the study was to implement Lean Six Sigma methods to improve stat troponin T TAT compliance rate (the time specimen received in the laboratory to the reporting of results through the laboratory information system [LIS]) from 86% to 95% in an 8-month period. METHODS: We conducted a quality and process improvement project to reduce stat troponin T TAT in the core laboratory. We used a 5-stage Six Sigma methodology to simplify the laboratory work process and decrease the TAT by eliminating non-value-added steps. Data from April 2021 (baseline) and January 2022 (improved) are included in the analysis. RESULTS: In the core laboratory, we improved the TAT for the preanalytical and analytical process by eliminating the batch processing and prioritizing the stat samples. We improved the TAT for the postanalytical process by replacing manual result verification with auto result verification via an LIS. Improved stat troponin T TAT compliance rate has the potential to enhance the overall quality of patient care, especially in the emergency and critical care departments. CONCLUSION: Using Lean Six Sigma methodologies in the core laboratory, we successfully improved the stat troponin T TAT compliance rate from 86% to 95% in an 8-month period.

Establishing a Green, Lean and Six Sigma implementation model for sustainable construction industry: an analysis of driving forces through ISM-MICMAC approach.

Green Lean Six Sigma adoption (GLSA) is an emerging concept that enhances product quality, safety and the overall performance of the firms. Few studies attempted to explore and examine the DFs of GLSA towards creating a strong urge in the stakeholders for long-term economic, ecological and productivity advantages. Major 32 DFs were initially explored to fill this gap by conducting a literature review and semi-structured interviews with different experts. Then, using the Delphi method, 28 DFs pertinent to the construction industry of Pakistan were screened, and established a seven-stages structural model by employing the interpretive structural modeling (ISM) technique. Through Matriced' Impacts Croise's Multiplication Appliquee a UN Classement (MICMAC) method, the DFs were categorized into "autonomous," "dependent," and "linking" segments. The results signify that "energy efficiency," "government incentives," "waste minimization," "resources conservation and recycling," and "water efficiency" were the top five DFs for GLSA. Further, the comparison with other countries reveals that the most significant driving force, "government incentives," is substantial for GLSA in the emerging economy of Pakistan. This article enhances knowledge of significant DFs for GLSA, furnishing a considerable source for managers, professionals, and policymakers to encourage GLSA at a wider level.

Conducting Action Research to Improve Operational Efficiency in Manufacturing: The Case of a First-Tier Automotive Supplier.

Optimising available resources and minimising production costs and throughput time is vital for first-tier suppliers in the worldwide automotive sector. To develop this type of optimisation and efficiency, MAHLE applied Action Research (AR) in one of its factories located in Spain. A multidisciplinary collaborative work team was created with the aim of deploying the AR initiative in combination with Lean Manufacturing and Six Sigma tools. Four improvement and learning cycles were deployed and key performance metrics were defined to collect and measure data in order to analyse the improvements achieved. The application of the AR initiative in the production line of a power filter device enabled improvements in both production times and quality indicators in the manufacturing process. The most outstanding results were the improvements made in the decrease in initial throughput time (34.78%) and in average daily rejections (73.53%). In addition, the AR initiative generated practical and theoretical contributions for business and academia, allowing the AR initiative to be applied in other areas of the company, and contributing to the current state of the art in the industrial application of this methodology.

A new efficient and integrated pathway for patient evaluation prior to atrial fibrillation ablation.

AIMS: In this quality improvement project, a care pathway for patients considered for atrial fibrillation (AF) ablation was optimized with the goals to improve the patient journey and simultaneously integrate prospective data collection into the clinical process. METHODS AND RESULTS: The Lean Six Sigma approach was used to map the pre-existing process, identify constraints, and formulate countermeasures. The percentage of patients going through the full pre-ablation preparation that eventually underwent AF ablation, number of hospital visits and consultations, pathway compliance, and completeness of scientific data were measured before and after pathway optimization. Constraints in the process were (i) lack of standardized processes, (ii) inefficient use of resources, (iii) lack of multidisciplinary integration, (iv) lack of research integration, and (v) suboptimal communication. The impact of the corresponding countermeasures (defining a uniform process, incorporating 'go/no-go' moment, introducing a 'one-stop-shop', integrating prospective data collection, and improving communication) was studied for 33 patients before and 26 patients after pathway optimization. After optimization, the percentage of patients receiving a full pre-ablation preparation that eventually underwent AF ablation increased from 59% to 94% (P < 0.01). Fewer hospital visits (3.2 ± 1.2 vs. 2.3 ± 0.8, P = 0.01) and electrophysiologist consultations (1.8 ± 0.7 vs. 1.0 ± 0.3, P < 0.01) were required after pathway optimization. Pathway compliance and complete collection of scientific data increased significantly (3% vs. 73%, P < 0.01 and 15% vs. 73%, P < 0.01, respectively). CONCLUSION: The optimization project resulted in a more efficient evaluation of patients considered for AF ablation. The new more efficient process includes prospective data collection and facilitates easy conduct of research studies focused on improvements of patient outcomes.

Transformational Performance Improvement: Why Is Progress so Slow?

In this chapter, we identify three distinct transformational performance improvement (TPI) approaches commonly used to redesign work processes in health care organizations. We describe the unique components or tools that each approach uses to improve the delivery of health services. We also summarize what is empirically known about the effectiveness of each TPI approach according to systematic reviews and recent studies published in the peer-reviewed literature. Based on examination of this research, we discuss what knowledge is still needed to strengthen the evidence for whole system transformation. This involves the use of conceptual frameworks to assess and guide implementation efforts, and facilitators and barriers to change as revealed in a recent evaluation of one major initiative, the Lean Enterprise Transformation (LET) at the Veterans Health Administration. The analysis suggests ways in which TPI facilitators can be developed and barriers reduced to improve the effectiveness and sustainability of quality initiatives. Finally, we discuss appropriate study designs to evaluate TPI interventions that may strengthen the evidence for their effectiveness in real world practice settings.

The Lean Six Sigma Define, Measure, Analyze, Implement, Control (LSS DMAIC) Framework: An Innovative Strategy for Quality Improvement of Pharmacist Vaccine Recommendations in Community Pharmacy.

Community pharmacies represent a highly accessible and convenient setting for vaccination. However, setting-specific barriers exist which contribute to suboptimal vaccination rates, particularly for pneumococcal vaccinations. One proven quality improvement framework growing in use within healthcare settings is Lean Six Sigma (LSS). This paper describes the application of the LSS framework in select locations of a national pharmacy chain. The implementation of a training program for improved recommendation techniques to promote higher rates of pneumococcal vaccinations in high-risk adult populations is also addressed. A mixed-methods approach including pre/post quasi-experimental design and in-depth key informant interviews was used.