Effectiveness of public health quality improvement training approaches: application, application, application.

CONTEXT: Quality improvement (QI) has been identified as a key strategy to improve the performance of state and local public health agencies. Quality improvement training effectiveness has received little attention in the literature. OBJECTIVES: To evaluate the effectiveness of 3 QI training types: webinars, workshops, and demonstration site activities on improving participant knowledge, skill, and ability to conduct QI through a questionnaire conducted after training participation. DESIGN: We used a natural experimental design hypothesizing that demonstration site participants would have the greatest gains on outcomes of interest compared with webinar and workshop participants. Bivariate and multivariate models were used to examine outcome differences between questionnaire respondents who participated in various training types. PARTICIPANTS: Local health department employees who participated in the 3 training strategies. MAIN OUTCOME MEASURES: Measures included knowledge and skill gain, skill application, QI receptivity, and ability to successfully participate in a QI project. RESULTS: Two hundred eighty-four unique individuals who work in 143 health departments completed the questionnaire for a 59% response rate. The majority of these health departments serve midsize populations. Demonstration site respondents had significantly greater gains in knowledge and skills, skill application, and ability to successfully participate in a QI project. Webcast training participants had significantly higher QI receptivity in adjusted models. Respondents who participated in both webcast and demonstration site trainings had higher mean scores on all outcomes when compared with demonstration site single training participants, these differences were significant in unadjusted models. CONCLUSION: Our findings suggest that QI training for public health agency employees should include both didactic training on QI content and opportunities for QI application. Future research should examine if this approach can effectively increase successful participation in QI projects for staff in LHDs of all sizes.

The prevalence of evidence-based drug use prevention curricula in U.S. middle schools in 2008.

The No Child Left Behind Act mandates the implementation of evidence-based drug prevention curricula in the nation's schools. The purpose of this paper is to estimate changes in the prevalence of such curricula from 2005 to 2008. We surveyed school staff in a nationally representative sample of schools with middle school grades. Using a web-based approach to data collection that we supplemented by telephone calls, we secured data from 1892 schools for a response rate of 78.2%. We estimate that the prevalence of evidence-based drug prevention curricula rose from 42.6% in 2005 to 46.9% in 2008, and that the prevalence of schools that used these curricula most frequently increased from 22.7% to 25.9% over this period. In addition, the proportion of schools using locally developed curricula also rose, from 17.6% to 28.1%. This study suggests the success of efforts by the Office of Safe and Drug-Free Schools to increase the prevalence of evidence-based curricula, as well as the need to continue to track the prevalence of these curricula in response to any reductions in the Office's fiscal support for evidence-based drug prevention curricula in the nation's schools.

Age Verification and Online Sales of Little Cigars and Cigarillos to Minors.

OBJECTIVES: In our cross-sectional study, we aimed to determine age verification and sales of little cigars and cigarillos (LCCs) online to underage teens. METHODS: We selected 100 popular Internet Little Cigar and Cigarillo Vendors (ILVs) for order attempts. From August to December 2015, we supervised 14 teens 14-17 years old making order attempts for LCCs. RESULTS: Of the 91 valid orders attempted, we received 89. For the valid orders attempted, 9.9% of ILVs used no age verification strategies at all, 84.6% used less effective forms of age verification, and 50.5% used more effective ones. Only one order was blocked during the order attempt and only one attempt was made to verify age at delivery. Most (79.8%) deliveries were left at the door and only 2 order attempts were rejected because of age verification strategies, resulting in a successful or valid buy rate of 97.8%. CONCLUSIONS: Our study demonstrated that ILVs selling LCCs were not making adequate efforts to verify the age of their customers, at the point-of-sale or point-of-delivery, facilitating easy access by minors. Few ILVs utilized age verification strategies that prevented online sales of LCCs to underage teens.

Random drug testing in US public school districts.

We estimated the proportion of the nation's public school districts that have high school grades in which random drug testing is conducted. We collected data in spring 2005 from 1343 drug prevention coordinators in a nationally representative sample of school districts with schools that have high school grades; of these districts, 14% conducted random drug testing. Almost all districts randomly tested athletes, and 65% randomly tested other students engaged in extracurricular activities; 28% randomly tested all students, exceeding the current sanction of the US Supreme Court.

Comprehensiveness of substance use prevention programs in U.S. middle schools.

PURPOSE: To assess how current practice in middle school substance use prevention programs compares with seven recommended guidelines adapted from the Centers for Disease Control and Prevention guidelines for school-based tobacco use prevention programs. METHODS: Substance use prevention practice was analyzed using data from a 1999 mailed questionnaire of a nationally representative sample of 1496 public and private schools with middle school grades that reported having a substance use prevention program. Respondents answered questions about substance use prevention education and activities in the whole school and in their own classroom. Weighted prevalence estimates for the seven recommendations are presented, and multiple regression was used to analyze correlates of implementation of the recommendations. RESULTS: An estimated 64.2% of schools met four or more of the recommendations for school-based substance use prevention practice; 4.0% met all seven recommendations. Schools were most likely to report having and enforcing substance use prevention policies (84.3%) and least likely to report training teachers in substance use prevention (17.9%). More recommendations were implemented in schools that were public and had larger enrollments, greater perceived availability of resources, greater school board and parental support for substance use prevention, and had hired a school substance use prevention coordinator. CONCLUSIONS: The low prevalence of comprehensive substance use prevention programs in U.S. middle schools may limit the potential impact of school programs on the prevalence of youth substance use.

Factors associated with fidelity to substance use prevention curriculum guides in the nation's middle schools.

Teachers' fidelity of implementation of substance use prevention curricula is widely considered desirable and is linked empirically to effectiveness. The authors examine factors pertinent to teachers' fidelity to curricula guides, using data from a nationally representative sample of 1,905 lead substance use prevention teachers in the nation's public and private schools. Findings suggest that about one-fifth of teachers of substance use prevention curricula did not use a curriculum guide at all, whereas only 15% reported they followed one very closely. Positively associated with adherence were teachers' discretion in their coverage of prevention lessons, beliefs concerning the effectiveness of the most recent training they received and the curricula they taught, and level of support they received from their principals for substance use prevention. The authors conclude that some degree of curriculum adaptation is inevitable and suggest how adherence to curricula guides may be improved through teacher training.

The principles of effectiveness: early awareness and plans for implementation in a national sample of public schools and their districts.

The US Department of Education's Principles of Effectiveness require recipients of Safe and Drug-Free Schools and Community Act funds to: a) base drug and violence prevention programs on needs assessment data, b) develop measurable program goals and objectives, c) implement programs with research evidence of effectiveness, and d) periodically evaluate programs relative to their goals and objectives. This paper reports the extent of awareness of the Principles of Effectiveness and plans for their implementation among public school districts and schools in the United States in the year following their announcement. Results showed a greater percentage of public school districts than individual schools knew about the principles and planned for implementation, but baseline levels of awareness for both districts and schools were relatively low. Schools were more likely to know about the principles when their associated school district also knew. Results suggest a need for greater communication about the principles to school districts, and in turn, a need for greater communication between district and school-level substance use prevention staff.

The effects of No Child Left Behind on the prevalence of evidence-based drug prevention curricula in the nation's middle schools.

BACKGROUND: Concerns have been expressed that No Child Left Behind (NCLB) may be reducing the amount of classroom time devoted to subjects other than those for which students are tested. The purpose of this article is to explore whether NCLB legislation has affected the provision of evidence-based drug prevention curricula (EBC) in the nation's middle schools, a subject area that is not assessed by standardized tests. METHODS: Data were collected in spring 2005 and spring 2008 from a nationally representative sample of middle schools. Respondents completed a survey regarding their provision of EBC (2005 response rate: 78.1%). We also collected data on schools' adequate yearly progress (AYP) status as of 2005 as a measure of their compliance with NCLB targets. We restricted our sample to schools that responded to our survey in both waves (n = 1324, or 76.9% of those schools responding in 2005) and conducted logistic regression analyses to determine whether those schools not making AYP in 2005 were less likely to be using an EBC in 2008. RESULTS: Our results revealed no relationship between AYP status in 2005 and EBC use in 2008. Analyses of demographic characteristics showed that schools making AYP were more likely to be small and rural, and to serve majority White student populations whose families were characterized by lower levels of poverty. CONCLUSIONS: Our failure to find any relationship between AYP status and the provision of EBC suggests that concerns about the potential adverse effects of NCLB on drug use prevention have yet to be validated. Implications of our results are discussed.

A short-term, quasi-experimental evaluation of D.A.R.E.'s revised elementary school curriculum.

We present the short-term results of a quasi-experimental evaluation of the revised D.A.R.E. (Drug Abuse Resistance Education) curriculum. Study outcomes examined were D.A.R.E.'s effects on three substances, namely students' lifetime and 30-day use of tobacco, alcohol, and marijuana, as well as their school attendance and academic performance. The study comprised students in 17 urban schools, each of which served as its own control; 5th graders in the 2006-2007 school year constituted the comparison group (n = 1490), and those enrolled as 5th graders in the 2007-2008 school year constituted the intervention group (n= 1450). We found no intervention effect on students' substance use for any of the substance use outcomes assessed. We did find that students were more likely to attend school on days they received D.A.R.E. lessons and that students in the intervention group were more likely to have been suspended. Study findings provide little support for the implementation and dissemination of the revised D.A.R.E. curriculum.

The prevalence of evidence-based substance use prevention curricula in the nation's elementary schools.

Current guidelines for school-based substance use prevention suggest that prevention efforts should begin in elementary grades, before students begin using substances. Previous research suggests, however, that the use of evidence-based curricula in these grades may be low. Using a 2005 survey of public school districts in the United States that include elementary grades (n = 1563), we assessed the prevalence of elementary curricula use, particularly those designated as evidence-based. We found that although 72% of districts administer a substance use prevention curriculum to their elementary students, only about 35% are using one that is evidence-based and only about 14% are using an evidence-based curriculum more so than any other prevention curriculum. We present prevalence estimates for specific evidence-based curricula and conclude by discussing possible reasons for and implications of our findings.

Implementing evidence-based substance use prevention curricula with fidelity: the role of teacher training.

It is widely recognized that teacher training affects the fidelity with which evidence-based substance use prevention curricula are implemented. We present the results of a 2005 survey of teachers from a nationally representative sample of 1721 public middle schools in the U.S. (78.1% response rate). We measured fidelity along two dimensions (adherence and dose) and also assessed the number of hours, recency, and perceived effectiveness of teachers' training, as well as the degree to which adherence was emphasized during training. Among teachers using evidence-based curricula, 35.3% reported following their curriculum guide very closely. The average proportion of lessons taught was 64.9%, and only 30.2% of teachers taught all the lessons in their curriculum. Analyses revealed that teachers whose training emphasized adherence were 5 times as likely to be more adherent. We present recommendations for training-related factors that may increase fidelity of implementation.

The prevalence of evidence-based drug use prevention curricula in U.S. middle schools in 2005.

Since the promulgation of its Principles of Effectiveness in 1998, the Office of Safe and Drug-Free Schools of the U.S. Department of Education has promoted the use of evidence-based drug prevention programs in the nation's schools. We report the results of a survey, conducted in 2005, of a nationally representative sample of 1,721 schools with middle school grades. Respondents comprised the staff member in the school identified as most knowledgeable about the school's drug prevention programs. The total response rate was 78%. Respondents answered questions concerning which drug use prevention curricula they used, and, if they used more than one, which one they used the most frequently. Three federally-sponsored registries were used to specify which curricula were considered evidence-based. Findings from 2005 were then compared to earlier estimates based on a similar 1999 survey. We found that 42.6% of the nation's schools with middle school grades were using an evidence-based curriculum, an increase of 8% from our 1999 estimate. The two most prevalent curricula in use, at 19% each, were Life Skills Training and Project ALERT. We note, however, that only 8% of Life Skills Training users and 9% of Project ALERT users reported using those curricula the most, and that only 23% of respondents overall reported that they used an evidence-based curriculum the most. More information is needed as to why over three-quarters of the nation's schools with middle school grades continue to administer curricula that have not been identified as effective.

Responses to positive results from suspicionless random drug tests in US public school districts.

BACKGROUND: Little is known about the context in which school-based suspicionless random drug testing (SRDT) occurs. The primary purpose of the current study was to describe school districts' responses to students' first positive result in districts with SRDT programs. METHODS: Data were collected in spring 2005 from 1612 drug prevention coordinators in a nationally representative sample of 1922 school districts (83.9% response rate), of which 205 districts reported SRDT in high school grades. RESULTS: Respondents reported an array of consequences for students with an initial positive SRDT, including requiring parents or guardians to meet with school officials (88.4%) and requiring students to participate in an education, counseling, or treatment program (60.8%). However, some districts also reported consequences contraindicated by federal advisory guides, such as notifying law enforcement officials (45.1%) and suspending the student from an athletic team (65.0%) or from school (31.0%). Some respondents may have conflated their districts' responses to for cause and random tests. Districts generally had available key services for students testing positive, including professional counseling for substance use problems (87.3%) and referrals to counseling services (91.9%). CONCLUSIONS: More understanding is needed of schools' responses to students who test positive following the administration of SRDT, available advisory guides concerning best practices should be more effectively disseminated, and appropriate training and technical assistance should be available to schools with SRDTs.

The prevalence of effective substance use prevention curricula in the nation's high schools.

Despite a substantial proportion of high school students who initiate substance use following middle school, the implementation of universal evidence-based prevention curricula appears to be scant. We report data collected in 2005 from 1392 school district-based drug prevention coordinators, from a national, representative study of school-based substance use prevention practices. Altogether, 10.3% of districts that included high school grades reported administering one of six such curricula that were then rated as effective by the Substance Abuse and Mental Health Services Administration's National Registry of Effective Programs and Practices or Blueprints for Violence Prevention, and 5.7% reported that they used one of these curricula the most. Only 56.5% of the nation's districts with high school grades administered any substance use prevention programming in at least one of their constituent high schools. Editors' Strategic Implications: The authors provide a powerful reminder that evidence-based prevention is not common in American high schools, in spite of federal mandates and the increasing availability of strong prevention programs. This should challenge researchers and government officials to improve dissemination and school officials to utilize evaluated programs.

Factors associated with adoption of evidence-based substance use prevention curricula in US school districts.

This paper examines factors associated with the adoption of evidence-based substance use prevention curricula (EBC) in a national sample of school districts. Substance abuse prevention coordinators in public school districts (n = 1593), which were affiliated with a random sample of schools that served students in Grades 5-8, completed a written survey in 1999. Results indicated that 47.5% of districts used at least one EBC in their schools with middle school grades. Substance use prevention coordinators reported they had the greatest input in decisions about curricula. In a multivariate analysis of factors positively associated with district-level decisions to adopt evidence-based programs, significant factors included input from a state substance use prevention group, use of information disseminated by the National Institute on Drug Abuse or Center for Substance Abuse Prevention, use of local needs assessment data, consideration of research showing which curricula are effective and allocation of a greater proportion of the coordinator's time to substance use prevention activities. State and federal agencies should increase their efforts to disseminate information about evidence-based programs, targeting in particular the district substance use prevention coordinator.

Reasons for teachers' adaptation of substance use prevention curricula in schools with non-white student populations.

In this study we investigate reasons why teachers adapt substance use prevention curricula in the nation's middle schools. We hypothesize that these reasons will be most salient in schools with racially and ethnically diverse student populations, for whom teachers may believe it appropriate to tailor their curricula. The study sample comprised a nationally representative sample of lead substance use prevention teachers in the nation's middle schools. Respondents answered questions concerning eight student problems or needs that constituted reasons why they might adapt their prevention lessons. Controlling for a variety of school and teacher characteristics, we found that teachers in high minority schools were more likely to adapt curricula in response to three of the eight characteristics presented: youth violence, limited English proficiency, and various racial/ethnic or cultural groups. We suggest that curriculum developers make a systematic effort to understand how teachers are adapting their curricula in high minority schools and incorporate these modifications, if found effective, into their curricula.

Students' special needs and problems as reasons for the adaptation of substance abuse prevention curricula in the nation's middle schools.

In this study we estimate the proportion of the nation's middle school teachers who have adapted substance abuse curricula in response to their students' special problems or needs. We also explore a variety of characteristics associated with schools, teachers, and the curricula implemented that are associated with adaptations made in response to the most prevalent of these student problems or needs. Study data were collected in 1999 from a representative sample of lead substance abuse prevention teachers in the nation's public and private schools. We found that 79.8% of respondents report adapting their prevention curricula in response to at least one of the dozen student problems and needs specified. The problems cited most frequently, by slightly more than half of all respondents, relate to the needs of students who are sexually active or have discipline problems. Associated most strongly with adaptations for these two reasons were teachers who were recently trained in their curricula, and substance abuse prevention lessons that could readily be integrated into the school's overall curriculum. We discuss the need for curriculum developers to recognize the frequency with which, and reasons for which, teachers are adapting their curricula, and to include appropriate optional content that addresses students' needs.

Who's calling the shots? Decision-makers and the adoption of effective school-based substance use prevention curricula.

This study investigates the relative roles of school district and school-level decision-makers in the implementation of effective substance use prevention curricula. Drawing on a "Site-Based Management" approach to effective decision-making, we hypothesized that schools whose personnel played active decision-making roles would be more likely to implement effective curricula than those in which decision-making was the prerogative of school district personnel. Study data comprised 1369 questionnaires completed by a representative national sample of both district-level prevention coordinators and middle school-based lead prevention teachers. From the perspective of the lead prevention teachers, the school district-level prevention coordinator was more influential than school staff in selecting effective prevention curricula. However, we did find some support for our hypothesis from our district-level informants, who indicated that community groups and advisory committees also play a modest role in the selection of such curricula.

The prevalence of effective substance use prevention curricula in U.S. middle schools.

Despite an abundance of evaluative evidence concerning the effectiveness of several school-based substance use prevention curricula, many of the nation's middle schools continue to implement curricula that are either untested or ineffective. This study reports the prevalence of substance use prevention curricula in the nation's public and private schools that contain middle school grades. We also report school- and respondent-related background characteristics differentiating schools using at least 1 effective curriculum from those using ineffective or untested curricula. Respondents comprised the lead staff who taught substance use prevention in a representative sample of 1,905 of the nation's public and private schools that include middle school grades. Data were collected in 1999 by means of a self-administered survey. Altogether, 26.8% of all schools, including 34.6% of public schools and 12.6% of private schools, used at least 1 of the 10 effective curricula specified. Few school or respondent characteristics were related to program implementation. Over two thirds of schools reported using more than 1 curriculum, and almost half reported using 3 or more. Results demonstrate the considerable gap between our understanding of effective curricula and current school practice. Prevention researchers and practitioners should work closely together to find ways to increase the proportion of schools implementing effective curricula.

A comparison of current practice in school-based substance use prevention programs with meta-analysis findings.

The series of seminal meta-analytic studies of school-based substance use prevention program studies conducted by the late Nancy S. Tobler and colleagues concluded that programs with content focused on social influences' knowledge, drug refusal skills, and generic competency skills and that use participatory or interactive teaching strategies were more effective than programs focused on knowledge and attitudes and favoring traditional didactic instruction. The present study compared current school practice against evidence-based standards for "effective content" and "effective delivery," derived from the Tobler findings. Respondents were the lead staff who taught substance use prevention in the 1998-1999 school year in a national sample of public and private schools that included middle school grades (N = 1,795). Results indicate that most providers (62.25%) taught effective content, but few used effective delivery (17.44%), and fewer still used both effective content and delivery (14.23%). Those who taught an evidence-based program (e.g., Life Skills Training, Project ALERT), however, were more likely to implement both effective content and delivery, as were those teachers who were recently trained in substance use prevention and were comfortable using interactive teaching methods. The findings indicate that the transfer to practice of research knowledge about school-based substance use prevention programming has been limited.