Implementing Science Based Prevention

Navigation: Overview,   18 case studies - Summary, 1. Aberdeen School District, 2. City of Othello, 3. Crossroads Treatment Center, 4. ESD 123, 5. Grant County Prevention & Recovery, 6. Lake Washington School District, 7. North Thurston School District, 8. Oak Harbor School District, 9. Olympic ESD 114, 10. Orcas Island School District,  11. Pacific County Health & Human Services and Willapa Children's Service, 12. Seattle Public Schools, 13.Snoqualmie Valley Community Network, 14. Spokane County Community Services, 15. Swinomish Tribe, 16.Together!/ROOF/Rochester, 17. Toppenish Police Department/City of Toppenish, 18. Walla Walla County Dept of Human Services

 

The Experiences of Eighteen Communities and Progress Towards Inter-Agency Coordination to Reduce Alcohol and Substance Abuse Among Adolescents Evaluation Report for the Washington State Incentive Grant (July 1998 – July 2002)

Executive Summary

Washington State was awarded a nine million State Incentive Grant (SIG) for the dual purposes of promoting prevention system changes among state agencies and of implementing more evidence-based prevention programs. The ultimate goal was to increase coordination and efficiency and to improve outcomes of prevention activities so that fewer youth would abuse alcohol and drugs.

Washington State visualized these system changes as occurring within a framework of decentralized power, where better coordination would follow from a common purpose and from a decision-making framework based on better information. The system would still give local constituents choices of local prevention foci, resource coordination, and program selection, but within a common, outcome-based framework established among diverse state agencies. State agency objectives within this framework were to:

  • Agree on a common set of prevention goals and outcome measures: Collect and report better community data based on a common risk and protective factor framework, to help local prevention planners use a more data driven planning model. Furthermore, agree on statewide prevention goals and benchmarks that could partially guide local choices.
  • Develop common assessment tools of community needs and resources: Create common and collaborative requirements for local need and resource assessments and planning, to decrease duplication of effort at the local level and improve outcomes.
  • identify selection criteria and a common set of science-based prevention programs: Agree on selection criteria to identify science-based programs that can best address the needs identified from common assessment and measures, and facilitate their local implementation.
  • Develop a uniform reporting mechanism for participant outcomes: Discuss data compatibility across agencies for community wide reporting, and pilot the Everest Prevention Outcomes Evaluation Management System to assess program impacts among participants.
  • Establish guidelines for leveraging and redirecting resources: Encourage both specialized independent prevention efforts and resource coordination, leveraging and redirection among community partners.
  • Create a cross-agency system of professional development: Create a common training system for prevention PROFESSIONALs working for different state agencies.

Summary Findings

At the state level, most prevention partners participated in creating the plan for interagency coordination, considered it realistic and supported it. Progress has been made in building the common prevention plan and implementing the planned state objectives. Some major issues still remain unresolved, but state agencies and community constituents have common interests in resolving these issues in the long run.

Evaluation findings among selected Washington communities show that science-based prevention can be learned and implemented, outcomes can be measured, and there are significant possibilities for leveraging community funds. It thus becomes strategically important for state agencies interested in efficiencies, effective coordination and demonstrable outcomes to promote such prevention efforts.

State Achievements

Significant progress was made on five of the six state-level objectives. They were:

  • A common set of prevention goals and outcome measures: Agreements were reached on statewide benchmarks, on measures for the Washington State Survey of Adolescent Health Behavior, and on scales for monitoring outcomes for SIG program participants. A common health and substance use survey is being implemented in the Fall of 2002. Student participation in the student survey has doubled. Furthermore, archival indicators are being validated against student survey results.
  • Common assessment tools of community needs and resources: All
    Washington counties conducted an inter-agency, collaborative needs assessment in the spring of 2001.
  • The identification of science-based prevention programs: SIG communities have used the programs listed on the Internet by the Western Center for the Application of Prevention Technology (WestCAPT).
  • Uniform reporting mechanisms for participant outcomes: SIG communities have piloted the Everest Prevention Outcomes Evaluation Management System.
  • System of professional development: A training curriculum was developed and four training sessions were implemented in 2001 by WestCAPT with state agency funding; voluntary certification was available in June 2002.

The decision was made not to develop guidelines for leveraging and redirecting money and resources (one of the six state objectives), but to wait until the inter-agency group had experience in monitoring statewide benchmarks

Strategy development for system changes in prevention was largely successful by channeling input from many different constituents pushing for more coordination; placing prevention system changes as part of the Governor's agenda; avoiding inter-agency funding issues by focusing on common statewide goals; and setting up a formal inter-agency group to monitor statewide benchmarks.

State agency representatives gained inter-agency knowledge and strengthened working relationships across agency boundaries by attending SIG meetings.

State-Level Challenges

Top managers of state agencies, in follow-up interviews, identified the following issues as the most important ones to resolve for further movement toward the prevention system vision developed during this grant.

  • How to gain long-term funding to support common, inter-agency prevention databases available to communities, including scales to measure program participant outcomes and community-level outcomes.
  • How to create and fund a system of collaborative, inter-agency technical assistance and training.
  • How to deal with accountability of the inter-agency group that will monitor the new substance abuse prevention system, and resolve issues of communication between local and state agencies.
  • How to resolve the disagreements about whether the model of collaboration for the substance abuse prevention system should stand alone or whether it could be a model for other types of prevention.
  • How to conduct further research, and with what funds, to demonstrate community wide outcomes in SIG sites when student survey data become available in the Fall of 2002, 2004 and 2006. -- to provide evidence for links between particular system change strategies in SIG sites and better, long term, community wide outcomes.

Community Achievements

Eighteen communities in Washington were awarded SIG grants based on a competitive selection process. They agreed to strive to accomplish five objectives:

  • Coordinate prevention planning with community partners,
  • Use a common ‘risk and protection factor' framework for assessing community prevention needs and for targeting prevention activities,
  • Consider relevant data on risk and protective factors in their planning,
  • Implement prevention programs shown to be effective using evidence-based criteria (i.e. science-based programs), and
  • Collect outcomes among program participants in order to improve program effectiveness.

All eighteen SIG communities achieved these objectives, even though most had never implemented science-based programs, and 28 percent had not used the risk and protective framework before. The use of a stepwise logic model, outcome driven and data driven, was important in this achievement.

Almost all SIG communities were also able to coordinate, leverage and redirect other resources among local partners, so as to increase and institutionalize overall prevention efforts, which served more people in the communities.

All SIG communities participated in the collection of outcomes among program participants. Many used a new Internet based software developed in Washington for this purpose: the Everest Prevention Outcomes Evaluation Management System. The scales used to measure relevant outcomes were found to be reliable. In many cases, where programs were implemented with fidelity to the original program design, expected changes occurred among program participants.
Whether these accomplishments (made mainly in the years 1999-2002) have led to community-wide changes in the SIG sites awaits the analysis of longer-term, trend data in student surveys and archival indicators. Student survey and archival data are available for the initial period of 1998, 1999 and 2000. SIG communities have committed to collecting further survey data in their schools in the Fall of 2002, 2004 and 2006. It is hoped that further archival data will also be available at the necessary level of small community geographies

Community Challenges

The availability and use of correct information is central to a more science-based approach to prevention. Communities experienced challenges that often related to limitations with existing data and how to interpret and use data. SIG communities struggled and asked for help in the following areas.

  • More technical assistance is needed in the selection of science-based programs that would meet local needs and in training to overcome difficulties in their implementation. Communities also need more training in the use and interpretation of data useful for prevention planning and monitoring program effectiveness.
  • Better and more local information for needs assessment and community outcomes: Community level data on risk and protection profiles is needed at more local, sub-county levels. Communities also need more help in interpreting often-partial data. County-wide archival indicators, based on data collected from various state agencies have been available for many years. However, more work needs to be accomplished to validate them against student survey information and make them available at smaller geographies for prevention planning.
  • Better reporting on participant outcomes: More user-friendly, interpretable reports on the outcomes of program participants.
    Resolution of the remaining state challenges would fulfill important community prevention needs by providing:
  • Better information on which to base community prevention decisions and better data to demonstrate effectiveness of prevention efforts.
  • Common tools and training to facilitate collaborative planning and reduce duplication.
  • Clearer, less contradictory mandates from diverse state agencies, who are together accountable for better results.
  • Better integration of substance abuse prevention efforts with other local prevention priorities.
  • Higher chance of obtaining both private and state funding for prevention programs that have been shown by research to have better outcomes.

Download Community Report

The report "Implementing Science Based Prevention to Reduce Alcohol and Substance Abuse Among Adolescents: The Experiences of Eighteen Communities and Progress Towards Inter-Agency Coordination" can be downloaded in two parts: Part 1 (content, 892 KB, 4.43) and Part 2 (appendices, 850KB, 4.43A).

Other Information

Included information:

Related information: