Evaluating Program Implementation

The Intervention Tracking System (ITS)
AMOD is a demonstration program where the participating campus-community coalitions are free to pursue interventions that they believe will be most suitable and effective in their communities. In order to facilitate meaningful, cross-site comparisons, we developed a standard method of evaluation for determining the nature of interventions as they are implemented and identifying the barriers and facilitators encountered during intervention development and implementation.

We track interventions as they occur using a system that describes the intervention type, goal, strategy, timing, reach, and cost. This system provides information about program balance and comprehensiveness, and may help to facilitate the replication of successful programs.

Key research questions include:

• Do coalitions build comprehensive prevention programs?
• What is the nature of the coalition's preventive activities?
• What are the barriers and facilitators to program development and implementation?

Elements of the ITS
AMOD staff evaluators at each site collect intervention tracking data during monthly meetings with the AMOD project director and coalition members. Each intervention is characterized using the public health model of Agent-Host-Environment as described by Torjman.

They also record the following:

• characteristics of barriers encountered during the development and implementation of each intervention
• dollar costs of each intervention
• labor commitments to each intervention
• post-intervention media-mentions

This design facilitates a systematic accounting of what happens at each site without requiring coalitions to conform to a set of fixed interventions. Moreover, because comparable descriptors are used for all sites, we can examine patterns in overall program development over time and across sites.