Intervention Summary

Project MAGIC (Making A Group and Individual Commitment)

Project MAGIC (Making A Group and Individual Commitment) is an alternative to juvenile detention for first-time offenders between the ages of 12 and 18. The program's goals include helping youths achieve academic success; modifying attitudes about alcohol, tobacco, and other drugs; and enhancing life skills development and internal locus of control. Project MAGIC is based on the ecological model, involving individual, family, school, and community domains. Over the 2-month course of the program, separate interventions are provided to the youths and their parents, who are trained to better monitor their children's behavior:

The youth component consists of a 20-session skills-building curriculum offered to groups of 8-10 youths. The curriculum addresses substance abuse education, anger management, conflict management, problem solving/decisionmaking, communication, personal responsibility, values, stress reduction, and community service, which the youths are required to participate in as part of the program. The sessions (1.5 hours each) are held three times per week over 2 months and are conducted by a trained facilitator.
Parents participate in four 2.5-hour parent education meetings, during which parents and their children read and respond to preprinted activities designed to increase communication and family management. The meetings also offer an opportunity for parents to discuss mutual concerns regarding parenting, enhance their awareness of substance abuse issues, improve communication skills, and develop skills for monitoring their children's behavior. In the studies reviewed for this summary, the intervention also included a take-home component for parents who could not attend all four meetings because of work schedules; parents could complete lesson plans at home at their own pace.

Facilitators collaborate with local agencies and other resources in the community to enhance a reciprocal investment by both the youth/parents and the community. The facilitators are usually members of the community where the program is going to be implemented and have at least a bachelor's degree as well as a background in teaching or experience working with at-risk youths.

Descriptive Information

Areas of Interest	Mental health promotion Substance use disorder prevention
Outcomes	1: Academic engagement and achievement 2: Attitudes toward substance use and perceived substance use by peers 3: Parental monitoring 4: Internal locus of control 5: Life skills development
Outcome Categories	Alcohol Drugs Education Family/relationships Mental health Social functioning Tobacco Physical aggression and violence-related behavior
Ages	6-12 (Childhood) 13-17 (Adolescent) 26-55 (Adult)
Genders	Male Female
Races/Ethnicities	American Indian or Alaska Native Asian Black or African American Hispanic or Latino White Race/ethnicity unspecified
Settings	School Other community settings
Geographic Locations	Urban Suburban Rural and/or frontier Tribal
Implementation History	Project MAGIC emanated from a needs assessment conducted by staff at the University of Nevada Cooperative Extension. The needs assessment included all adjudicated youths in Nevada's State-run juvenile detention facilities. Following a Federal grant award to fund program development and a successful pilot in Elko, Nevada, the program was expanded to two neighboring rural communities, and the curriculum was published. State funding enabled the program to continue in the three rural communities and expand to six urban and Indian reservation sites. A statewide detention center in Elko adopted Project MAGIC. Evaluations at a number of sites in Nevada are ongoing. More than 3,000 youths have graduated from the program to date.
NIH Funding/CER Studies	Partially/fully funded by National Institutes of Health: No Evaluated in comparative effectiveness research studies: No
Adaptations	Spanish-language materials have been developed for the program. Some implementers located on Indian reservations have modified the curriculum to meet the cultural needs of tribal youths and parents.
Adverse Effects	No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories	Selective Indicated

Quality of Research

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Study 1

Evans, W., & Smith, M. Project MAGIC: Building communities of support for alternatives to juvenile detention: Cumulative data report. Unpublished manuscript.

Study 2

Smith, M., & Evans, W. Project "MAGIC": Building communities of support for alternatives to juvenile detention. In Bringing excellence to substance abuse services in rural and frontier America: Award for Excellence award-winning papers (pp. 15-27). Rockville, MD: Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Treatment.

Outcomes

Outcome 1: Academic engagement and achievement
Description of Measures	Academic engagement and achievement were assessed using the Nevada Bureau of Alcohol and Drug Abuse (BADA) Youth/Parent Questionnaire at pretest; a shorter version called the MAGIC Youth/Parent Questionnaire was used at posttest. The shorter version measures the same constructs as the full questionnaire but is designed to reduce respondent burden. Both instruments are self-report and include multiple-choice questions that measure knowledge about and attitudes toward substance abuse, parental monitoring, goals, school, and other key areas addressed by the Project MAGIC curriculum.
Key Findings	Survey responses of youths indicated that from pre- to posttest, Project MAGIC youth participants improved in aspects of academic engagement and achievement: Reduction in school absences (p < .01) Better grades (p < .001), with more youths reporting that they were now earning better grades than most of their classmates (p < .001) Greater recognition that the things the youths were learning in school are important for later life (p < .01) Greater participation in extracurricular activities (p < .01) and in community service or volunteer work (p < .001) Survey responses of parents showed pre- to posttest improvements among Project MAGIC youth participants in many of the same aspects: Reduction in school absences (p < .001) Better grades (p < .05) Greater recognition that the things the youths were learning in school are important for later life (p < .001) Greater recognition that schoolwork is meaningful and important (p < .01) Greater participation in community service or volunteer work (p < .001).
Studies Measuring Outcome	Study 1
Study Designs	Preexperimental
Quality of Research Rating	2.6 (0.0-4.0 scale)

Outcome 2: Attitudes toward substance use and perceived substance use by peers
Description of Measures	Attitudes toward substance use and perceived substance use by peers were assessed using the Nevada BADA Youth/Parent Questionnaire at pretest; a shorter version called the MAGIC Youth/Parent Questionnaire was used at posttest. The shorter version measures the same constructs as the full questionnaire but is designed to reduce respondent burden. Both instruments are self-report and include multiple-choice questions that measure knowledge about and attitudes toward substance use, parental monitoring, goals, school, and other key areas addressed by the Project MAGIC curriculum.
Key Findings	Survey responses of youths indicated that from pre- to posttest, Project MAGIC youth participants developed greater awareness that consuming one or two alcoholic drinks per day presents a risk of harm (p < .05). In addition, from pre- to posttest, youth participants reported having fewer peers who had tried alcohol (p < .001), smoked cigarettes (p < .001), used marijuana (p < .001), or used other illegal drugs (p < .01) in the past year. Survey responses of parents suggested that from pre- to posttest, there was an increase in Project MAGIC youth participants' agreement with each of the following statements: It is wrong (for themselves) to drink alcohol (p < .001). It is wrong (for themselves) to smoke cigarettes (p < .01). It is wrong for their peers to drink alcohol (p < .001).
Studies Measuring Outcome	Study 1
Study Designs	Preexperimental
Quality of Research Rating	2.6 (0.0-4.0 scale)

Outcome 3: Parental monitoring
Description of Measures	Parental monitoring was assessed using the Nevada BADA Youth/Parent Questionnaire at pretest; a shorter version called the MAGIC Youth/Parent Questionnaire was used at posttest. The shorter version measures the same constructs as the full questionnaire but is designed to reduce respondent burden. Both instruments are self-report and include multiple-choice questions that measure knowledge about and attitudes toward substance abuse, parental monitoring, goals, school, and other key areas addressed by the Project MAGIC curriculum.
Key Findings	Survey responses of parents indicated that from pre- to posttest, parents gained greater awareness of where their children are when not at home (p < .05) and greater awareness of their children's friends/associates (p < .05).
Studies Measuring Outcome	Study 1
Study Designs	Preexperimental
Quality of Research Rating	2.6 (0.0-4.0 scale)

Outcome 4: Internal locus of control
Description of Measures	Internal locus of control was assessed using the Nowicki-Strickland Locus of Control Scale, a 40-item questionnaire that measures the perception of connection between actions and outcomes. Lower scores indicate higher levels of internal control; for example, a juvenile with a lower score (representing good internal control) would be more likely to report that luck has little to do with getting a good grade in school.
Key Findings	From pre- to posttest, mean internal locus of control scores for Project MAGIC youth participants improved 14% (from 15.74 to 14.19; p < .01).
Studies Measuring Outcome	Study 2
Study Designs	Preexperimental
Quality of Research Rating	2.6 (0.0-4.0 scale)

Outcome 5: Life skills development
Description of Measures	Over the course of 20 program sessions, youth participants completed tasks in the Portfolio Student Workbook which are designed to measure the development of life skills related to conflict resolution, communication skills, cooperation behaviors, aggression, and school performance and involvement. The portfolio tasks were scored by trained facilitators on a continuum from 0 to 100, with 0-15 representing a beginning level of life skills development, 16-45 an emerging level, 46-75 a developing level, and 76-100 a maturing level. A perfect score of 100 would indicate mastery of all skills taught in the project.
Key Findings	From pre- to posttest, mean life skills scores for Project MAGIC youth participants increased 130% (from 21.87 to 51.15; p < .01), showing a progression from the emerging level to the developing level in the areas of life skills taught in the program.
Studies Measuring Outcome	Study 2
Study Designs	Preexperimental
Quality of Research Rating	2.2 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study	Age	Gender	Race/Ethnicity
Study 1	6-12 (Childhood) 13-17 (Adolescent) 26-55 (Adult)	73% Male 27% Female	88% White 12% Race/ethnicity unspecified
Study 2	6-12 (Childhood) 13-17 (Adolescent)	50% Female 50% Male	72% White 17% Hispanic or Latino 7% American Indian or Alaska Native 3% Black or African American 1% Asian

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

Reliability of measures
Validity of measures
Intervention fidelity
Missing data and attrition
Potential confounding variables
Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome	Reliability of Measures	Validity of Measures	Fidelity	Missing Data/Attrition	Confounding Variables	Data Analysis	Overall Rating
1: Academic engagement and achievement	3.0	2.5	3.0	3.5	1.5	2.0	2.6
2: Attitudes toward substance use and perceived substance use by peers	3.0	2.5	3.0	3.5	1.5	2.0	2.6
3: Parental monitoring	3.0	2.5	3.0	3.5	1.5	2.0	2.6
4: Internal locus of control	3.5	3.0	2.5	2.3	1.5	3.0	2.6
5: Life skills development	2.3	2.3	2.5	2.3	1.5	2.5	2.2

Study Strengths

The Nevada BADA Youth/Parent Questionnaire, MAGIC Youth/Parent Questionnaire, and Nowicki-Strickland Locus of Control Scale have acceptable validity and reliability levels. A number of fidelity measures were put into place; for example, checklists of program components were filled out by facilitators and reviewed by the project director for completion, attendance at youth sessions and parent meetings was tracked, and site visits were conducted. One of the two studies had very low attrition and minimal missing data.

Study Weaknesses

The Portfolio Student Workbook is not an established measure and lacks reliability and validity information. Both studies are vulnerable to confounds due to their pretest/posttest-only design. Despite the establishment of many fidelity measures, it is unclear how the parents' process was monitored, particularly because it was self-paced and done at home. One study had a high attrition rate for parents; only 37% of the parents completed the posttest evaluation.

Readiness for Dissemination

Materials Reviewed

The materials below were reviewed for Readiness for Dissemination. The implementation point of contact can provide information regarding implementation of the intervention and the availability of additional, updated, or new materials.

Smith, M., & Evans, W. Making A Group and Individual Commitment student workbook. Reno: University of Nevada Cooperative Extension.

University of Nevada Cooperative Extension. Making A Group and Individual Commitment: Give juvenile offenders a chance to change. Curriculum guide. Reno: Author.

Other program materials:

Job descriptions
MAGIC Annual Compliance Monitor Form
MAGIC Certification Report
MAGIC: Introduction & Start-Up Information
MAGIC Reporting Forms
Quality Assurance Plan--Project MAGIC
Research Involving Children
Research Involving Prisoners
Research Protocol
Resources for training and support

Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the intervention's Readiness for Dissemination using three criteria:

Availability of implementation materials
Availability of training and support resources
Availability of quality assurance procedures

For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.

Implementation Materials	Training and Support Resources	Quality Assurance Procedures	Overall Rating
3.5	2.0	3.5	3.0

Dissemination Strengths

Extremely comprehensive, well organized, and easy to understand, the curriculum guide details the educational content and processes for the youth sessions and parent meetings. All materials required for implementation are available electronically on CD-ROM. The curriculum guide comes with supplemental materials to augment facilitator knowledge, pretest/posttest surveys and protocols, and various process and outcome evaluation forms. The tools for measuring participant progress are accompanied by clear scoring guidance and examples to support consistent use across implementers.

Dissemination Weaknesses

Implementation planning guidance is vague and general; the materials do not address recommended group size or target audience and do not provide much helpful guidance about facilitator qualifications. No information accompanies the organizational Certification Report to explain how it is used, and it is not clear if the review elements it lists must be verified before the program can be started. Little information is available about training and support. Some quality assurance tools are available only by request or are not accompanied by complete guidance for intended use.

Costs

The cost information below was provided by the developer. Although this cost information may have been updated by the developer since the time of review, it may not reflect the current costs or availability of items (including newly developed or discontinued items). The implementation point of contact can provide current information and discuss implementation requirements.

Item Description	Cost	Required by Developer
Curriculum guide and student workbook (includes all quality assurance tools)	Shipping costs only for CD-ROM $150 for a set of hard-copy materials materials	Yes
Phone support from program staff	$50 per hour	No
3-day, off-site training in Elko, Nevada	$500 per participant	No
3-day, on-site training	$400 per participant, plus travel expenses for two trainers	No

Additional Information

The developer will negotiate a discounted rate for trainings with more than 10 participants.