Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Mason, W. A., Kosterman, R., Haggerty, K. P., Hawkins, J. D., Redmond, C., Spoth, R. L., et al. Gender moderation and social developmental mediation of the effect of a family-focused substance use preventive intervention on young adult alcohol abuse. Addictive Behaviors, 34(6-7), 599-605. Study 2Spoth, R., Trudeau, L., Guyll, M., Shin, C., & Redmond, C. Universal intervention effects on substance use among young adults mediated by delayed adolescent substance initiation. Journal of Consulting and Clinical Psychology, 77(4), 620-632.
Outcomes
| Outcome 1: Alcohol abuse disorder |
|
Description of Measures
|
Alcohol abuse disorder was assessed using the short form of the Diagnostic Interview Schedule (DIS). The DIS is a structured interview tool with a section assessing alcohol abuse as well as sections on drug use and other psychiatric conditions. Sample questions about alcohol include, "Has there ever been a period in your life when you often had more to drink than you intended to?" and "Have there been times in your life when you have often been under the influence of alcohol in situations where you could get hurt, for example, when riding a bicycle, driving, operating a machine, or anything else?" Responses were analyzed by a computer algorithm to determine whether one or more of the following DSM-IV criteria for alcohol abuse disorder was met in the past 12 months: clinically significant impairment in the form of either failure to fulfill major role obligations due to drinking, drinking in physically hazardous situations, recurrent alcohol-related legal problems, or persistent drinking despite adverse consequences.
|
|
Key Findings
|
Schools were randomly assigned to the intervention group, which implemented GGC with the parents of 6th-grade students, or to a minimal-contact control group, which provided the parents of 6th-grade students with factsheets about adolescent development. Alcohol abuse disorder was assessed when the students reached age 22. At this assessment, the proportion of women meeting the criteria for alcohol abuse disorder was smaller in the intervention group than in the control group (6% vs. 16%; p = .04). There was not a significant difference between the intervention and control group for men.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 2: Drunkenness frequency |
|
Description of Measures
|
Drunkenness frequency was assessed using a single question: "How often do you usually get drunk?" Responses were given on a 6-point scale ranging from 0 (never) to 5 (about every day).
|
|
Key Findings
|
Schools were randomly assigned to the intervention group, which implemented GGC with the parents of 6th-grade students, or to a minimal-contact control group, which provided the parents of 6th-grade students with leaflets about adolescent development. Drunkenness frequency was assessed at several intervals between the 6th and 12th grades and again at age 21. Compared with control group students, students in the intervention group were significantly less likely to have progressed to reporting drunkenness more than once per month by age 21 (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 3: Alcohol-related problems |
|
Description of Measures
|
Alcohol-related problems were assessed using a short, modified form of the Rutgers Alcohol Problems Index (RAPI). For each of 8 items describing a specific alcohol-related problem (e.g., "You had trouble remembering what you had done when you were drinking," "You got picked up by the police because of your drinking"), respondents indicated the frequency of the problem in the past 12 months using a scale ranging from 0 (never) to 4 (four or more times).
|
|
Key Findings
|
Schools were randomly assigned to the intervention group, which implemented GGC with the parents of 6th-grade students, or to a minimal-contact control group, which provided the parents of 6th-grade students with leaflets about adolescent development. Alcohol-related problems were assessed at several intervals between the 6th and 12th grades and again at age 21. Compared with control group students, students in the intervention group were significantly less likely to have progressed to reporting more than one alcohol-related problem (at any level of frequency) by age 21 (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 4: Illicit drug use frequency |
|
Description of Measures
|
Illicit drug use frequency was assessed using 9 open-ended questions phrased as follows: "How many times in the past 12 months did you use [specific substance]?" The substances included in the questions were marijuana; cocaine; ecstasy; methamphetamine; LSD; and amphetamines, barbiturates, narcotics, and tranquilizers (in each case, not by prescription). Individual items were adjusted to take into account appropriate weighting and were summed to create a single score.
|
|
Key Findings
|
Schools were randomly assigned to the intervention group, which implemented GGC with the parents of 6th-grade students, or to a minimal-contact control group, which provided the parents of 6th-grade students with leaflets about adolescent development. Illicit drug use frequency was assessed at several intervals between the 6th and 12th grades and again at age 21. Compared with control group students, students in the intervention group were significantly less likely to have progressed to reporting any illicit drug use by age 21 (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 5: Substance use |
|
Description of Measures
|
Substance use was assessed using an index created by combining measures of drunkenness frequency, illicit drug use frequency, and cigarette use frequency:
- Drunkenness frequency was assessed using a single question: "How often do you usually get drunk?" Responses were given on a 6-point scale ranging from 0 (never) to 5 (about every day). Responses were dichotomized, with any drunkenness coded 1 and no drunkenness coded 0.
- Illicit drug use frequency was assessed using 9 open-ended questions phrased as follows: "How many times in the past 12 months did you use [specific substance]?" The substances included in the questions were marijuana; cocaine; ecstasy; methamphetamine; LSD; and amphetamines, barbiturates, narcotics, and tranquilizers (in each case, not by prescription). Responses were dichotomized, with any illicit drug use coded 1 and no illicit drug use coded 0.
- Cigarette use frequency was assessed using a single question: "During the past 12 months how often did you smoke cigarettes?" Responses were given on a 7-point scale ranging from 1 (not at all) to 7 (about 2 packs/day). Responses were dichotomized, with any cigarette use coded 1 and no cigarette use coded 0.
The dichotomous items were summed for a total score ranging from 0 (indicating no occurrence of any of the three substance use behaviors) to 3 (indicating at least some occurrence of all three substance use behaviors).
|
|
Key Findings
|
Schools were randomly assigned to the intervention group, which implemented GGC with the parents of 6th-grade students, or to a minimal-contact control group, which provided the parents of 6th-grade students with leaflets about adolescent development. Substance use was assessed at several intervals between the 6th and 12th grades and again at age 21. Compared with control group students, students in the intervention group were significantly less likely to have progressed to reporting any substance use by age 21 (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.4
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
|
52.1% Female
47.9% Male
|
95% White
5% Race/ethnicity unspecified
|
|
Study 2
|
6-12 (Childhood)
|
50.9% Female
49.1% Male
|
98.6% White
1.4% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Alcohol abuse disorder
|
4.0
|
4.0
|
3.0
|
3.5
|
3.0
|
3.8
|
3.5
|
|
2: Drunkenness frequency
|
1.8
|
2.3
|
3.0
|
3.0
|
3.0
|
4.0
|
2.8
|
|
3: Alcohol-related problems
|
4.0
|
4.0
|
3.0
|
3.0
|
3.0
|
4.0
|
3.5
|
|
4: Illicit drug use frequency
|
3.0
|
2.8
|
3.0
|
3.0
|
3.0
|
4.0
|
3.1
|
|
5: Substance use
|
0.5
|
1.0
|
3.0
|
3.0
|
3.0
|
4.0
|
2.4
|
Study Strengths Some of the measurement instruments used--specifically, the DIS and RAPI--have well-established and strong psychometric properties. The researchers provided a standardized training program to staff that delivered the intervention, and they tracked fidelity of implementation using systematic observations. Attrition was low in both studies, especially given the long duration of follow-up. No differential attrition across experimental conditions was found for demographics, psychosocial characteristics, or risk for alcohol abuse. The analytic methods employed were highly sophisticated and well executed and were helpful in trying to account for confounds.
Study Weaknesses The reliability and validity of some of the measures is a concern. The drunkenness frequency measure is a single item ("How often do you usually get drunk?") of unknown origin phrased in a way that is subject to variations in interpretation by respondents. The measure of substance use is an additive scale that lacks its own psychometric properties and is composed of some other study measures that themselves lack psychometric properties. Due to the lengthy period between the intervention and final follow-up, potential confounding variables could have been introduced.
|
|
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Kosterman, R., Hawkins, J. D., Haggerty, K. P., Spoth, R., & Redmond, C. Preparing for the Drug Free Years: Session-specific effects of a universal parent-training intervention with rural families. Journal of Drug Education, 31(1), 47-68.
Kosterman, R., Hawkins, J. D., Spoth, R., Haggerty, K. P., & Zhu, K. Effects of a preventive parent-training intervention on observed family interactions: Proximal outcomes from Preparing for the Drug Free Years. Journal of Community Psychology, 25(4), 337-352.
Spoth, R., Redmond, C., Haggerty, K., & Ward, T. A controlled parenting skills outcome study examining individual difference and attendance effects. Journal of Marriage and the Family, 57(2), 449-464. Study 2Mason, W. A., Kosterman, R., Hawkins, J. D., Haggerty, K. P., & Spoth, R. L. Reducing adolescents' growth in substance use and delinquency: Randomized trial effects of a preventive parent-training intervention. Prevention Science, 4(3), 203-212.
Mason, W. A., Kosterman, R., Hawkins, J. D., Haggerty, K. P., Spoth, R. L., & Redmond, C. Influence of a family-focused substance use preventive intervention on growth in adolescent depressive symptoms. Journal of Research on Adolescence, 17(3), 541-564.
Park, J., Kosterman, R., Hawkins, J. D., Haggerty, K. P., Duncan, T. E., Duncan, S. C., et al. Effects of the "Preparing for the Drug Free Years" curriculum on growth in alcohol use and risk for alcohol use in early adolescence. Prevention Science, 1(3), 125-138.
Redmond, C., Spoth, R., Shin, C., & Lepper, H. S. Modeling long-term parent outcomes of two universal family-focused preventive interventions: One-year follow-up results. Journal of Consulting and Clinical Psychology, 67(6), 975-984.
Spoth, R., Redmond, C., & Shin, C. Direct and indirect latent-variable parenting outcomes of two universal family-focused preventive interventions: Extending a public health-oriented research base. Journal of Consulting and Clinical Psychology, 66(2), 385-399.
Spoth, R. L., Redmond, C., & Shin, C. Randomized trial of brief family interventions for general populations: Adolescent substance use outcomes 4 years following baseline. Journal of Consulting and Clinical Psychology, 69(4), 627-642.
Spoth, R., Redmond, C., Shin, C., & Azevedo, K.. Brief family intervention effects on adolescent substance initiation: School-level curvilinear growth curve analyses 6 years following baseline. Journal of Consulting and Clinical Psychology, 72(3), 535-542.
Spoth, R., Reyes, M. L., Redmond, C., & Shin, C. Assessing a public health approach to delay onset and progression of adolescent substance use: Latent transition and log-linear analyses of longitudinal family preventive intervention outcomes. Journal of Consulting and Clinical Psychology, 67(5), 619-630.
Supplementary Materials Aos, S., Lieb, R., Mayfield, J., Miller, M., & Pennucci, A.. Benefits and costs of prevention and early intervention programs for youth. Olympia, WA: Washington State Institute for Public Policy.
Outcomes
| Outcome 1: Substance use |
|
Description of Measures
|
Substance use was measured by youth self-reports of the frequency and quantity of use of alcohol, tobacco, marijuana, and other illicit drugs. Data were collected at pretest and 9, 21, 33, 51, and 75 months after the intervention.
|
|
Key Findings
|
Adolescents from families assigned to the intervention who reported they had not used substances 1 year after the intervention were more likely to remain nonusers 2 years later compared with adolescents from families not assigned to the intervention.
Adolescents from families assigned to the intervention who did report having used substances 1 year after the intervention were more likely to remain at the same level of use 1 year later compared with adolescents from families not assigned to the intervention (p < .05).
Through 4 years following the intervention, adolescents from families assigned to the intervention reported less increase in lifetime marijuana use and drunkenness and less growth in alcohol use compared with adolescent from families not assigned to the intervention (p < .05). Overall, substance use increased at a slower rate for the GGC group compared with the control group.
Adolescents from families assigned to the intervention also had a slower overall rate of increase in self-reported lifetime cigarette use and total tobacco use index through 6 years following the intervention (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
| Outcome 2: Parenting behaviors and family interactions |
|
Description of Measures
|
Parenting behaviors and family interactions (known risk and protective factors for adolescent substance use) were assessed using self-report measures and direct observation of family interactions in a general discussion task and a problem-solving task. Parental behaviors included intervention-specific skills and general child management skills. Intervention-specific skills included communicating clear rules about substance use, explaining consequences and rewarding compliance with substance use rules, helping the child learn how to express and control anger, and finding ways to keep the child involved in family activities and decisions. General child management skills included rewarding positive child behavior, child monitoring, and effective discipline.
|
|
Key Findings
|
Parents assigned to the intervention reported or demonstrated better intervention-specific and general child management skills compared with parents in the control group (p < .05). Outcomes were best for parents who attended the intervention classes regularly and reported higher readiness for parenting change.
Observations of family interactions indicated that mothers assigned to the intervention exhibited less negative interaction in the general discussion task and more proactive communication in both tasks compared with control group mothers (p < .05). Mothers assigned to the intervention also used a less interrogating style and less antagonistic behavior in interacting with their children compared with control group mothers (p < .03). Fathers assigned to the intervention exhibited more proactive communication and better relationship quality in the problem-solving task compared with control group fathers (p < .05).
On self-report measures, mothers assigned to the intervention were more likely than control group mothers to report that they reward their child for prosocial behavior, communicate rules about substance use, punish their child for misbehavior, restrict their child's alcohol use, expect their child to refuse a beer from a friend, express less conflict with their spouse, and work at being more involved with their child (p < .05). Fathers assigned to the intervention were more likely than control group fathers to report more communication with their child regarding rules on substance use and more involvement from their child (p < .05).
In a subsequent study, parents assigned to the intervention reported better intervention-specific parental behaviors compared with control group parents (e.g., communicating clear rules about substance use, explaining consequences and rewarding compliance with substance use rules, helping the child learn how to express and control anger, and finding ways to keep the child involved in family activities and decisions). The effect size for this finding was small (Cohen's d = 0.45).
Intervention parents also reported better general child management and parent-child affective quality (p < .05); this result was maintained 1 year after the intervention with a small effect size (Cohen's d = 0.29).
Parents assigned to the intervention also reported establishing stronger norms against alcohol use relative to control group parents 3.5 years after the intervention (p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.9
(0.0-4.0 scale)
|
| Outcome 3: Delinquency |
|
Description of Measures
|
Adolescents were asked to report their involvement in a range of non-drug-related delinquent activities in the past 12 months. The range of activities included items such as taking something worth $25 or more and purposely damaging public property. Data were collected at pretest and 9, 21, 33, and 51 months after pretest.
|
|
Key Findings
|
Adolescents from families assigned to the intervention had a slower rate of increase in self-reported activities associated with delinquency compared with adolescents from families not assigned to the intervention (p < .05). In addition, the frequency of participation in these activities served as a reliable predictor of substance use (p < .01).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
| Outcome 4: Symptoms of depression (adolescents) |
|
Description of Measures
|
Adolescents were asked to report feelings and behaviors associated with depression at the time of assessment or in the preceding 6 months. The measure included 8 items such as "I feel worthless or inferior," "I am unhappy, sad, or depressed," and "I think about killing myself." Data were collected at pretest and 9, 21, 33, and 51 months after pretest.
|
|
Key Findings
|
Adolescents from families assigned to the intervention had a slower rate of increase in self-reported depressive symptoms compared with adolescents from families not assigned to the intervention (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
13-17 (Adolescent)
26-55 (Adult)
|
Data not reported/available
|
100% White
|
|
Study 2
|
6-12 (Childhood)
13-17 (Adolescent)
26-55 (Adult)
|
Data not reported/available
|
100% White
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Substance use
|
2.8
|
2.5
|
2.5
|
2.5
|
2.5
|
3.0
|
2.6
|
|
2: Parenting behaviors and family interactions
|
3.0
|
3.0
|
3.0
|
3.0
|
2.5
|
3.0
|
2.9
|
|
3: Delinquency
|
2.8
|
2.5
|
2.5
|
2.5
|
2.5
|
3.0
|
2.6
|
|
4: Symptoms of depression (adolescents)
|
3.0
|
3.0
|
3.0
|
3.0
|
2.5
|
4.0
|
3.1
|
Study Strengths Measures of substance use are typical of those used in similar research. The authors provided a standardized training program to staff who delivered the intervention, tracked fidelity of implementation using videotapes and systematic observations, made efforts to address potential confounds, and statistically accounted for missing data.
Study Weaknesses In one study, 43% of the sample pool declined to participate, so it appears that the participants were highly motivated; it is unclear how this might have affected the results. Between 18% and 26% of the intervention curriculum was not covered in one study.
|
|