Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Huber, M. J., Workman, J., Ford, J. A., Moore, D., & Mayer, T. Evaluating the Prevention through Alternative Learning Styles program. Journal of Drug Education, 39(3), 239-259.
Outcomes
| Outcome 1: Intentions to use ATOD |
|
Description of Measures
|
Intentions to use ATOD were assessed using the PALS Questionnaire, a self-report survey that was modeled after the National Household Survey on Drug Abuse (NHSDA). Survey items were read out loud to students, who completed the survey before and after the intervention. The pretest survey contains 24 items (e.g., "After I turn 21, I will probably drink alcohol," "If friends offer to share alcohol with me, I will drink with them," "In high school, do you think you will use marijuana?"), and the posttest survey contains 26 items (the 24 pretest items and 2 additional items). Response options include multiple-choice, true-false, and yes-no formats.
|
|
Key Findings
|
In year 1 of the study, 6th- through 8th-grade classrooms in six schools were randomized to the intervention group, which received PALS, or the control group, which received the school's traditional prevention program (i.e., information on decisionmaking, illegal drugs, environmental health, keyboarding, and food). From pre- to posttest, the number of students in the intervention group responding "yes" or "maybe" to questions regarding their intentions to use alcohol, tobacco, or other drugs decreased, and the number of students responding "no" increased (p = .001 for all three substances); data from students in the control group were not analyzed.
In year 2 of the study, all 6th- through 8th-grade students from five of the six original schools were offered PALS. From pre- to posttest, the number of students responding "yes" or "maybe" to questions regarding their intentions to use ATOD decreased, and the number of students responding "no" increased (p = .001 for alcohol and tobacco; p = .002 for other drugs).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.4
(0.0-4.0 scale)
|
| Outcome 2: Knowledge of ATOD |
|
Description of Measures
|
Knowledge of ATOD was assessed using the PALS Questionnaire, a self-report survey that was modeled after the NHSDA. Survey items were read out loud to students, who completed the survey before and after the intervention. The pretest survey contains 24 items (e.g., "One or two drinks of alcohol can make it hard to make good decisions," "If someone really wants to, it's pretty easy to quit smoking," "Can people become addicted to marijuana?"), and the posttest survey contains 26 items (the 24 pretest items and 2 additional items). Response options include multiple-choice, true-false, and yes-no formats.
|
|
Key Findings
|
In year 1 of the study, 6th- through 8th-grade classrooms in six schools were randomized to the intervention group, which received PALS, or the control group, which received the school's traditional prevention program (i.e., information on decisionmaking, illegal drugs, environmental health, keyboarding, and food). At baseline, there were no significant differences between groups in regard to knowledge of ATOD. At posttest, compared with students in the control group, those in the intervention group had greater knowledge of alcohol (p < .001), tobacco (p < .001), and other drugs (p = .003).
In year 2 of the study, all 6th- through 8th-grade students from five of the six original schools were offered PALS. From pre- to posttest, these students had an increase in their knowledge of alcohol (p < .001), tobacco (p < .001), and other drugs (p < .002).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.4
(0.0-4.0 scale)
|
| Outcome 3: Knowledge of peer pressure and healthy choices |
|
Description of Measures
|
Knowledge of peer pressure and healthy choices was assessed using the PALS Questionnaire, a self-report survey that was modeled after the NHSDA. Survey items were read out loud to students, who completed the survey before and after the intervention. The pretest survey contains 24 items (e.g., "Advertisers on TV must tell everything about their products," "Some alcohol and tobacco advertisements are designed for children," "Which of the following is a healthy lifestyle choice?"), and the posttest survey contains 26 items (the 24 pretest items and 2 additional items). Response options include multiple-choice, true-false, and yes-no formats.
|
|
Key Findings
|
In year 1 of the study, 6th- through 8th-grade classrooms in six schools were randomized to the intervention group, which received PALS, or the control group, which received the school's traditional prevention program (i.e., information on decisionmaking, illegal drugs, environmental health, keyboarding, and food). At baseline, there were no significant differences between groups in regard to knowledge of peer pressure and healthy choices. At posttest, compared with students in the control group, those in the intervention group had greater knowledge of peer pressure and healthy choices (p < .002).
In year 2 of the study, all 6th- through 8th-grade students from five of the six original schools were offered PALS. From pre- to posttest, these students had an increase in their knowledge of peer pressure and healthy choices (p < .001).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.4
(0.0-4.0 scale)
|
| Outcome 4: Knowledge of learning styles |
|
Description of Measures
|
Knowledge of learning styles was assessed using the PALS Questionnaire, a self-report survey that was modeled after the NHSDA. Survey items were read out loud to students, who completed the survey before and after the intervention. The pretest survey contains 24 items (e.g., "People with disabilities can have successful jobs, relationships, and a happy life," "There is more than one type of intelligence," "Some disabilities are hidden"), and the posttest survey contains 26 items (the 24 pretest items and 2 additional items). Response options include multiple-choice, true-false, and yes-no formats.
|
|
Key Findings
|
In year 1 of the study, 6th- through 8th-grade classrooms in six schools were randomized to the intervention group, which received PALS, or the control group, which received the school's traditional prevention program (i.e., information on decisionmaking, illegal drugs, environmental health, keyboarding, and food). At baseline, there were no significant differences between groups in regard to knowledge of learning styles. At posttest, compared with students in the control group, those in the intervention group had greater knowledge of learning styles (p < .001).
In year 2 of the study, all 6th- through 8th-grade students from five of the six original schools were offered PALS. From pre- to posttest, these students had an increase in their knowledge of learning styles (p < .002).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.4
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
13-17 (Adolescent)
|
52% Male
48% Female
|
65% White
17% Black or African American
12% Race/ethnicity unspecified
3% Asian
2% Hispanic or Latino
1% American Indian or Alaska Native
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Intentions to use ATOD
|
2.3
|
2.3
|
2.3
|
3.3
|
2.0
|
2.5
|
2.4
|
|
2: Knowledge of ATOD
|
2.3
|
2.3
|
2.3
|
3.3
|
2.0
|
2.5
|
2.4
|
|
3: Knowledge of peer pressure and healthy choices
|
2.3
|
2.3
|
2.3
|
3.3
|
2.0
|
2.5
|
2.4
|
|
4: Knowledge of learning styles
|
2.3
|
2.3
|
2.3
|
3.3
|
2.0
|
2.5
|
2.4
|
Study Strengths The PALS Questionnaire has face validity. Teachers received initial and booster trainings to learn procedures for delivering PALS, followed a curriculum, and kept daily lesson logs to track the completion of each unit. Research staff visited classrooms and performed in-class monitoring to determine whether the materials were being presented as intended. Missing data were negligible for both years of the study. A quasi-experimental design was used during the first year of the study. The sample sizes for both years of the study were large, providing good statistical power.
Study Weaknesses Despite the use of correlational analysis, evidence of the evaluation instrument's reliability is limited. Neither criterion validity nor construct validity was documented. Although there were strong efforts to establish and maintain intervention fidelity, there was a lack of a psychometrically sound fidelity instrument to monitor the quality of intervention delivery. The study did not adequately address threats to internal validity, including the possibility of selection bias and contamination. In year 2 of the study, no control group was used for comparison with the intervention group. For analyses of data from year 1 of the study, which compared intervention and control groups, the use of t-tests alone is limiting.
|
|