Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Kupersmidt, J. B., Scull, T. M., & Benson, J. W. Improving media interpretation processing skills to promote healthy decision making about substance use: The effects of the middle school Media Ready curriculum. Manuscript submitted for publication.
Supplementary Materials Kupersmidt, J. B., & Barrett, T. Media Ready. Durham, NC: innovation Research and Training.
Scull, T. M., Kupersmidt, J. B., Parker, A. E., Elmore, K. C., & Benson, J. W. Adolescents' media-related cognitions and substance use in the context of parental and peer influences. Journal of Youth and Adolescence, 39(9), 981-998.
Outcomes
| Outcome 1: Intentions to use alcohol |
|
Description of Measures
|
Project staff administered a paper-and-pencil questionnaire to students at pre- and posttest. Four items assessed each student's intention to use alcohol:
- "Before you are 21 years old, do you think you will drink beer, wine, or hard liquor (more than just a few sips)?"
- "Before you are 21 years old, do you think you will get drunk or drink a lot of alcohol at one time?"
- "During the next year, do you think you will drink beer, wine, or hard liquor (more than just a few sips)?"
- "During the next year, do you think you will get drunk or drink a lot of alcohol at one time?"
Students responded using a 4-point Likert scale that ranged from 0 (I definitely will not) to 3 (I definitely will). Scores for intentions to use alcohol were derived from the average of responses across the 4 items, with higher scores indicating a stronger intention to use alcohol. Pretest scores for the outcome variable were included as covariate prediction variables; therefore, outcome variable means were reported as adjusted posttest scores.
|
|
Key Findings
|
At posttest, boys in the intervention group had lower mean scores for intentions to use alcohol compared with boys in the wait-list control group (0.48 vs. 0.65; p < .05). The scores for girls did not differ significantly between groups.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
| Outcome 2: Intentions to use tobacco |
|
Description of Measures
|
Project staff administered a paper-and-pencil questionnaire to students at pre- and posttest. Four items assessed each student's intention to use tobacco:
- "Before you are 18 years old, do you think you will smoke cigarettes?"
- "Before you are 18 years old, do you think you will chew tobacco or use snuff?"
- "During the next year, do you think you will smoke cigarettes?"
- "During the next year, do you think you will chew tobacco or use snuff?"
Students responded using a 4-point Likert scale that ranged from 0 (I definitely will not) to 3 (I definitely will). Scores for intentions to use tobacco were derived from the average of responses across the 4 items, with higher scores indicating a stronger intention to use tobacco. Pretest scores for the outcome variable were included as covariate prediction variables; therefore, outcome variable means were reported as adjusted posttest scores.
|
|
Key Findings
|
At posttest, students in the intervention group had lower mean scores for intentions to use tobacco compared with students in the wait-list control group (0.08 vs. 0.23; p < .0001). Among students who had previously used tobacco, those in the intervention group had lower mean scores for intentions to use tobacco compared with those in the wait-list control group (0.42 vs. 0.96; p < .001). The scores for students who had not previously used tobacco did not differ significantly between groups.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.5
(0.0-4.0 scale)
|
| Outcome 3: Media deconstruction skills for alcohol and tobacco |
|
Description of Measures
|
Project staff administered a paper-and-pencil questionnaire to students at pre- and posttest. The questionnaire asked students to deconstruct one alcohol and one tobacco print advertisement. Three open-ended items assessed each student's critical thinking in regard to each advertisement:
- "Tell me about this advertisement in the space below (the more detail the better)."
- "How are the advertisers trying to get someone to buy this product?"
- "What type of magazine would have this advertisement in it and why?"
The resulting qualitative data were coded by four trained coders, producing scores in five categories:
- The product score, which ranged from 0 to 3, captured a student's ability to recognize the product being advertised.
- The graphic elements score, which ranged from 0 to 2, assessed a student's understanding of how advertisers use graphic elements (e.g., font, color, placement of items such as warning labels) to capture attention or to make the product seem more appealing.
- The language score, which ranged from 0 to 3, evaluated a student's understanding of how slogans and advertising claims are used in advertisements.
- The target audience score, which ranged from 0 to 3, assessed a student's understanding of the term "target audience" and also his or her ability to recognize the target audience of a particular advertisement.
- The implied messages score, which ranged from 0 to 3, assessed a student's ability to recognize implied messages in advertisements.
The five scores were summed to create the deconstruction skills score, which had a possible range of 0 to 14, with higher scores indicating greater ability to apply critical thinking about advertisements. Each student's score was calculated as the average of the four coders' scores for that individual. Pretest scores for the outcome variable were included as covariate prediction variables; therefore, outcome variable means were reported as adjusted posttest scores.
|
|
Key Findings
|
At posttest, students in the intervention group had higher mean scores for deconstruction skills compared with students in the wait-list control group (10.78 vs. 9.00; p < .005).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.3
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
13-17 (Adolescent)
|
56.6% Female
43.4% Male
|
60.4% White
13.6% Hispanic or Latino
10.7% Race/ethnicity unspecified
8.3% Black or African American
6.3% Asian
0.7% American Indian or Alaska Native
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Intentions to use alcohol
|
2.5
|
3.5
|
2.3
|
2.8
|
2.1
|
2.8
|
2.6
|
|
2: Intentions to use tobacco
|
2.5
|
2.5
|
2.3
|
2.8
|
2.1
|
2.8
|
2.5
|
|
3: Media deconstruction skills for alcohol and tobacco
|
1.8
|
2.0
|
2.3
|
2.8
|
2.1
|
2.8
|
2.3
|
Study Strengths The internal consistency of the scales was good. The scale items have face validity and are similar to other well-known items that have shown good predictive validity in both cross-sectional and longitudinal contexts. The study used a randomized wait-list control group design. Teachers completed a fidelity checklist after teaching each lesson. Most students (73%) attended all scheduled lessons, and missing data and attrition were minimal (only about 3% of the students who completed the pretest did not complete the posttest). Statistical analyses were appropriate and thorough.
Study Weaknesses Neither test-retest reliability nor validity of the measures (other than face validity) was provided. Only the students' and teachers' self-reports were provided as fidelity measures. Only five schools were randomized into the intervention and control groups, which increased the chances of unmeasured differences between the schools in each group. Relative to the control group, students in the intervention group were much more likely to be in the 6th or 7th grade than in the 8th grade; although the investigators included students' grade level as a covariate, this approach did not necessarily equalize the groups because of the very small number of schools. The interval between the administration of the pre- and posttests was very short and only captured short-term effects.
|
|
Readiness for Dissemination
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
innovation Research and Training. Media Ready: Media Literacy Substance Abuse Prevention Project (grades 6-8). Teacher manual. Durham, NC: Author.
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.5
|
3.5
|
3.4
|
3.4
|
Dissemination Strengths The teacher manual is well organized, detailed, and easy to read, and it includes clear, logically sequenced sessions and guidance for conducting a hands-on practicum and other interactive activities. The facilitator's guide for conducting teacher training workshops is also well organized and logically sequenced, with clear guidance on how and when to incorporate other implementation tools. Each training module contains interactive components that help teachers build mastery of the intervention. Teachers must successfully complete an online certification test to validate their ability to conduct the program. Fidelity checklists include open-ended questions that give users an opportunity to detail adaptations.
Dissemination Weaknesses No guidance is provided on the optimal class size or methods for incorporating the intervention into existing classroom curricula. Because the teacher training focuses on the procedures associated with each session, it is unclear whether there are skills particular to the program model as a whole that teachers need to learn to implement the program effectively. The fidelity checklist is self-administered by teachers, which raises questions about whether its results can serve as an objective basis for improvement. No guidance is provided for measuring the outcomes of objectives.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Curriculum kit (includes teacher manual, CD with multimedia presentation, 1 Key Questions poster, 30 student workbooks, and 30 bookmarks)
|
$300 each
|
Yes
|
|
Additional student workbooks (pack of 10)
|
$30 per pack
|
No
|
|
1-day, on-site teacher training workshop
|
$2,800 for up to 20 participants, plus travel expenses
|
No
|
|
Teacher certification test
|
$25 each
|
No
|
|
Limited phone and email consultation
|
Free
|
No
|
|
Pre- and posttest outcome assessment instruments
|
Free
|
No
|
|
Fidelity checklists
|
Free
|
No
|
|
Implementation design and monitoring consultation
|
$175 per hour
|
No
|
|
Evaluation services consultation
|
$175 per hour
|
No
|
|
Contractual evaluation services
|
Varies depending on the number of participants, types of services, and number of evaluation reports needed
|
No
|
Additional Information The curriculum kit and additional student workbooks are available in English or Spanish.
|