Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Clarke, A. M., & Barry, M. M. An evaluation of the Zippy's Friends emotional wellbeing programme for primary schools in Ireland. Health Promotion Research Centre, National University of Ireland, Galway. Study 2Mishara, B. L., & Ystgaard, M. Effectiveness of a mental health promotion program to improve coping skills in young children: "Zippy's Friends." Early Childhood Research Quarterly, 21(1), 110-123. Study 3Dufour, S., Denoncourt, J., & Mishara, B. L. Improving children's adaptation: New evidence regarding the effectiveness of Zippy's Friends, a school mental health promotion program. Advances in School Mental Health Promotion, 4(3), 18-28.
Supplementary Materials Clarke, A. M. An evaluation of Zippy's Friends, an emotional wellbeing programme for children in primary schools (Unpublished doctoral thesis). National University of Ireland, Galway.
Ryan-Wenger, N. M. Development and psychometric properties of the Schoolagers' Coping Strategy Inventory. Nursing Research, 39(6), 344-349.
Outcomes
| Outcome 1: Emotional literacy |
|
Description of Measures
|
Emotional literacy was measured using the Emotional Literacy Checklist, a 20-item instrument with 5 subscales: self-awareness, self-regulation, motivation, empathy, and social skills. Items are rated on a scale from 1 (not at all true) to 4 (very true). Items within each subscale are summed to obtain subscale scores, and an overall emotional literacy score is obtained by summing all 20 items. Higher scores indicate greater emotional literacy. Teachers completed the checklist for each student before and after the intervention.
|
|
Key Findings
|
In a study in Ireland, students who received the intervention showed a significant increase in emotional literacy from pre- to posttest compared with a control group that received the standard, compulsory social, personal, and health curriculum (p < .001).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 2: Hyperactivity |
|
Description of Measures
|
Hyperactivity was measured using:
- Strengths and Difficulties Questionnaire (SDQ). This brief, standardized questionnaire is used to measure children's emotional and behavioral well-being. The questionnaire has 25 items divided into 5 subscales (5 items each): emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems, and prosocial behavior. All items are scored on a scale from 0 to 2; lower scores on the hyperactivity/inattention subscale indicate less hyperactivity. Teachers used the SDQ to rate each student before and after the intervention.
- Social Skills Questionnaire Teacher Form (SSQTF), Elementary Level. The questionnaire is used to obtain teacher reports on the frequency of three types of problem behavior: hyperactivity, externalizing, and internalizing. Examples of behavior measured by the hyperactivity subscale include "Interrupts conversations with others" and "Acts impulsively." Assessments occurred before and after the intervention.
|
|
Key Findings
|
In a study in Ireland, students who received the intervention showed a significant decrease in hyperactivity from pre- to posttest compared with a control group that received the standard, compulsory social, personal, and health curriculum (p = .05).
In jointly reported studies conducted in Lithuania and Denmark, Lithuanian students who received the intervention had a significant decrease in hyperactivity from pre- to posttest compared with wait-list control students (p < .001). This outcome was not consistently measured in the Denmark study.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
| Outcome 3: Coping skills |
|
Description of Measures
|
Coping skills were measured using an observation form developed for the study. The form was based on the Schoolagers Coping Strategies Inventory, a self-report instrument with 26 questions that ask children to rate on a scale of 0-3 how often they take various actions when they feel stressed, nervous, or worried, such as "ask someone for help," "hit someone," or "get mad." In the study, teachers used the form to report on children's behavior observed at school. The form asked teachers to describe the most important conflict or problem the child had recently experienced. Teachers were further asked to describe how the child behaved in that situation and to indicate which of the 26 items of the Schoolagers Coping Strategies Inventory the child used in that situation. The situations were categorized by content, and the frequencies of the individual coping skills were examined individually and together. Teachers completed the form for each student before and after the intervention.
|
|
Key Findings
|
In jointly reported studies conducted in Lithuania and Denmark, Lithuanian students who received the intervention had a significant increase in coping strategies compared with wait-list control students (p < .001). This outcome was not consistently measured in the Denmark study.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.3
(0.0-4.0 scale)
|
| Outcome 4: Social skills |
|
Description of Measures
|
Social skills were measured using:
- Emotional Literacy Checklist. This 20-item checklist includes 5 subscales that measure various dimensions of emotional literacy: self-awareness, self-regulation, motivation, empathy, and social skills. Each item is rated from 1 (not at all true) to 4 (very true). Items within each subscale are summed to obtain subscale scores, and an overall emotional literacy score is obtained by summing all 20 items. Higher scores indicate greater emotional literacy. This outcome was measured using scores on the social skills subscale. Teachers completed the checklist for each student before and after the intervention.
- Social Skills Questionnaire Teacher Form, (SSQTF), Elementary Level. This questionnaire involves teacher ratings of the frequency of various observed behaviors. Subscale scores were calculated for three dimensions of social skills: cooperation, assertion, and self-control. Examples of behaviors specified in the form are "Attends to your instructions," "Easily makes transition from one classroom activity to another" (cooperation); "Joins ongoing activity or group without being told to do so," "Invites others to join in activities" (assertion); and "Controls temper in conflict situations with peers," "Receives criticism well" (self-control). Assessments occurred before and after the intervention.
- Socio-Emotional Profile. This instrument includes 80 items measuring social competencies and adaptation problems on a 6-point scale (from never to always). Three of the 12 subscales in the Socio-Emotional Profile address social skills: cooperation with adults, autonomy, and internalization of behaviors. Examples of items include "helping teachers" (cooperation); "perseverance in solving their problems themselves" (autonomy); and "anxious behaviors, dependency, nervousness and symptoms of depression such as crying without any reason" (internalization of behaviors). Teachers completed the full Socio-Emotional Profile to rate students before and after the intervention.
|
|
Key Findings
|
In a study in Ireland, students who received the intervention showed significant improvement in social skills from pre- to posttest compared with a control group that received the standard, compulsory social, personal, and health curriculum, after controlling for baseline scores (p = .01)
In jointly reported studies conducted in Lithuania and Denmark, Lithuanian students who received the intervention had significantly greater improvement in social skills from pre- to posttest compared with wait-list control students (p < .001). Danish students who received the intervention had significantly greater improvement in social skills from pre- to posttest compared with historical data for students tracked the year prior to program implementation (p < .001).
In a Canadian study, students who received the intervention showed significant pre- to posttest improvements in cooperation with adults (p < .001), autonomy (p < .02), and internalization of behaviors (p = .001) compared with peers in a no-intervention control group.
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
0-5 (Early childhood)
6-12 (Childhood)
|
52.3% Male
47.7% Female
|
100% Non-U.S. population
|
|
Study 2
|
0-5 (Early childhood)
6-12 (Childhood)
|
51.3% Male
48.7% Female
|
100% Non-U.S. population
|
|
Study 3
|
0-5 (Early childhood)
6-12 (Childhood)
|
52.9% Male
47.1% Female
|
100% Non-U.S. population
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Emotional literacy
|
3.0
|
2.0
|
2.0
|
3.0
|
3.0
|
3.5
|
2.8
|
|
2: Hyperactivity
|
3.3
|
2.0
|
2.0
|
2.5
|
2.8
|
3.3
|
2.6
|
|
3: Coping skills
|
3.0
|
2.0
|
2.0
|
1.5
|
2.5
|
3.0
|
2.3
|
|
4: Social skills
|
2.5
|
2.0
|
2.0
|
3.8
|
2.5
|
3.0
|
2.6
|
Study Strengths The research design of one study used random assignment. The measures have acceptable and demonstrated reliability and at least face validity. Information on intervention fidelity collected by the investigators generally showed that the program was implemented as planned, even when teachers had the freedom to do otherwise. Missing data and attrition were well described and within an acceptable range given the nature of the study. Some problems with confounding variables were present, but on the whole, these were handled appropriately in the statistical methods. The analytic methodology used by the investigators was appropriate overall.
Study Weaknesses Validity data were not explicitly presented for most measures, and where validity data were reported, the measure had been modified for use in the study. In the jointly reported Danish and Lithuanian studies, collection of data for the program evaluation was not carried out as intended in one of the two countries. Outcomes were reported using teacher ratings only, even for measures originally intended to collect student ratings. One study reported only one-tailed p values.
|
|