Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Outcomes
| Outcome 1: Intellectual performance |
|
Description of Measures
|
Intellectual performance was assessed using the Stanford-Binet Intelligence Scale, Third Revision Form L-M, which was administered by trained program staff to each participant at study entry and annually from ages 4 to 10. The test consists of a series of age-appropriate items progressing from easy to difficult. Items for 5-year-olds, for example, include completing a picture of a man, folding paper into a triangle, copying a square, and identifying similarities and differences in pictures.
|
|
Key Findings
|
A study compared individuals who received the HighScope Curriculum starting at ages 3 and 4 with those who did not attend a preschool program. Mean intellectual performance scores were higher among intervention participants than their comparison group counterparts from the end of their first preschool year to age 7 (p < .01 at the end of the first and second preschool year, p < .05 at ages 6 and 7). The effect size was large at the end of the first preschool year (Cohen's d = 0.88) and second preschool year (Cohen's d = 0.87) and small at age 6 (Cohen's d = 0.32) and age 7 (Cohen's d = 0.30). There were no statistically significant differences between the groups at subsequent assessments.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 2: Vocabulary |
|
Description of Measures
|
Vocabulary was assessed using the first version of the Peabody Picture Vocabulary Test, which was administered by trained program staff to each study participant at study entry and annually from ages 4 to 9. For each item, the child is given a word (e.g., cow, accident, yawning) and asked to select the picture that depicts the word from among several options. The words and pictures are age appropriate and progress from easy to difficult.
|
|
Key Findings
|
A study compared individuals who received the HighScope Curriculum starting at ages 3 and 4 with those who did not attend a preschool program. Mean vocabulary scores were higher among intervention participants than their comparison group counterparts at 3 months after study entry (p < .05), the end of their first preschool year (p < .01), and the end of their second preschool year (p < .01). The effect size was small at 3 months (Cohen's d = 0.35), medium at the end of the first preschool year (Cohen's d = 0.74), and large at end of the second preschool year (Cohen's d = 0.91). There were no statistically significant differences between groups at subsequent assessments, but results yielded a small effect size at age 6 (Cohen's d = 0.29) and a very small effect size at age 7 (Cohen's d = 0.19) in favor of the intervention group.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 3: Educational achievement |
|
Description of Measures
|
Educational achievement was assessed using the following measures:
-
California Achievement Tests, a series of tests measuring student knowledge and skills in a variety of areas, including reading, language, and arithmetic. Each of the tests is standardized and administered in group settings by a trained test administrator. Students select the answer to each item (e.g., remembering information from a brief story, multiplication) from a set of four or five options. The tests were administered annually from ages 7 to 11 and at age 14.
- Rate of high school graduation, including award of a General Education Development (GED) certificate. Data on high school graduation status were obtained by trained program staff by searching public school records and adult high school records and by administering the HighScope Age 27 and Age 40 Follow-Up Interviews in face-to-face meetings with each study participant. The department of education and the intermediate school district searched GED records for participants with no record of high school graduation. Requests for information on enrollment status were sent to all postsecondary education programs identified by study participants in their follow-up interview.
|
|
Key Findings
|
A study compared individuals who received the HighScope Curriculum starting at ages 3 and 4 with those who did not attend a preschool program. Mean overall achievement test scores were higher among intervention participants than their comparison group counterparts at ages 9 (p < .05), 10 (p < .05), and 14 (p < .01). The effect sizes for these differences were small (Cohen's d = 0.29, 0.34, and 0.49, respectively). Specifically, in reading achievement, the intervention group outscored the comparison group at ages 10 and 14 (p values < .05), with small effect sizes (Cohen's d = 0.40 and 0.34, respectively). In arithmetic achievement, the intervention group outscored the comparison group at ages 9 and 14 (p values < .05), with small effect sizes (Cohen's d = 0.31 and 0.33, respectively). Lastly, in language achievement, the intervention group outscored the comparison group at ages 9 (p < .05) and 14 (p < .01), with a small effect size at age 9 (Cohen's d = 0.33) and a medium effect size at age 14 (Cohen's d = 0.63).
The high school graduation rate was higher for intervention participants than for comparison group participants at ages 19 (67% vs. 45%; p < .05), 27 (72% vs. 53%; p < .05), and 40 (77% vs. 60%; p < .01). The effect sizes for these differences were small (odds ratio = 2.36), medium (odds ratio = 2.94), and small (odds ratio = 2.25), respectively.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 4: Employment rate and earnings |
|
Description of Measures
|
Employment rate and earnings were assessed by self-report using the HighScope Age 27 and Age 40 Follow-Up Interviews as well as through data collected from social services, welfare, and employment records. The High Scope Follow-Up Interviews were administered by a trained interviewer in face-to-face meetings with each study participant.
|
|
Key Findings
|
A study compared individuals who received the HighScope Curriculum starting at ages 3 and 4 with those who did not attend a preschool program. Participants in the intervention group were more likely than their comparison group counterparts to be employed at age 27 (69% vs. 56%; p < .05) and age 40 (76% vs. 62%; p < .05). The effect size was small at both follow-ups (odds ratio = 2.29 and 2.45, respectively). Participants who received the HighScope Curriculum also had significantly greater annual and monthly earnings than participants from the comparison group at ages 27 and 40. The effect sizes were small for these results (odds ratios ranged from 1.8 to 2.5).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 5: Criminal arrests |
|
Description of Measures
|
Criminal arrests were assessed using data from arrest records collected by trained program staff. Data were collected from the Federal district courts in the State as well as from county circuit and district courts, including those in neighboring counties. Records were searched for names of all study participants (including maiden and married names, nicknames, aliases, and spelling variants). Matches were confirmed using birthdates and sometimes addresses and Social Security numbers. Staff linked names with criminal case numbers and accessed records of felonies, misdemeanors, and civil infractions. At the State level, data were collected from the Law Enforcement Information Network (LEIN), a database used by police that maintains statewide records of felony and high misdemeanor convictions and related information, as well as from State driving records.
When study participants reported living or having lived in another State, the State police agency was contacted to request a criminal record search for those individuals. Searches were also conducted using various State law enforcement Web sites and Rapsheets, an online criminal records database available to the public.
Criminal arrest data were collected at age 19 (juvenile arrest), age 27, and age 40. Arrests were divided into four categories:
-
Violent crimes (e.g., kidnapping, intent to murder, larceny, assault and/or battery)
-
Property crimes (e.g., taking or damaging someone else's property, fraud, theft, arson, breaking and entering)
-
Drug crimes (e.g., sale, possession or delivery of controlled substances, possession of drug paraphernalia)
- Other crimes (e.g., passing bad checks, driving offenses, disorderly conduct, criminal mischief, weapons offenses)
|
|
Key Findings
|
A study compared individuals who received the HighScope Curriculum starting at ages 3 and 4 with those who did not attend a preschool program. The intervention group had significantly fewer overall arrests by ages 27 and 40 than the comparison group (p values < .05) but no fewer juvenile arrests. The intervention group also had significantly fewer arrests for violent (p < .05), property (p < .01), and drug (p < .05) crimes by age 40 than the comparison group but no fewer arrests for other crimes. Within specific age groups, intervention participants had significantly fewer arrests for other crimes by age 19 (p < .05), drug crimes by age 27 (p < .05), and violent crimes (p < .05) and property crimes (p < .01) between ages 28 and 40 than comparison group participants.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.8
(0.0-4.0 scale)
|
| Outcome 6: Socioemotional development |
|
Description of Measures
|
Socioemotional development was assessed using the HighScope Child Observation Record (COR), a tool used to observe children ages 2.5 to 6 years in their natural setting and rate them along a variety of behavioral dimensions. Trained program observers used the COR to assess children-initiated behaviors in areas of development including initiative (e.g., complex play, participation in program activities) and social relations (e.g., relating to peers, social problem solving). The COR consists of several age-appropriate items, each with a 5-point rating scale.
|
|
Key Findings
|
Children with teachers who received in-service training and implemented the HighScope Curriculum during the school year were rated significantly higher on initiative (p = .005) and social relations (p = .01) than children from the comparison group, who participated in another early childhood program.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
0-5 (Early childhood)
|
50% Female
50% Male
|
100% Black or African American
|
|
Study 2
|
0-5 (Early childhood)
|
54% Female
46% Male
|
43% White
32% Black or African American
18% Race/ethnicity unspecified
6% Hispanic or Latino
1% American Indian or Alaska Native
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Intellectual performance
|
4.0
|
4.0
|
3.0
|
4.0
|
3.0
|
3.5
|
3.6
|
|
2: Vocabulary
|
4.0
|
4.0
|
3.0
|
4.0
|
3.0
|
3.5
|
3.6
|
|
3: Educational achievement
|
3.8
|
3.8
|
3.0
|
4.0
|
3.5
|
3.5
|
3.6
|
|
4: Employment rate and earnings
|
3.5
|
3.5
|
3.0
|
4.0
|
3.5
|
4.0
|
3.6
|
|
5: Criminal arrests
|
4.0
|
4.0
|
3.0
|
4.0
|
3.5
|
4.0
|
3.8
|
|
6: Socioemotional development
|
3.5
|
3.3
|
3.3
|
2.5
|
3.3
|
3.0
|
3.1
|
Study Strengths Most of the measures used have good, adequately documented psychometric properties. Systematic, manualized training was provided to teachers to enhance fidelity. Intervention fidelity was examined in several ways, including visits by outside experts, and documentation indicated the high degree to which teachers followed the intervention curriculum; for example, teachers provided written notes on fidelity for several months. Attrition was minimal considering the length of the follow-up (i.e., age 27, age 40). The analyses used were sufficient.
Study Weaknesses In one of the studies, confounding variables were not adequately addressed. Further, nearly one third of the children had missing data on some items of the Preschool Child Observation Record used in this study, posing a threat to the statistical power of the results.
|
|