Challenging Horizons Program (CHP)
The Challenging Horizons Program (CHP) is a school-based set of interventions for middle/junior high school students with attention-deficit/hyperactivity disorder (ADHD). Building on behavioral and cognitive theories about the nature of the disorder, CHP aims to provide a safe learning environment enhanced by supportive counseling relationships between students and staff. CHP targets the most common areas of impairment associated with ADHD, including social impairment, family conflict, and academic failure. CHP interventions focus on strengthening enabling skills such as organization; specific educational skills such as homework management, studying, and notetaking; socialization; goal setting and self-regulation of behavior; and recreational activities to improve the student's group cooperation, fitness, and sports skills. Parent involvement is encouraged through group parent training and weekly reports. Each intervention is provided in the context of supportive and enjoyable activities.
Two versions of CHP have been evaluated: an after-school model and a consulting model. In the after-school model, a small group of counselors delivers CHP interventions to 5-20 students 2-3 days per week as part of an after-school program. The counselors consult regularly with the students' teachers and hold monthly meetings with the students' parents. In the consulting model, CHP interventions are provided during the school day by school staff (e.g., teachers, school counselors). Staff members volunteer to serve as mentors for individual students, establishing a supportive relationship with the student and providing selected CHP interventions through one-on-one meetings once or twice per week at times established by the mentor (e.g., before school, after school, at lunch, during homeroom, during time assigned for the use of a special education resource room). In the consulting model, up to three CHP interventions are provided to each student at any given time. The after-school model may provide a greater number of concurrent interventions.
CHP is implemented throughout the school year and may be led by a master's or doctoral-level psychologist, social worker, counselor, or behavior specialist, or others who have received training in the CHP. The after-school version of the CHP also requires paraprofessional counselors in a ratio of one staff for every 2 or 3 students.
Descriptive Information
|
Areas of Interest
|
Mental health treatment
|
|
Outcomes
|
1: ADHD symptoms
2: Social functioning
3: Academic performance
4: School functioning
|
|
Outcome Categories
|
Education
Mental health
Social functioning
|
|
Ages
|
6-12 (Childhood)
13-17 (Adolescent)
|
|
Genders
|
Male
Female
|
|
Races/Ethnicities
|
Black or African American
Hispanic or Latino
White
Race/ethnicity unspecified
|
|
Settings
|
School
|
|
Geographic Locations
|
Urban
Suburban
Rural and/or frontier
|
|
Implementation History
|
The after-school model of CHP was first implemented in a middle school in Harrisonburg, Virginia. Since then, it has been implemented in Columbia, South Carolina; Pittsburgh, Pennsylvania; and Athens, Cincinnati, Lancaster, and Logan, Ohio. The consulting model was implemented in Harrisonburg and has been implemented in all four of the Ohio sites. More than 1,500 students have participated in one of the two models of CHP.
|
|
NIH Funding/CER Studies
|
Partially/fully funded by National Institutes of Health: Yes
Evaluated in comparative effectiveness research studies: No
|
|
Adaptations
|
No population- or culture-specific adaptations of the intervention were identified by the developer.
|
|
Adverse Effects
|
No adverse effects, concerns, or unintended consequences were identified by the developer.
|
|
IOM Prevention Categories
|
IOM prevention categories are not applicable.
|
Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Outcomes
| Outcome 1: ADHD symptoms |
|
Description of Measures
|
ADHD symptoms were assessed using the Behavior Assessment System for Children (BASC). The BASC is a comprehensive behavior rating scale that measures internalizing and externalizing behaviors. The standardized scores on the inattention and hyperactivity subscales were used for analyses of ADHD symptoms. Parents completed the BASC at baseline and at follow-up evaluations.
|
|
Key Findings
|
Sixth-grade students were randomly assigned to an intervention group that received CHP or a community care control condition (i.e., students were free to pursue educational, psychosocial, or medication treatment of their choice but were not provided any treatment by the investigators). Students were followed through the end of 7th grade. From the initial evaluation to the final follow-up, CHP participants showed significant improvement in attentiveness, but not in hyperactivity. For inattention, change over time was significantly different for the two groups (p < .05), with the control group getting worse over time and the CHP participants showing improvement; the effect size for this finding was medium (Cohen's d = 0.76). Both the CHP participants and the control group showed a decrease in hyperactivity, with hyperactivity ratings decreasing more sharply over time for the CHP participants compared with the control group, but the difference between groups was not statistically significant.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
| Outcome 2: Social functioning |
|
Description of Measures
|
Social functioning was assessed using the Impairment Rating Scale (IRS). The IRS is a brief rating scale that measures general functioning in various domains (including self-esteem and relationships with peers, siblings, parents, and teachers) and overall problem severity. Parents or teachers indicate the degree to which functioning in the domains is or is not a problem that requires additional treatment. Scores range from 0 (no problem, does not need intervention) to 6 (extreme problem, needs intervention). Higher scores indicate greater impairment and need for additional treatment. Parents and teachers completed the IRS at baseline and then either on a monthly basis (in one study) or one time following completion of the first semester of school (in another study).
|
|
Key Findings
|
In one study, 6th-grade students with ADHD were randomly assigned to an intervention group that received CHP or a community care control condition (i.e., students were free to pursue educational, psychosocial, or medication treatment of their choice but were not provided any treatment by the investigators). Students were followed through the end of 7th grade. Parent ratings showed a significantly different change over time in overall social functioning for the two groups, with the control group exhibiting increasing impairment over time and CHP participants improving over time (p < .05). The effect size for this finding was small (Cohen's d = 0.40). Teacher ratings showed no significant between-group difference over time in social functioning.
In another study, 6th- and 7th-graders experiencing a combination of learning and behavior problems were randomly assigned to an intervention group that received CHP or a control condition that included involvement in a district-run after-school program. Parent ratings indicated that CHP participants improved in self-esteem from baseline to the end of the semester while the control group showed a worsening in self-esteem (p = .021). The effect size for this finding was medium (Cohen's d = 0.50). In addition, parent ratings indicated that CHP participants moved from exhibiting overall problem severity at baseline to no longer exhibiting problems at the end of the semester; there was no change for the control group over this period (p = .004). The effect size for this finding was medium (Cohen's d = 0.77). Teacher ratings did not indicate any significant between-group difference over time in self-esteem.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 3: Academic performance |
|
Description of Measures
|
Academic performance was assessed using grade point averages (GPAs) for core subjects. GPAs were computed for each grading period as follows: A = 4.0, B = 3.0, C = 2.0, D = 1.0, and F = 0.
|
|
Key Findings
|
In one study, 6th-grade students with ADHD were randomly assigned to an intervention group that received CHP or a community care control condition (i.e., students were free to pursue educational, psychosocial, or medication treatment of their choice but were not provided any treatment by the investigators). Students were followed through the end of 7th grade. The investigators' focus in this study was the prevention of academic failure, defined as grading periods for which the student's academic performance was unsatisfactory for most (if not all) core subjects, as agreed on by teachers. By the end of 6th grade, 77% of CHP participants did not experience academic failure in core subjects (reading/English, math, science, and social studies) compared with 59% of students in the control group (p = .03). By the end of 7th grade, 74% of CHP participants did not experience academic failure, compared with 48% of students in the control group (p = .04).
In another study, 6th- and 7th-graders experiencing a combination of learning and behavior problems were randomly assigned to an intervention group that received CHP or a control condition that included involvement in a district-run after-school program. A comparison of mean grades at quarter 1 and quarter 2 in four core subjects (science, history, math, and language arts) revealed that the only significant difference between the intervention and control groups was for science grades at quarter 2: CHP participants had better grades in science than students from the control group (p = .016).
In a third study, students with ADHD in grades 4-7 were randomly assigned to an intervention group that immediately received CHP or a wait-list control group. Results of this study showed a significant improvement in GPAs combined across the four core subjects (math, science, language arts, and history) from pretest to 8-week follow-up for CHP participants but not for the control group (p < .01).
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 4: School functioning |
|
Description of Measures
|
School functioning was assessed using the following:
- Impairment Rating Scale (IRS): The IRS is a brief rating scale that measures general functioning in various domains including school functioning. The IRS includes six questions pertaining to school, family, self-esteem, and social functioning. Parents or teachers indicate the degree to which functioning in the domains is or is not a problem that requires additional treatment. Scores range from 0 (no problem, does not need intervention) to 6 (extreme problem, needs intervention). Higher scores indicate greater impairment and need for additional treatment. Parents and teachers completed the IRS at baseline and following completion of the first semester of school.
- Conners Abbreviated Symptom Questionnaire (ASQ): The 10-item Conners Global Index (CGI) portion of the scale evaluates the reported severity of childhood problems related to behaviors important for school functioning. The CGI response format requires parents to answer 10 questions related to their child's behavior (e.g., disturbs other children, restless or overactive, fails to finish things he/she starts) using a 4-point scale (0 = not at all true; 1 = just a little true; 2 = pretty much true; and 3 = very much true). The ASQ (and CGI) was administered to parents at baseline and following the completion of the first semester of school.
- Homework Problems Checklist (HPC): The HPC is a parent report instrument consisting of 20 items. It is commonly used as a screening tool and outcome measure to assess homework problems. HPC has two subscales: Inattention/Avoidance of Homework (IA) and Poor Productivity/Nonadherence to Homework Rules (NA). The IA and NA factors can be combined to produce an HPC total score. The HPC was completed by parents at baseline and after the intervention.
|
|
Key Findings
|
In one study, 6th- and 7th-graders experiencing a combination of learning and behavior problems were randomly assigned to an intervention group that received CHP or a control condition that included involvement in a district-run after-school program. Results of this study showed that:
- Parent ratings on the IRS showed that CHP participants improved in school functioning from baseline to the end of the semester (from a high level of impairment to a lower level of impairment), whereas for students in the control group, school functioning was highly impaired at baseline and was still highly impaired at the end of the semester (p = .004). The effect size for this finding was medium (Cohen's d = 0.74). Teacher ratings of school functioning using the IRS did not reveal any significant between-group difference in school functioning.
- Parent reports on the ASQ indicated that CHP participants made large improvements in organizational skills important in school functioning over the semester compared with students in the control group (p = .002). The effect size for this finding was medium (Cohen's d = 0.65). Parent reports on the ASQ revealed no significant between-group difference in homework completion.
- Parent reports on the CGI showed that CHP participants had fewer behavioral functioning difficulties in school following completion of the intervention, whereas parents of students in the control group reported that behavioral functioning became more impaired over the semester (p < .001). The effect size for this finding was medium (Cohen's d = 0.74).
In another study, students with ADHD in grades 4-7 were randomly assigned to an intervention group that immediately received CHP or a wait-list control group. HPC total scores showed that CHP participants improved overall in homework performance over the course of the 8-week intervention, whereas the control group had no significant change (p < .01). The effect size for this finding was medium (Cohen's d = 0.71). Ratings on the HPC subscales showed that CHP participants made significant improvement in both the IA factor (p < .001) and NA factor (p < .001), whereas the control group had no significant change in either factor.
|
|
Studies Measuring Outcome
|
Study 2, Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
13-17 (Adolescent)
|
77% Male
23% Female
|
94% White
4% Hispanic or Latino
2% Race/ethnicity unspecified
|
|
Study 2
|
6-12 (Childhood)
13-17 (Adolescent)
|
66.7% Male
33.3% Female
|
66% Black or African American
34% White
|
|
Study 3
|
6-12 (Childhood)
13-17 (Adolescent)
|
83.8% Male
16.2% Female
|
70% White
30% Black or African American
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: ADHD symptoms
|
4.0
|
4.0
|
2.0
|
3.0
|
2.0
|
4.0
|
3.2
|
|
2: Social functioning
|
3.5
|
3.5
|
2.3
|
3.5
|
2.3
|
3.8
|
3.1
|
|
3: Academic performance
|
3.0
|
2.5
|
2.5
|
3.5
|
2.5
|
4.0
|
3.0
|
|
4: School functioning
|
3.5
|
3.5
|
2.8
|
3.8
|
2.5
|
3.8
|
3.3
|
Study Strengths The studies used measures that have been well researched and have strong psychometric properties. Manualized interventions were used, and training was provided to the interventionists. Some attempts were made to assess intervention fidelity using, for example, counselor's checklists, attendance logs, on-site staff supervision, and training. Attrition was generally low, and the researchers addressed missing data in the analyses. Appropriate statistical analyses were used in all three studies.
Study Weaknesses Treatment integrity was not adequately addressed. For example, in one study, there was variability in the services received within the intervention group, and only one of the three studies employed an outside observer with a checklist to assess integrity. A number of confounding variables raise concerns about internal validity. Specifically, raters (parents and teachers) were not blinded to condition; analyses did not control for students' medication status, intelligence, and disciplinary status; and there was no external validation of the criteria used to assess students' learning disabilities or other comorbid diagnoses. The small sample size in all three studies limits the ability to evaluate potential moderators of treatment efficacy and may have affected the power of the analyses.
|
|
Readiness for Dissemination
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Other program materials:
- 21st Century Grant CHP After-School Program Timeline
- Accreditation Visit Procedures and Criteria
- Challenging Horizons Program Staff Manual for the Boys and Girls Club at Dent Middle School
- Challenging Horizons Program Treatment Fidelity Assessment
- Challenging Horizons Program Weekly Report Card
- CHP Rules Quiz
- CHP Rules Quiz Answer Key
- CHP Training Schedule
- Classroom Performance Survey
- Low-Emotion Reprimand [PowerPoint slides]
- Low-Emotion Reprimand Worksheet
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.5
|
4.0
|
3.8
|
3.8
|
Dissemination Strengths The curriculum is thorough and supports the program's goals and concepts. Many well-developed forms and protocols are provided to guide implementation of the highly manualized intervention. The Weekly Report Card offers good linkage between staff and parents. Planning materials are customized to each site and include clearly written manuals, useful tools, and a comprehensive list of tasks to be completed; considerable focus is placed on implementation readiness. The program Web sites offer information for a variety of audiences. Detailed and comprehensive fidelity measures are provided for quality assurance. High-quality training materials and other support resources are available, and trainers conduct site visits after implementation begins to determine whether additional training and continued telephone consultation are needed.
Dissemination Weaknesses The student manual lacks engaging qualities such as color, pictures, and graphics and uses language that may not be ideal for the target age group. Further guidance is needed to aid decisionmakers in justifying the adoption of this program, such as a justification for the high staff-to-student ratio. The Treatment Fidelity Assessment is fairly long and unwieldy.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Manuals
|
Included in the cost of training
|
Yes
|
|
Support materials
|
Included in the cost of training
|
Yes
|
|
15-hour off-site training
|
$4,000 per site for up to 12 participants, plus travel expenses
|
Yes
|
|
Phone consultation
|
Included in the cost of training (up to 16 hours)
|
Yes
|
|
Accreditation site visit
|
Included in the cost of training (travel expenses additional)
|
Yes
|
|