Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Blum, N., St. John, D., Pfohl, B., Stuart, S., McCormick, B., Allen, J., et al. Systems Training for Emotional Predictability and Problem Solving (STEPPS) for outpatients with borderline personality disorder: A randomized controlled trial and 1-year follow-up. American Journal of Psychiatry, 165(4), 468-478. Study 2Bos, E.H., van Wel, E. B., Apello, M. T., & Verbraak, M. J. A randomized controlled trial of a Dutch version of Systems Training for Emotional Predictability and Problem Solving for borderline personality disorder. Journal of Nervous and Mental Disease, 198(4), 299-304. Study 3Harvey, R., Black, D. W., & Blum, N. STEPPS (Systems Training for Emotional Predictability and Problem Solving) in the United Kingdom: A preliminary report. Journal of Contemporary Psychotherapy, 40, 225-232.
Outcomes
| Outcome 1: Depression symptoms |
|
Description of Measures
|
Depression symptoms were assessed using the Beck Depression Inventory II (BDI-II), a 21-item self-report instrument that assesses the presence and severity of depression symptoms over the past 2 weeks. The BDI-II is based on the original Beck Depression Inventory but contains changes to align the instrument with current DSM-IV criteria for depression. For each item, respondents choose one of four statements that best represents their recent feelings or behavior.
|
|
Key Findings
|
In a study conducted in the United States, BPD patients were randomly assigned to the intervention group, which received STEPPS plus treatment as usual, or to the control group, which received only treatment as usual. Treatment as usual included individual psychotherapy, medication, and case management. The BDI-II was administered at pretest and weeks 4, 8, 12, 16, and 20 of the 20-week intervention period. Results from the BDI-II indicated the intervention group experienced significantly greater improvement in depression symptoms from pretest to week 20 compared with the control group (p = .03).
In a study conducted in the United Kingdom, BPD patients who received STEPPS were assessed for depression symptoms at pre- and posttest using the BDI-II. Mean scores on this instrument showed a significant improvement in participants' depression symptoms over the intervention period (p < .001).
|
|
Studies Measuring Outcome
|
Study 1, Study 3
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
| Outcome 2: BPD symptoms |
|
Description of Measures
|
BPD symptoms were assessed using the following instruments:
- The Zanarini Rating Scale for Borderline Personality Disorder (ZAN-BPD). This structured clinical interview schedule provides diagnostic information and measures change in DSM-IV borderline psychopathology using a 5-point rating scale. Subscales measure affective disturbance, cognition, impulsivity, and relationship difficulties.
- The Borderline Evaluation of Severity Over Time (BEST). The BEST is a 15-item self-report questionnaire that assesses the BPD symptom severity and change in symptoms over time. The instrument has three subscales. The first two subscales are based on the DSM-IV criteria for BPD. The first subscale (8 items) addresses problematic thoughts and feelings that are characteristic of BPD (i.e., suicidal thoughts), while the second subscale (4 items) addresses problematic or negative behaviors (i.e., problems with impulsive behavior). Items on these subscales are rated from 1 to 5 (1 = no distress caused by this symptom, 5 = extreme distress). The third subscale of the BEST consists of three items that assess the use of positive behaviors learned in therapy (i.e., following through with therapy plans). These items are rated based on frequency over the course of a week.
- The Borderline Personality Disorder Checklist-40 (BPD-40). This is a self-report questionnaire with 40 items covering the DSM-IV criteria for BPD. It assesses the severity of borderline-related symptoms during the past month.
|
|
Key Findings
|
In a study conducted in the United States, BPD patients were randomly assigned to the intervention group, which received STEPPS plus treatment as usual, or to the control group, which received only treatment as usual. Treatment as usual included individual psychotherapy, medication, and case management. The ZAN-BPD was administered at pretest and weeks 8 and 20 of the 20-week intervention period, and the BEST was administered at pretest and weeks 4, 8, 12, 16, and 20. The intervention group had a significantly greater decrease in BPD symptoms from pretest to week 20 compared with the control group, as demonstrated by the ZAN-BPD total score (p = .001) and subscales assessing affective (p = .007), cognitive (p = .027), interpersonal (p = .026), and impulsive domains (p = .010). Scores on one of the BEST subscales (thoughts and feelings) also showed a significantly greater decrease in BPD symptoms among the intervention group (p = .02), but no significant differences were found with the other two BEST subscales.
In a study conducted in the Netherlands, BPD patients were randomly assigned to the intervention group, which received STEPPS plus an adjunctive individual therapy, or to the control group, which received treatment as usual. Treatment as usual consisted of individual therapy provided by a psychotherapist, psychologist, or psychiatric nurse. Data on BPD symptoms were collected using the BPD-40 at pretest, posttest, and 6-month follow-up. Results showed that symptoms of BPD generally decreased from pre- to posttest and from pretest to follow-up in both groups, but the decreases were significantly greater in the intervention group (p = .001).
In a study conducted in the United Kingdom, BPD patients who received STEPPS were assessed using the ZAN-BPD at pretest and posttest. In addition, the BEST was administered weekly over the 20-week intervention period. Participants' BPD symptoms improved significantly from pre- to posttest as indicated by the ZAN-BPD total score (p < .001) and the ZAN-BPD subscales assessing affective (p < .001), cognitive (p < .001), interpersonal (p < .010), and impulsive domains (p < .001). Participants' total BEST scores also showed significant improvement over the intervention period (p < .01).
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
| Outcome 3: Impulsivity |
|
Description of Measures
|
Symptoms of impulsivity were assessed using the following instruments:
- The Zanarini Rating Scale for Borderline Personality Disorder (ZAN-BPD). This structured clinical interview schedule provides diagnostic information as well as measuring change in DSM-IV borderline psychopathology. Subscales measure affective disturbance, cognition, impulsivity, and relationship difficulties. Items are scored on a 5-point scale.
- The Barratt Impulsiveness Scale (BIS). The BIS includes 30 items, which may be scored to yield 6 first-order factors (attention, motor, self-control, cognitive complexity, perseverance, and cognitive instability impulsiveness) and 3 second-order factors (attentional, motor, and nonplanning impulsiveness). A total score is obtained by summing the first- or second-order factors. Items are scored on a 4-point scale (1 = rarely/never, 4 = almost always/always).
|
|
Key Findings
|
In a study conducted in the United States, BPD patients were randomly assigned to the intervention group, which received STEPPS plus treatment as usual or to the control group, which received only treatment as usual. Treatment as usual included individual psychotherapy, medication, and case management. The ZAN-BPD and BIS were administered at pretest and weeks 4, 8, 12, 16, and 20 of the 2-week intervention period. From pretest to week 20, the intervention group had significantly greater improvement than the control group in symptoms of impulsivity, as measured by mean scores on both the ZAN-BPD impulsivity subscale group (p = .010) and the BIS (p = .004).
In a study conducted in the United Kingdom, BPD patients who received STEPPS were assessed using the ZAN-BPD at pretest and posttest. In addition, the BEST was administered weekly over the 20-week intervention period. From pre- to posttest, participants showed significant improvements on both the ZAN-BPD impulsivity subscale (p < .01) and the BEST total score (p < .001).
|
|
Studies Measuring Outcome
|
Study 1, Study 3
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
| Outcome 4: Positive and negative affect |
|
Description of Measures
|
Positive and negative affect was assessed using the following instruments:
- The Positive and Negative Affect Schedule (PANAS). The PANAS is a 20-item self-report measure of mood states. The instrument includes 10 items representing positive affect and 10 items representing negative affect. Items are scored on a 5-point Likert scale (1 = very slightly or not at all, 5 = extremely). For each category (positive and negative), a score between 10 and 50 is possible.
- The PANAS-X. The PANAS-X is a longer version of the PANAS that measures fear, hostility, guilt, sadness, joviality, self-assurance, attentiveness, shyness, fatigue, serenity, and surprise, as well as positive and negative affect. The PANAS-X's 74 self-report items are measured on the same 5-point Likert scale used in the original PANAS.
|
|
Key Findings
|
In a study conducted in the United States, BPD patients were randomly assigned to the intervention group, which received STEPPS plus treatment as usual, or to the control group, which received only treatment as usual. Treatment as usual included individual psychotherapy, medication, and case management. The PANAS was administered at pretest and weeks 4, 8, 12, 16, and 20 of the 20-week intervention period. The intervention group had a significantly greater decrease in negative affect from pretest to week 20 compared with the control group (p = .03). The intervention group also had a greater increase in positive affect compared with the control group, but this difference was not statistically significant.
In a study conducted in the United Kingdom, BPD patients who received STEPPS were assessed using the PANAS-X at pretest and posttest. Participants' mean scores on this instrument indicated a significant decrease in negative affect (p < .001) and a significant increase in positive affect (p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 3
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
| Outcome 5: Emergency department visits |
|
Description of Measures
|
Participants were asked to report the number of times they visited an emergency department within the past month.
|
|
Key Findings
|
In a study conducted in the United States, BPD patients were randomly assigned to the intervention group, which received STEPPS plus treatment as usual, or to the control group, which received only treatment as usual. Treatment as usual included individual psychotherapy, medication, and case management. Participants in the intervention group averaged 0.97 month per year with at least one emergency department visit, while participants in the control group averaged 1.52 (p = .040).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.1
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
|
83% Female
17% Male
|
94% White
3% Race/ethnicity unspecified
2% Black or African American
|
|
Study 2
|
26-55 (Adult)
|
86% Female
14% Male
|
100% Non-U.S. population
|
|
Study 3
|
18-25 (Young adult)
26-55 (Adult)
|
84% Female
16% Male
|
100% Non-U.S. population
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Depression symptoms
|
4.0
|
4.0
|
2.6
|
2.0
|
2.0
|
1.8
|
2.7
|
|
2: BPD symptoms
|
3.7
|
4.0
|
2.3
|
2.1
|
2.1
|
2.2
|
2.7
|
|
3: Impulsivity
|
4.0
|
3.5
|
2.6
|
2.0
|
2.0
|
1.8
|
2.6
|
|
4: Positive and negative affect
|
4.0
|
4.0
|
2.6
|
2.0
|
2.0
|
1.8
|
2.7
|
|
5: Emergency department visits
|
1.3
|
2.0
|
2.8
|
2.0
|
2.5
|
2.0
|
2.1
|
Study Strengths Most of the measures used are standardized and well established with strong psychometric properties. In all three studies, steps were taken to monitor intervention fidelity, such as periodic training and phone supervision and the use of independent observers to rate adherence to the model. Two of the studies used random assignment to the intervention and control conditions.
Study Weaknesses One outcome, emergency department visits, was measured using only participants' self-report, without verification using administrative service records or other collateral reports. In all three studies, the instruments used to monitor intervention fidelity had unspecified or unknown psychometric properties. High attrition rates were reported in two studies, and only one study included an intent-to-treat analysis. One of the studies used a comparatively weak pre- and posttest design. Sample sizes were small, which might have affected the studies' statistical power to infer relationships between intervention and control groups.
|
|
Readiness for Dissemination
|
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Blum, N., Bartels, N., St. John, D., & Pfohl, B. STEPPS: Group Treatment Program for Borderline Personality Disorder [manual on CD-ROM]. Coralville, IA: Level 1 Publishing.
Blum, N., Bartels, N., St. John, D., & Pfohl, B. STEPPS: Group Treatment Program for Borderline Personality Disorder: Skill pocket cards.
Blum, N., Bartels, N., St. John, D., & Pfohl, B. STEPPS: Group Treatment Program for Borderline Personality Disorder: Quick reference (2nd ed.).
Blum, N., & Pfohl, B. BEST: Borderline Evaluation of Severity over Time (excerpted from STEPPS manual)
STEPPS Adherence Rating Scale.
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.8
|
1.8
|
3.0
|
2.8
|
Dissemination Strengths The program materials for both participants and facilitators are well developed and well organized. The sessions are designed to be interactive, and the materials provide facilitators with tips on how to follow the intended structure. The developer offers a 2-day, on-site training and email consultation after training. Ongoing weekly phone supervision is available through the developer. Easy-to-use facilitator adherence and client outcome measures are provided to support quality assurance.
Dissemination Weaknesses Additional, organization-level implementation guidance is needed, including information on what qualifications implementers should have and further details about recommended implementation steps. There is little information available about the content and structure of trainings offered by the developer. The Web site offers minimal support. The rating scales do not take into account details such as the client's difficulty level or stage of therapy. Little information is provided on how implementers should use quality assurance data to improve program delivery.
|
|