Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Robbins, M. S., Feaster, D. J., Horigian, V. E., Rohrbaugh, M., Shoham, V., Bachrach, K., Miller, M., Burlew, K. A., Hodgkins, C., Carrion, I., Vandermark, N., Schindler, E., Werstlein, R., & Szapocznik, J. Brief Strategic Family Therapy versus treatment as usual: Results of a multisite randomized trial for substance using adolescents. Journal of Consulting and Clinical Psychology, 79(6), 713–727.
Supplementary Materials Robbins, M. S., Feaster, D. J., Horigian, V. E., Puccinelli, M. J., Henderson, C., & Szapocznik, J. Therapist adherence in Brief Strategic Family Therapy for adolescent drug abusers. Journal of Consulting and Clinical Psychology, 79(6), 43–53.
Outcomes
| Outcome 1: Drug use |
|
Description of Measures
|
Adolescent drug use was assessed using the Timeline Follow-Back (TLFB) method, a semistructured, self-report, calendar-based interview that asks participants to retrospectively estimate their drug consumption over the previous month. The TLFB was administered at baseline and at 12 monthly follow-up assessments to establish a pretreatment rate of use and 365 continuous days of data on daily drug use after randomization.
|
|
Key Findings
|
The median number of self-reported drug-use days at 12 months was significantly lower in the BSFT® condition than in the treatment-as-usual condition (p < .02).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 2: Treatment engagement |
|
Description of Measures
|
On a monthly basis, using participants' clinical charts and billing data, therapists reported the number of therapy sessions of any type delivered since the last interview. Participants were considered to be engaged if they participated in two or more sessions.
|
|
Key Findings
|
Participants in the BSFT® condition were significantly more engaged in treatment than those who received treatment-as-usual (p < .001). Adolescents receiving treatment-as-usual were 2.5 times more likely to fail to engage into therapy (rate of failure = 26.8%) than adolescents receiving the BSFT treatment (rate of failure = 11.4%).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 3: Treatment retention |
|
Description of Measures
|
On a monthly basis, using participants' clinical charts and billing data, therapists reported the number of therapy sessions of any type delivered since the last interview. Participants were classified as retained if they attended more than seven sessions.
|
|
Key Findings
|
Participants in the BSFT® condition had lower rates of failure to retain in treatment than those who received treatment-as-usual (p < .02). Adolescents receiving treatment-as-usual were 1.4 times more likely to fail to retain in treatment for at least eight sessions (rate of failure = 56.6%) than adolescents in the BSFT® treatment condition (rate of failure = 40%).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 4: Family functioning |
|
Description of Measures
|
Family functioning was assessed using a composite scale created from two separate scales. The Parenting Practices Questionnaire was completed by adolescents and their parents to identify positive and negative parenting behaviors in four domains: (a) positive parenting, (b) discipline effectiveness, (c) avoidance of discipline, and (d) monitoring. Discipline effectiveness and avoidance of discipline were assessed only of the parents. The Family Environmental Scale was administered to adolescents and their parents to measure cohesion and conflict.
Individual subscales were converted to
|
|
Key Findings
|
There were significant differences in the trajectories of parent-reported family functioning (p < .011), suggesting that BSFT was significantly more effective than the control condition in improving family processes. This pattern held true for the parenting practices (p < .023) and family environment subcomponents (p < .033).
Adolescents in both conditions reported significant improvements over time in family functioning; however, no statistically significant differences between treatment conditions were observed.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
|
78.5% Male
21.5% Female
|
44.4% Hispanic or Latino
30.8% White
22.9% Black or African American
1% American Indian or Alaska Native
0.4% Asian
0.4% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Drug use
|
3.5
|
3.5
|
3.5
|
3.0
|
3.0
|
4.0
|
3.4
|
|
2: Treatment engagement
|
3.5
|
3.5
|
3.5
|
3.0
|
3.0
|
4.0
|
3.4
|
|
3: Treatment retention
|
3.5
|
3.5
|
3.5
|
3.0
|
3.0
|
4.0
|
3.4
|
|
4: Family functioning
|
3.5
|
3.5
|
3.5
|
3.0
|
3.0
|
4.0
|
3.4
|
Study Strengths The outcome instruments were well-established with acceptable levels of reliability and validity. The study was implemented with considerable attention to assuring adherence to the BSFT protocol, including manuals, training, supervision, and pilot cases that were worked on prior to beginning the study. Random sessions were assessed for adherence, using an instrument with known psychometric properties by independent raters with high interrater reliability across the four principal domains. No differences were found between the two conditions on any of the key demographic variables or baseline levels of the outcomes. Factorial validity was ensured for the family functioning measure across the three major groups, African American, Hispanics and Whites. Families were randomly assigned within each provider agency to study condition using an urn randomization procedure, which was intended to balance participants across conditions on ethnicity/race and level of drug use at baseline. Within each provider agency, therapist randomization was carried out within therapist pairs that were balanced, insofar as feasible, in regard to academic degrees and years of clinical experience. The participants in the treatment-as-usual condition received standard agency services in reducing adolescent drug use, including individual and/or group therapy, parent-training groups, nonmanualized family therapy, and case management. All agencies were expected to provide at least one intervention session per week, which is at least as many sessions as participants in the intervention condition. The analytic strategy was very thorough and appropriate, and included random effects for both sites and therapists, nested within site, to account for these two levels of nesting. These models are robust to data that are missing at random.
Study Weaknesses The researchers acknowledged that some therapists had little or no experience in family therapy and occasionally no experience with adolescents. This, and also allowing BSFT therapists to have multiple roles beyond BSFT treatment, may have had a negative impact on the effectiveness of the intervention. Although attrition is expected in longitudinal research, and rates of attrition were comparable across study conditions, the level of attrition was notably high, particularly among African American youths. In addition, 25 percent of the clinicians left the study while it was in process and it is unclear how this impacted the study findings. The researchers mentioned that adolescent drug use was likely depressed in the period preceding treatment and during the intervention itself, which created a floor effect that permitted only the examination of prevention of relapse or escalation of drug use.
|
|
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Santisteban, D. A., Szapocznik, J., Perez-Vidal, A., Kurtines, W. M., Murray, E. J., & LaPerriere, A. Efficacy of intervention for engaging youth and families into treatment and some variables that may contribute to differential effectiveness. Journal of Family Psychology, 10, 35-44. Study 2Santisteban, D. A., Coatsworth, J. D., Perez-Vidal, A., Kurtines, W. M., Schwartz, S., LaPerriere, A., et al. The efficacy of Brief Strategic Family Therapy in modifying Hispanic adolescent behavior problems and substance use. Journal of Family Psychology, 17(1), 121-133. Study 3Santisteban, D. A., Coatsworth, J. D., Perez-Vidal, A., Mitrani, V., Jean-Gilles, M., & Szapocznik, J. Brief Structural/Strategic Family Therapy with African American and Hispanic high-risk youth. Journal of Community Psychology, 25(5), 453-471. Study 4Coatsworth, J. D., Santisteban, D. A., McBride, C. K., & Szapocznik, J. Brief Strategic Family Therapy versus community control: Engagement, retention, and an exploration of the moderating role of adolescent symptom severity. Family Process, 40(3), 313-332. Study 5Szapocznik, J., Rio, A., Murray, E., Cohen, R., Scopetta, M., Rivas-Vazquez, A., et al. Structural family versus psychodynamic child therapy for problematic Hispanic boys. Journal of Consulting and Clinical Psychology, 57(5), 571-578. Study 6Szapocznik, J., Perez-Vidal, A., Brickman, A. L., Foote, F. H., Santisteban, D., Hervis, O. E., et al. Engaging adolescent drug abusers and their families in treatment: A strategic structural systems approach. Journal of Counseling and Clinical Psychology, 56(4), 552-557. Study 7Nickel, M., Luley, J., Krawczyk, J., Nickel, C., Widermann, C., Lahmann, C., et al. Bullying girls--Changes after Brief Strategic Family Therapy: A randomized, prospective, controlled trial with one-year follow-up. Psychotherapy and Psychosomatics, 75(1), 47-55.
Outcomes
| Outcome 1: Engagement in therapy |
|
Description of Measures
|
Engagement was defined as attendance by the adolescent and at least one adult family member at the intake session and one therapy session within a 4-week period following initial contact.
|
|
Key Findings
|
In one study, families who received BSFT were significantly more engaged in therapy than families in the comparison groups, who received standard family therapy or standard group therapy (p < .006). Two other studies resulted in similar findings, with families receiving BSFT being significantly more engaged in therapy than control families receiving individual and family therapy (p < .05) and control families receiving standard family therapy (p < .0001), respectively.
|
|
Studies Measuring Outcome
|
Study 1, Study 4, Study 6
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 2: Conduct problems |
|
Description of Measures
|
Conduct problems were measured using the Conduct Disorder subscale of the Revised Behavior Problem Checklist (RBPC). The subscale consists of 22 items that focus on physical aggression, difficulty controlling anger, open disobedience, defiance, and oppositionality. For each adolescent, an informed observer, such as a parent or guardian, rated the severity of each behavior on a 3-point scale (0 = no problem, 1 = mild problem, 2 = severe problem).
|
|
Key Findings
|
In one study, adolescents who participated in BSFT showed a significantly greater reduction in conduct problems than adolescents in the comparison condition, who received a participatory-learning group intervention (p < .01). In another study, adolescents receiving BSFT showed a significant reduction in conduct problems (p < .001).
|
|
Studies Measuring Outcome
|
Study 2, Study 3
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 3: Socialized aggression (delinquency in the company of peers) |
|
Description of Measures
|
The Socialized Aggression subscale of the RBPC was used to assess adolescents' delinquent behaviors in the company of peers. The subscale consists of 17 items that focus on conduct-disordered behaviors in the company of others, including substance use, truancy from school, gang membership, stealing, and lying. For each adolescent, an informed observer, such as a parent or guardian, rated the severity of each behavior on a 3-point scale (0 = no problem, 1 = mild problem, 2 = severe problem).
|
|
Key Findings
|
In one study, adolescents who participated in BSFT showed a significantly greater reduction in socialized aggression than adolescents in the comparison condition, who received a participatory-learning group intervention (p < .01). In another study, adolescents receiving BSFT showed a significant reduction in socialized aggression (p < .001).
|
|
Studies Measuring Outcome
|
Study 2, Study 3
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 4: Substance use |
|
Description of Measures
|
Alcohol and other drug use was measured using the following instruments:
- Items from the Addiction Severity Index (ASI) measuring the number of days respondents used various drugs during the month prior to assessment
- The Alcohol and Drug Use scale of the Adolescent Drug Abuse Diagnosis (ADAD), a 150-item structured interview instrument with a 10-point severity rating for each of nine life problem areas
- The Drug Use subscale of the Adolescents' Risk-taking Behavior Scale (ARBS), with scores ranging from 0 to 4 on each scale, 4 indicating the most marked risk-taking behavior
|
|
Key Findings
|
In one study, adolescents who participated in BSFT showed significantly greater reductions in marijuana use than adolescents in the comparison group, who received a participatory-learning group intervention (p < .05). In another study, adolescents receiving BSFT showed a significant reduction in overall substance use (p < .05). In a third study, adolescent girls who participated in BSFT showed significantly greater reductions in substance use at posttest (p < .001) and at the 1-year follow-up (p < .05) than adolescent girls in the comparison group, who received an intervention consisting of structural, detailed question sessions.
|
|
Studies Measuring Outcome
|
Study 2, Study 3, Study 7
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 5: Family functioning |
|
Description of Measures
|
Family functioning was measured using the following instruments:
- The adolescent- and parent-reported Cohesion and Conflict scales from the Family Environment Scale (FES). The Cohesion scale measures the extent to which the adolescent and parent view the family as harmonious and close. The Conflict scale measures the extent to which the adolescent and parent view the family as characterized by frequency of quarrels and disagreements.
- The General Scale of the Family Assessment Measure, which consists of 50 items focusing on the family as a system and provides an overall score of family functioning, rated by any member of the family.
- The Structural Family Systems Rating (SFSR), a measure of family interactions as reported by an observer (i.e., a clinical psychologist or other trained staff). It consists of five scales: structure (the family's organizational system and flow of communication), resonance (closeness, distance, and boundaries between family members), developmental stage (age appropriateness of family members' behaviors), identified patienthood (the extent to which a family member, usually the adolescent, is labeled as the family's "problem"), and conflict resolution (the extent to which the family is able to resolve differences of opinion).
|
|
Key Findings
|
In one study, adolescents who participated in BSFT reported significantly better family functioning on the FES Cohesion scale than adolescents in the comparison group, who received a participatory-learning group intervention (p < .05). Families in the BSFT group also showed significantly greater improvement on overall SFSR scores than families in the comparison group (p < .05).
In another study, reports by both parents and adolescents who received BSFT showed significant improvements in family functioning on the General Scale of the Family Assessment Measure (p < .001 for both parents and adolescents). The effect sizes were medium for the parent report (Cohen's d = 0.58) and small for the adolescent report (Cohen's d = 0.42).
In a third study, families receiving BSFT demonstrated significantly better family functioning on the SFSR at the 1-year follow-up than families assigned to either an individual psychodynamic child therapy group or a recreational control condition (p < .02).
|
|
Studies Measuring Outcome
|
Study 2, Study 3, Study 5
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
13-17 (Adolescent)
|
70% Male
30% Female
|
100% Hispanic or Latino
|
|
Study 2
|
6-12 (Childhood)
13-17 (Adolescent)
|
75% Male
25% Female
|
100% Hispanic or Latino
|
|
Study 3
|
6-12 (Childhood)
13-17 (Adolescent)
|
66.4% Male
33.6% Female
|
84.4% Hispanic or Latino
15.6% Black or African American
|
|
Study 4
|
6-12 (Childhood)
13-17 (Adolescent)
|
75% Male
25% Female
|
76% Hispanic or Latino
24% Black or African American
|
|
Study 5
|
6-12 (Childhood)
|
100% Male
|
100% Hispanic or Latino
|
|
Study 6
|
6-12 (Childhood)
13-17 (Adolescent)
|
67% Male
33% Female
|
100% Hispanic or Latino
|
|
Study 7
|
13-17 (Adolescent)
|
100% Female
|
Data not reported/available
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Engagement in therapy
|
3.5
|
3.5
|
3.5
|
3.5
|
3.0
|
3.5
|
3.4
|
|
2: Conduct problems
|
4.0
|
3.8
|
3.3
|
2.8
|
2.8
|
3.8
|
3.4
|
|
3: Socialized aggression (delinquency in the company of peers)
|
4.0
|
3.8
|
3.3
|
2.8
|
2.8
|
3.8
|
3.4
|
|
4: Substance use
|
3.3
|
2.8
|
3.0
|
3.3
|
2.5
|
3.4
|
3.0
|
|
5: Family functioning
|
3.5
|
3.5
|
3.3
|
2.8
|
2.5
|
3.5
|
3.2
|
Study Strengths Most of the studies were well designed and involved random assignment of subjects to the study conditions. Attrition was minimal, and there were few compelling confounding variables that could reasonably account for the overall positive pattern of findings. For most of the studies, the investigators sufficiently addressed the psychometric properties of the measures, the analyses, and the study limitations.
Study Weaknesses Results might have been more compelling if the authors had used an intent-to-treat model and more sophisticated methods to document engagement strategies. One of the studies used a weak design. Another study did not provide enough detail regarding the psychometric properties of the instruments or the fidelity of implementation.
|
|
Readiness for Dissemination
|
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
SOP for Engagement
Curriculum
Implementation Paper
Progress Note V8
Safety Plan Final
Scope of Work V11
Site Readiness Plan V8
Weekly On-Site Supervisor Meeting Checklist V2
Workshop PowerPoint
Therapist Eligibility V7
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.3
|
3.0
|
3.5
|
3.3
|
Dissemination Strengths The developer presents a sound theoretical basis for the program/therapy, complete with a research paper and PowerPoint presentation that explains the theoretical underpinnings of the therapeutic approach.
The clinical manual presents a clear theoretical and conceptual base for understanding the clinical components and sequencing of intervention phases. Guidance is provided for using the intervention with diverse and complex family systems.
The BSFT® standard operating procedures (SOP) and scope of work (SOW) are succinct but simultaneously contain enough information to provide a sense of the program, and what is needed to implement it. The link is accessible and active. The site is informative, well organized, and contains related training. It is easy to navigate.
The materials include information and qualitative measures aimed at both the supervisor (i.e., the SOP, therapist eligibility, and weekly meeting checklist) and the therapist (i.e., progress notes and safety plan), which support quality assurance of the intervention.
QA materials adequately address the four main competencies of BSFT® which include 1. joining, 2. tracking/diagnostic enactments, 3. reframing, and 4. restructuring skills.
Dissemination Weaknesses The delivery of the training, using the curriculum provided, would be challenging without the actual slides referenced in the curriculum. No significant document or tool is found in the packet presented.
|
|
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
BSFT for Adolescents--Adherence Form (ADH), Version 2.15
BSFT for Adolescents--Clinical Supervision Checklist (CSC), Version 2.15
BSFT for Adolescents--Overall Supervision Evaluation Checklist (OSC), Version 2.15
BSFT for Adolescents--Videotape Certification Rating Checklist (VRC), Version 2.15
Robbins, M. S., Perez, G. A., Hervis, O., & Santisteban, D. Overall supervision evaluation checklist: Procedure and rating manual. Miami, FL: University of Miami Center for Family Studies.
Robbins, M. S., Perez, G. A., Hervis, O., & Santisteban, D. Weekly clinical supervision checklist: Procedure and rating manual. Miami, FL: University of Miami Center for Family Studies.
Robbins, M. S., Perez, G. A., Mayorga, C. C., Hervis, O., & Santisteban, D. BSFT adherence checklist: Procedures and rating manual. Miami, FL: University of Miami Center for Family Studies.
Robbins, M. S., Perez, G. A., Mayorga, C. C., Hervis, O., & Santisteban, D. Videotape certification rating checklist: Procedure and rating manual. Miami, FL: University of Miami Center for Family Studies.
Szapocznik, J., & Hervis, O. E.. Brief Strategic Family Therapy training manual. Miami, FL: University of Miami Center for Family Studies.
Szapocznik, J., Hervis, O. E., & Schwartz, S. Brief Strategic Family Therapy for adolescent drug abuse (NIDA Therapy Manuals for Drug Addiction, Manual 5, NIH Publication No. 03-4751). Rockville, MD: National Institute on Drug Abuse.
University of Miami Center for Family Studies.. Brief Strategic Family Therapy: An empirically validated therapy [PowerPoint slides]. Miami, FL: Author.
University of Miami Center for Family Studies.. BSFT curriculum. Miami, FL: Author.
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.3
|
3.0
|
3.5
|
3.3
|
Dissemination Strengths The clinical manual presents a clear theoretical and conceptual base for understanding the clinical components and sequencing of intervention phases. Guidance is provided for using the intervention with diverse and complex family systems. A detailed training curriculum is provided and is supplemented by ongoing weekly clinical supervision to support implementation. Four highly detailed instruments with manuals are available to document clinician competency.
Dissemination Weaknesses The intensive supervision and clinical consultation components necessitate additional guidance for assessing and bolstering organizational readiness for implementation. The required level and sequence of training is unclear. The training manual is very dense, and its content and sequencing do not clearly correspond with training slides. It is unclear who administers some quality assurance instruments. No guidance is provided to implementers for clinical outcomes measurement.
|
|