Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Outcomes
| Outcome 1: Substance use |
|
Description of Measures
|
Use of opiates and cocaine was measured by urinalysis and self-report. Urine samples collected from every participant twice per month on random days and times were analyzed for the presence of cocaine and opioid metabolites, including all synthetic opioids except for methadone, using the enzyme multiplication immunoassay technique (EMIT). Positive urinalysis results were compared at intake, 2 and 3 months after intake (averaged), 4, 5, and 6 months after intake (averaged), and 12 months after treatment discharge. Clients reported their past-month use of heroin, cocaine, and speedball (heroin mixed with cocaine) on an 8- or 9-point scale ranging from "none" to "> 4 times per day." Self-reported drug use was compared at intake, 2 and 3 months after intake (averaged), 6 months after intake, and 12 months after treatment discharge.
|
|
Key Findings
|
In a randomized clinical trial (RCT), opiate-abusing clients admitted to one of three outpatient methadone maintenance clinics participating in the DATAR project were assigned either to node-link mapping-enhanced counseling or to standard counseling. Study findings included:
- Although the number of cocaine- or opioid-positive urine samples decreased for all clients during the first 6 months of treatment, regardless of condition assignment (p < .0001), mapping clients had fewer positive urine samples overall than clients receiving standard counseling (66% vs. 75%, p < .01).
- Although clients assigned to mapping-enhanced counseling had more opiate- and cocaine-positive urine samples at intake compared with clients in standard counseling, they had fewer positive urine samples (p < .023) and reported significantly less speedball use (p < .01) during the first 6 months of treatment.
- While Mexican Americans assigned to mapping-enhanced counseling had more cocaine-positive urine samples at intake relative to Mexican Americans in standard counseling, they had comparatively fewer positive samples by the 6-month follow-up (p < .045). The same pattern held for African Americans, but this finding failed to reach statistical significance.
- While clients assigned to mapping-enhanced counseling had more cocaine-positive urine samples at intake relative to clients in standard counseling (59% vs. 49%), they had fewer positive samples by the 6-month follow-up (37% vs. 47%, p < .01). These differences were largely due to a higher percentage of weekly or daily ("high") cocaine users with positive urine samples in the mapping condition at intake and a lower percentage of these high cocaine users with positive urine samples in the mapping condition at the 6-month follow-up (89% and 50% for mapping-enhanced counseling vs. 69% and 68% for standard counseling, p = .049).
- Clients who were high cocaine users at intake reported more frequent use of cocaine (p < .0001) and speedball (p < .0001) and less frequent use of heroin (p < .001) across the first 6 months of treatment compared with clients reporting low or no use at intake, regardless of condition assignment. However, reported cocaine and speedball use declined more among the high cocaine group than among other clients (p < .0001).
- Among clients receiving less than 6 months of treatment, those assigned to mapping-enhanced counseling had fewer opioid-positive urine samples at the 12-month follow-up than those in standard counseling (44% vs. 63%, p < .048).
- Clients who remained in treatment at least 6 months, regardless of condition assignment, had fewer cocaine-positive urine samples at the 12-month follow-up compared with clients in treatment less than 6 months (33% vs. 48%, p < .046).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 2: Counseling session attendance |
|
Description of Measures
|
In one study, counseling session attendance was measured as the total number of group and individual counseling sessions attended by each client and the number of sessions scheduled but missed during the first 6 months of treatment. Data for months 2 and 3 were averaged, as were data for months 4, 5, and 6.
In another study, session attendance was calculated using monthly tracking forms. The number of sessions attended during the preceding 30 days was divided by the total number of sessions. Averages were calculated for the first 6 months and last 6 months of treatment.
|
|
Key Findings
|
In an RCT, opiate-abusing clients admitted to one of three outpatient methadone maintenance clinics participating in the DATAR project were assigned either to node-link mapping-enhanced counseling (active intervention) or to standard counseling (control). This study found that:
- Clients assigned to mapping-enhanced counseling missed fewer scheduled sessions than standard counseling clients in the first 6 months of treatment (p < .001). However, counseling session attendance decreased over time for all clients, regardless of condition assignment (p < .0001).
- In general, White clients attended more counseling sessions than African American clients and missed fewer scheduled sessions than either African Americans or Mexican Americans in the first 6 months of treatment (p < .015).
- African American clients assigned to mapping-enhanced counseling attended more and missed fewer scheduled counseling sessions in the first 6 months of treatment compared with African Americans assigned to standard counseling (p < .05).
Another RCT compared the effects of three types of individual drug abuse counseling--mapping-enhanced using free-format maps (f-maps), mapping-enhanced using both free-format and guide-maps (f/g-maps), and standard counseling alone (control)--over 6 and 12 months of treatment in an outpatient methadone clinic. For both experimental conditions, counselors used mapping at their own discretion, according to their clinical opinions. Among the findings of this study:
- Clients with high levels of attention-deficit/hyperactivity disorder (ADHD) behaviors at baseline who were assigned to f/g-map counseling attended more sessions than those who were assigned to either f-map or standard counseling (2.4 vs. 1.9 and 1.9, respectively, p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 3: Client rapport, motivation, and self-confidence |
|
Description of Measures
|
Client rapport, motivation, and self-confidence were rated by counselors using a 24-item instrument originally developed as part of DATAR-1. Each item was rated on a 5-point scale ranging from 0 (never) to 4 (almost always). Client rapport included items such as easy to talk to, warm and caring, honest, and sincere. Motivation included items such as dependable, well organized, and cooperative. Self-confidence included items such as self-confident, persuasive, motivated, and assertive. Counselors rated each client at 1, 2, 3, and 6 months after admission to treatment.
|
|
Key Findings
|
In an RCT, opiate-abusing clients admitted to one of three outpatient methadone maintenance clinics participating in the DATAR project were assigned either to node-link mapping-enhanced counseling (active intervention) or to standard counseling (control). Among the findings from this study:
- Although counselors generally rated clients in both conditions progressively higher on rapport and self-confidence over the first 6 months of treatment (p < .0003), counselor ratings of client motivation declined during the same period (p < .002).
- Overall, mapping-enhanced counseling clients were rated higher than standard counseling clients on rapport (p < .05), self-confidence (p < .001), and motivation (p < .05) across the first 6 months of treatment. However, counselor ratings differed significantly across the three ethnic groups in the study. Specifically:
- African Americans who received mapping-enhanced counseling were rated higher in self-confidence in all three follow-up assessments in the first 6 months of treatment compared with African Americans in standard counseling (p < .0001). They also had higher ratings of motivation in months 2 and 3 (p < .002).
- Mexican Americans who received mapping-enhanced counseling had higher counselor ratings of motivation in months 4 through 6 compared with Mexican Americans in standard counseling (p < .002).
- White clients who received the intervention had higher counselor ratings of self-confidence in months 2 and 3 compared with Whites in standard counseling (p < .002).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.9
(0.0-4.0 scale)
|
| Outcome 4: HIV risk behavior |
|
Description of Measures
|
HIV risk behavior was measured as the self-reported occurrence or absence of (1) drug injections with a needle and (2) drug injections with a needle previously used by others (i.e., "dirty needle") during the month prior to the 12-month postdischarge follow-up.
|
|
Key Findings
|
In an RCT, opiate-abusing clients admitted to one of three outpatient methadone maintenance clinics participating in the DATAR project were assigned either to node-link mapping-enhanced counseling (active intervention) or to standard counseling (control). Among the findings from this study:
- Clients with less than 6 months in treatment were more likely to have used a dirty needle than those who remained in treatment longer than 6 months (30% vs. 16%, p < .025), regardless of condition assignment.
- Clients who received mapping-enhanced counseling were less likely than those in standard counseling to have used a dirty needle (17% vs. 29%, p < .054).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 5: Criminal behavior |
|
Description of Measures
|
Criminal behavior was measured as the self-reported presence or absence of arrests, jail time, and illegal activities in the month prior to the 12-month postdischarge follow-up.
|
|
Key Findings
|
In an RCT, opiate-abusing clients admitted to one of three outpatient methadone maintenance clinics participating in the DATAR project were assigned either to node-link mapping-enhanced counseling (active intervention) or to standard counseling (control). Among the findings from this study:
- Clients assigned to mapping-enhanced counseling were less likely than those in standard counseling to report illegal activity (15% vs. 30%, p < .024), an arrest (3% vs. 22%, p < .002), or incarceration (6% vs. 23%, p < .004) during the month prior to the 12-month follow-up.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 6: Participation in group meetings |
|
Description of Measures
|
Active participation in group meetings was self-rated and rated by counselors using a 7-point scale that ranged from 1 (disagree strongly) to 7 (agree strongly). Ratings occurred halfway through treatment (approximately 8 weeks) and toward the end of treatment (approximately 14 weeks).
|
|
Key Findings
|
An RCT compared mapping-enhanced and standard counseling among offenders mandated to attend a 4-month residential criminal justice program followed by a 12-week aftercare program for probation or parole violations related to substance abuse. As part of the CETOP project, consenting probationers were randomly assigned to one of two types of residential MTCs--one that exclusively used mapping-enhanced group counseling, and one that used only standard group counseling. Except for the counseling approach, all communities were identical, with traditional community meetings and education classes. Findings from this study included:
- High motivation ("desire for help in dealing with drug use") self-ratings at intake were associated with greater participation in group meetings, both self- and counselor-rated (p < .05).
- Clients with high motivation at intake who were assigned to mapping communities rated themselves as more active participants in group meetings at the end-of-treatment follow-up (p < .02) and were similarly rated by counselors at the midterm follow-up (p < .001).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 7: Perceived treatment progress, affect, and engagement |
|
Description of Measures
|
Clients rated their individual treatment progress, affective responses to treatment, and level of treatment engagement using the TCU Self-Rating Form. The form contains a set of brief scales that assess psychosocial and motivational barriers to discontinuing substance abuse. Therapeutic progress (e.g., "You have made progress with your…") was rated in several areas including emotional/psychological problems, drug/alcohol problems, and program goals. The 7-point rating scale ranged from 1 (disagree strongly) to 7 (agree strongly). The same scale was used to rate affective responses to treatment (confident, excited, valuable, pleased) and treatment engagement ("You have decided to change," "You have decided to work on your personal problems," "You have been working hard to change"). Ratings occurred halfway through treatment (approximately 8 weeks) and toward the end of treatment (approximately 14 weeks).
|
|
Key Findings
|
An RCT compared mapping-enhanced and standard counseling among offenders mandated to attend a 4-month residential criminal justice program followed by a 12-week aftercare program for probation or parole violations related to substance abuse. As part of the CETOP project, consenting probationers were randomly assigned to one of two types of residential MTCs--one that exclusively used mapping-enhanced group counseling, and one that used only standard group counseling. Except for the counseling approach, all communities were identical, with traditional community meetings and education classes. Findings from this study included:
- While clients in both types of communities rated themselves higher on treatment progress over time (p < .001), those in mapping-enhanced counseling reported higher ratings than those in standard group counseling at both measurement points (p < .01).
- High motivation ("desire for help in dealing with drug use") self-ratings at intake were associated with reports of greater treatment progress overall (p < .05), regardless of condition assignment.
- Clients in both types of communities reported more positive affect toward treatment over time (p < .01), with clients assigned to mapping-enhanced counseling communities reporting greater positive affect at the end-of-treatment follow-up than those assigned to standard counseling (p < .0001).
- High motivation self-ratings at intake were associated with reports of greater treatment engagement overall (p < .05), regardless of condition assignment.
- Clients in mapping-enhanced counseling communities had higher reported levels of treatment engagement at the end-of-treatment follow-up compared with clients receiving standard counseling (p < .01).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.9
(0.0-4.0 scale)
|
| Outcome 8: Treatment retention |
|
Description of Measures
|
Treatment retention was measured as (1) the number of clients in each counseling condition who entered treatment, (2) the number who were still in treatment 6 months after intake, and (3) the number who were still in treatment 12 months after intake. A monthly tracking report was used to compute the number of clients still in each condition at 6 and 12 months.
|
|
Key Findings
|
An RCT compared the effects of three types of individual drug abuse counseling--mapping-enhanced using free-format (f-maps), mapping-enhanced using both free-format and guide-maps (f/g-maps), and standard counseling alone (control)--over 6 and 12 months of treatment in an outpatient methadone clinic. For both experimental conditions, counselors used mapping at their own discretion, according to their clinical opinions. Findings of this study included:
- The number of clients still in treatment 12 months after intake was higher among those assigned to mapping-enhanced counseling than among those assigned to either standard or f/g-map counseling (35 vs. 24 and 31 clients, respectively, p < .05).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
65% Male
35% Female
|
38% White
36% Hispanic or Latino
22% Black or African American
4% Race/ethnicity unspecified
|
|
Study 2
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
73.1% Male
26.9% Female
|
56.2% White
35.4% Black or African American
6.3% Hispanic or Latino
1.6% American Indian or Alaska Native
0.5% Race/ethnicity unspecified
|
|
Study 3
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
67% Male
33% Female
|
60% Hispanic or Latino
21% White
19% Black or African American
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Substance use
|
3.3
|
3.0
|
3.0
|
3.0
|
2.5
|
3.5
|
3.0
|
|
2: Counseling session attendance
|
3.3
|
3.4
|
3.3
|
3.0
|
2.5
|
3.5
|
3.1
|
|
3: Client rapport, motivation, and self-confidence
|
3.0
|
2.5
|
3.0
|
3.0
|
2.5
|
3.5
|
2.9
|
|
4: HIV risk behavior
|
2.5
|
2.5
|
3.0
|
3.0
|
2.5
|
3.5
|
2.8
|
|
5: Criminal behavior
|
2.5
|
2.5
|
3.0
|
3.0
|
2.5
|
3.5
|
2.8
|
|
6: Participation in group meetings
|
2.5
|
2.5
|
2.5
|
3.0
|
2.5
|
3.5
|
2.8
|
|
7: Perceived treatment progress, affect, and engagement
|
3.0
|
3.0
|
2.5
|
3.0
|
2.5
|
3.5
|
2.9
|
|
8: Treatment retention
|
3.0
|
3.0
|
3.0
|
3.0
|
2.5
|
3.5
|
3.0
|
Study Strengths Random, monthly urinalysis for opiate and/or cocaine use using state-of-the-science assay technology is a reasonable supplement to self-report. Session attendance is a strong proxy measure for treatment dose and participation. Treatment retention is a widely used measure of treatment dose and a good predictor of posttreatment outcome. The research team established the psychometric properties of the measures used in all three studies. Collecting information from both clients and counselors is an excellent way of obtaining multiple perspectives on treatment effectiveness. Randomization of both intervention and counselor assignment is a strong experimental design element. There was a strong emphasis on treatment fidelity across all three studies, with consistent delivery of the intervention through manual-driven training, ongoing monitoring, and on-site visits. An on-site coordinator ensured the timely administration of all outcome measures. A team of trained interviewers conducted follow-up client and counselor assessments, and trained editorial personnel checked forms for accuracy and completeness.
Study Weaknesses Missing data across studies were generally handled with listwise deletion, a statistical technique likely to create bias. Follow-up selection bias was a significant issue in two of the three studies. Specifically, in one study, a sizable percentage of the original sample were in prison at the 1-year follow-up and were not interviewed; additional cases were dropped due to missing urine samples; only a subset of the participants received counselor ratings; and clients had to have been in treatment for a minimum of 6 months to be included in the primary analyses. In another study, clients were offered up to 12 months of no-fee methadone treatment in return for study participation.
|
|