•  

Intervention Summary

OQ-Analyst

The OQ-Analyst (OQ-A) is a computer-based feedback and progress tracking system designed to help increase psychotherapy treatment effectiveness. OQ-A assesses the attainment of expected progress during therapy and provides feedback to therapists on whether patients are staying on track toward positive treatment outcomes. This information may be shared with the patient at the therapist's discretion, either verbally or using a computer-generated patient report. The OQ-A system also supports clinical decisionmaking via the included clinical support tool (CST). The CST provides information on the quality of the therapeutic alliance, the patient's motivation, social supports, and negative life events, and the possible need for medication referral.

Data for the OQ-A are collected from the patient immediately before each treatment session using the Outcome Questionnaire 45 (OQ-45), a self-report instrument that takes about 5-7 minutes to complete. The OQ-45 can be administered on paper or electronically using a computer terminal or handheld PDA (personal digital assistant). The OQ-A software automatically scores the responses and generates a report for use by the therapist before or during the session. The report tells the therapist whether the patient is improving, worsening, or showing no change and recommends general next steps such as continuing the current treatment course, considering discharge, reviewing the treatment plan, or providing intensive and immediate intervention.

The OQ-A is designed to detect treatment effectiveness regardless of treatment modality, diagnosis, or discipline of the treating professional. It is suitable for use in inpatient and outpatient settings.

Descriptive Information

Areas of Interest Mental health treatment
Substance use disorder treatment
Outcomes
1: Psychosocial dysfunction
2: Substance use


1: Psychological dysfunction
Outcome Categories Alcohol
Drugs
Mental health
Social functioning
Ages 18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
Genders Male
Female
Races/Ethnicities American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other Pacific Islander
White
Race/ethnicity unspecified
Settings Outpatient
Geographic Locations Urban
Suburban
Implementation History The OQ-A was first used by a managed behavioral health care company. Hundreds of sites have used the OQ-A system with more than 300,000 clients. Outside the United States, it has been used in Australia, Canada, England, Germany, the Netherlands, Norway, the Republic of Singapore, Scotland, Sweden, and Switzerland.
NIH Funding/CER Studies Partially/fully funded by National Institutes of Health: No
Evaluated in comparative effectiveness research studies: Yes
Adaptations The OQ-A system has been translated into Arabic, Chinese, Dutch, French, German, Hebrew, Norwegian, and Spanish.
Adverse Effects No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories IOM prevention categories are not applicable.

Quality of Research

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Study 1

Slade, K., Lambert, M. J., Harmon, S. C., Smart, D. W., & Bailey, R. Improving psychotherapy outcome: The use of immediate electronic feedback and revised clinical support tools. Clinical Psychology and Psychotherapy, 15(5), 287-303.  

Study 2

Crits-Christoph, P., Ring-Kurtz, S., Hamilton, J., Lambert, M. J., Gallop, R., McClure, B., et al. Preliminary study of the effects of individual patient-level feedback in outpatient substance abuse treatment programs. Journal of Substance Abuse Treatment, 42(3), 301-309.  

Study 3

Simon, W., Lambert, M. J., Busath, G., Vazquez, A., Berkeljon, A., Hyer, K., et al. Effects of providing patient progress feedback and clinical support tools to psychotherapists in an inpatient eating disorders treatment program: A randomized controlled study. Psychotherapy Research, 23(3), 287-300.  

Supplementary Materials

Lambert, M. J. Outcome Questionnaire: Annotated references.

Shimokawa, K., Lambert, M. J., & Smart, D. W. Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298-311.  

Spielmans, G. I., Masters, K. S., & Lambert, M. J. A comparison of rational versus empirical methods in the prediction of psychotherapy outcome. Clinical Psychology and Psychotherapy, 13, 202-214.

Outcomes

Outcome 1: Psychosocial dysfunction
Description of Measures Psychological dysfunction was assessed using the OQ-45, a 45-item self-report instrument. The OQ-45 is administered immediately before each treatment session for up to 12 sessions or 6 months. The instrument monitors client progress in three dimensions: (a) subjective discomfort (e.g., anxiety and depression: ''I feel blue''), (b) interpersonal relationships (e.g., ''I feel lonely''), and (c) social role performance (e.g., ''I have too many disagreements at work/school''). The OQ-45 is scored using a 5-point scale (0 = never, 1 = rarely, 2 = sometimes, 3 = frequently, 4 = almost always), yielding a total score between 0 and 180. The total score serves as a global assessment of client psychosocial functioning, with higher scores reflecting more severe distress.
Key Findings In one study, patients with mental health disorders seeking treatment at a large university counseling center were randomly assigned to one of two intervention groups. For one intervention group, feedback reports were provided to both the therapist and the patient; for the other group, feedback reports were provided to the therapist only. For a control condition, the study used an archival sample of patients who had completed OQ-45 assessments at each session without feedback being provided to either the patient or therapist. At each session, mathematical modeling of OQ-45 results was used to determine if the patient's progress in therapy was on track or off track. Off-track patients were subsequently randomly assigned to two conditions, one in which the therapist had access to OQ-45 decision support tools (CST), and one in which the therapist did not have access to CST. Before treatment, there were no statistically significant differences between the three study groups in OQ-45 scores. OQ-45 scores decreased (showing improvement in symptoms) for all groups from pretreatment to posttreatment, but the decrease was significantly greater among the treatment groups than among the no-feedback control group (p < .001). Because the therapist-only and patient–therapist feedback groups did not differ significantly in the change in OQ-45 scores, further analyses were conducted to compare a combined feedback group (therapist-only and patient–therapist) with the no-feedback archival sample and the feedback plus CST group. These analyses showed significant group differences in OQ-45 scores at posttreatment (p < .001). Specifically, the feedback plus CST group had fewer patients with deterioration in psychosocial dysfunction (4.8%) compared with the feedback group (9.5%) and no-feedback archival group (21.3%). In addition, 63.9% of patients in the feedback plus CST group had reliable clinical improvement or recovery from their psychosocial dysfunction, compared with 37.3% of patients in the feedback group and 21% of patients in the no-feedback archival group.

In a second study, patients seeking treatment for alcohol or drug use problems in outpatient substance abuse clinics were randomly assigned to an intervention or control group. All participants were administered the OQ-45 before each treatment session for up to 12 sessions. Counselors for intervention group patients received OQ-45 feedback reports, while counselors for control group patients received no feedback. At each session, mathematical modeling of OQ-45 results was used to determine if the patient's progress in therapy was on track or off track. Patients who were off-track were administered a questionnaire that was used as a second feedback report for counselors. Among off-track patients, from the off-track point to session 12, those in the intervention group showed significant improvement in symptoms of psychosocial dysfunction compared with those in the control group (p = .013).

In a third study, patients seeking outpatient psychotherapy services at a hospital-based outpatient clinic were randomly assigned to an intervention or treatment-as-usual control group. OQ-45 feedback reports were provided to intervention group patients and their therapists; therapists of patients in the intervention group also had access to OQ-A clinical decision support tools (CST). In this study, intervention group patients had two times the improvement in symptoms of psychosocial dysfunction compared with patients who received treatment as usual (p = .04). This finding had a very small effect size (Cohen's d = 0.12).
Studies Measuring Outcome Study 1, Study 2, Study 3
Study Designs Experimental, Quasi-experimental
Quality of Research Rating 3.0 (0.0-4.0 scale)
Outcome 2: Substance use
Description of Measures Substance use was assessed using a modified version of the OQ-45, a 45-item self-report instrument. The OQ-45 is administered immediately before each treatment session for up to 12 sessions or 6 months. The instrument monitors client progress in three dimensions: (a) subjective discomfort (e.g., anxiety and depression: ''I feel blue''), (b) interpersonal relationships (e.g., ''I feel lonely''), and (c) social role performance (e.g., ''I have too many disagreements at work/school''). The primary modification made to the OQ-45 was to add two items measuring the number of days in the past week that the patients used (a) alcohol and (b) drugs. The scores on the individual alcohol and drug use items were used for this outcome.
Key Findings Patients seeking treatment for alcohol or drug use problems in outpatient substance abuse clinics were randomly assigned to an intervention or control group. All participants were administered the OQ-45 before each treatment session. Counselors for intervention group patients received OQ-45 feedback reports, while counselors for control group patients received no feedback. At each session, mathematical modeling of OQ-45 results was used to determine if the patient's progress in therapy was on track or off track. Patients who were off-track were administered a questionnaire that was used as a second feedback report for counselors. Over 12 sessions, off-track patients in the intervention group had significantly greater reductions in alcohol use compared with off-track patients in the control group (p = .023); no significant difference was found for drug use. However, analyses looking only at the time period from the off-track point to session 12 found that intervention group patients had a significantly greater reduction in drug use compared with the control group (p = .04) but not for alcohol use.
Studies Measuring Outcome Study 2
Study Designs Quasi-experimental
Quality of Research Rating 2.4 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study Age Gender Race/Ethnicity
Study 1 26-55 (Adult) 61% Female
39% Male 		86% White
6% Hispanic or Latino
2.5% Race/ethnicity unspecified
2% Asian
1.5% Native Hawaiian or other Pacific Islander
1% American Indian or Alaska Native
1% Black or African American
Study 2 26-55 (Adult) 60% Male
40% Female 		44% Black or African American
36% White
13% Hispanic or Latino
7% Race/ethnicity unspecified
Study 3 26-55 (Adult) 65% Female
35% Male 		92.7% White
2.4% Hispanic or Latino
1.9% Asian
1.9% Black or African American
1.6% Native Hawaiian or other Pacific Islander

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

  1. Reliability of measures
  2. Validity of measures
  3. Intervention fidelity
  4. Missing data and attrition
  5. Potential confounding variables
  6. Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome Reliability
of Measures 		Validity
of Measures 		Fidelity Missing
Data/Attrition 		Confounding
Variables 		Data
Analysis 		Overall
Rating
1: Psychosocial dysfunction 3.4 3.4 2.8 2.4 2.4 3.4 3.0
2: Substance use 2.3 2.3 2.9 2.3 1.9 3.1 2.4

Study Strengths

The version of the OQ-45 instrument that was used to assess psychosocial dysfunction has well-researched and well-documented reliability and validity. Therapists received an orientation and training on the use of the OQ-45 feedback. The first study used the "last observation carried forward" approach for missing data. All three studies used appropriate analyses.

Study Weaknesses

No psychometric information was presented for the items added to the OQ-45 to measure substance use, although the items appear to have face and content validity. No quantitative data were reported on the results of fidelity monitoring. Two of the three studies reviewed did not discuss the impact of attrition and how missing data were addressed in the analyses. All three studies may have been considerably impacted by confounding variables such as therapist use of feedback and the number and spacing of therapy sessions. None of the studies monitored whether therapists used the feedback data from the OQ-45 to alter their treatment approach or to increase the number of sessions they conducted.

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Study 1

Hawkins, E. J., Lambert, M. J., Vermeersch, D. A., Slade, K. L., & Tuttle, K. C.. The therapeutic effects of providing patient progress information to therapists and patients. Psychotherapy Research, 14(3), 308-327.

Study 2

Harmon, S. C., Lambert, M. J., Smart, D. W., Hawkins, E. J., Nielsen, S. L., Slade, K., & Lutz, W. Enhancing outcome for potential treatment failures: Therapist-client feedback and clinical support tools. Psychotherapy Research, 17(4), 379-392.

Supplementary Materials

Lambert, M. J., Whipple, J. L., Bishop, M. J., Vermeersch, D. A., Gray, G. V., & Finch, A. E. Comparison of empirically-derived and rationally-derived methods for identifying patients at risk for treatment failure. Clinical Psychology and Psychotherapy, 9, 149-164.

Vermeersch, D. A., Whipple, J. L., Lambert, M. J., Hawkins, E. J., Burchfield, C. M., & Okiishi, J. C.. Outcome questionnaire: Is it sensitive to changes in counseling center clients? Journal of Counseling Psychology, 51, 38-49.

Outcomes

Outcome 1: Psychological dysfunction
Description of Measures Psychological dysfunction was assessed using the OQ-45, a 45-item self-report instrument designed to measure client progress repeatedly throughout the course of therapy. Client progress is monitored along three dimensions: subjective discomfort, interpersonal relationships, and social role performance. Possible scores range from 0 to 180; higher scores reflect more severe distress. The OQ-45 total score, a global assessment of client functioning, was used in the reviewed studies.
Key Findings In one study, participants were randomly assigned to one of three groups: feedback provided to patients and therapists, feedback provided to therapists only, or no feedback/usual care. Patients' progress in therapy was categorized as on track or not on track based on OQ-A assessments of the trajectory of recovery. This study found that:

  • The patient/therapist feedback group showed the most improvements, while the usual care group showed the least improvements.
  • The difference between the patient/therapist feedback group and the usual care group was significant (p < .05) with a small effect size (eta-squared = 0.04).
  • The difference between the feedback groups (both patient/therapist and therapist-only) and the usual care group was significant (p < .05) with a small effect size (eta-squared = 0.02).
  • The difference between the patient/therapist and therapist-only feedback groups was significant (p < .05) with a small effect size (eta-squared = 0.02).
  • Among not-on-track clients, there were no significant differences between the feedback groups and the usual care group, nor between the two feedback groups.
In another study, participants were randomly assigned to one of two groups: feedback provided to therapists and patients, or feedback provided to therapists only. For a comparison condition, the study used an archival group that received no feedback. Patients' progress in therapy was categorized as on track or not on track based on OQ-A assessments of the trajectory of recovery. This study found that:

  • The feedback groups showed greater improvements than the archival comparison group (p < .001). There were no significant differences between the two feedback groups.
  • Among on-track patients, both feedback groups showed greater improvements than the archival comparison group (p < .001). Similarly, among not-on-track patients, both feedback groups showed greater improvements than the archival comparison group (p < .001).
Studies Measuring Outcome Study 1, Study 2
Study Designs Experimental, Quasi-experimental
Quality of Research Rating 3.3 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study Age Gender Race/Ethnicity
Study 1 18-25 (Young adult)
26-55 (Adult) 		68.2% Female
31.8% Male 		94.5% White
1.5% Black or African American
1.5% Hispanic or Latino
1.5% Native Hawaiian or other Pacific Islander
1% Asian
Study 2 18-25 (Young adult)
26-55 (Adult)
55+ (Older adult) 		64% Female
36% Male 		88% White
5% Hispanic or Latino
2% Native Hawaiian or other Pacific Islander
2% Race/ethnicity unspecified
1% American Indian or Alaska Native
1% Asian
1% Black or African American

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

  1. Reliability of measures
  2. Validity of measures
  3. Intervention fidelity
  4. Missing data and attrition
  5. Potential confounding variables
  6. Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome Reliability
of Measures 		Validity
of Measures 		Fidelity Missing
Data/Attrition 		Confounding
Variables 		Data
Analysis 		Overall
Rating
1: Psychological dysfunction 3.5 3.5 3.0 3.1 3.0 3.4 3.3

Study Strengths

The OQ-45 is a reliable and valid measure. Both studies were well controlled and randomized. The sample sizes were sufficient. The analyses were appropriate for the data and questions posed.

Study Weaknesses

Although the investigators had control over the presentation of the feedback therapists gave to the clients, there were no measures described that specifically examined the fidelity of the intervention.
Readiness for Dissemination

Materials Reviewed

The materials below were reviewed for Readiness for Dissemination. The implementation point of contact can provide information regarding implementation of the intervention and the availability of additional, updated, or new materials.

Burlingame, G. M., Wells, M. G., Cox, J. C., Lambert, M. J., & Latkowski, M. Administration and scoring manual for the Y-OQ (youth outcome questionnaire). Salt Lake City, UT: OQ Measures.

Lambert, M. J., Morton, J. J., Hatfield, D., Harmon, C., Hamilton, S., Reid, R. C., et al.. Administration and scoring manual for the OQ-45.2 (outcome questionnaire). Salt Lake City, UT: OQ Measures.

OQ Measures. OQ-Analyst user's guide v 3.05.01. Salt Lake City, UT: Author.

OQ Measures. Orientation and training [DVD]. Salt Lake City, UT: Author.

OQ Measures. OQ Analyst Web demonstration instructions. Salt Lake City, UT: Author.

OQ Measures. Implementation guidelines for the OQ-Analyst (OQ-A). Salt Lake City, UT: Author.

OQ Measures. OQ Analyst system overview. Salt Lake City, UT: Author.

OQ Measures. Step x step training on OQ-A for clinical staff [PowerPoint slides]. Salt Lake City, UT: Author.

OQ Measures. Step x step training on OQ-A for support staff [PowerPoint slides]. Salt Lake City, UT: Author.

Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the intervention's Readiness for Dissemination using three criteria:

  1. Availability of implementation materials
  2. Availability of training and support resources
  3. Availability of quality assurance procedures

For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.

Implementation
Materials 		Training and Support
Resources 		Quality Assurance
Procedures 		Overall
Rating
4.0 3.8 4.0 3.9

Dissemination Strengths

Implementation and user guides are detailed and logically organized. Clear, step-by-step instructions are provided for installing software, administering the questionnaire, and interpreting reports. Specialized on-site or teleconferenced training is available for multiple staff roles. The questionnaire itself serves as a quality assurance instrument to track client progress and assess clinician skills.

Dissemination Weaknesses

The mix of extensive research data with implementer information in some materials can be difficult to follow. No standardized training curriculum is provided to trainees to supplement the implementation materials.
Costs

The cost information below was provided by the developer. Although this cost information may have been updated by the developer since the time of review, it may not reflect the current costs or availability of items (including newly developed or discontinued items). The implementation point of contact can provide current information and discuss implementation requirements.

Item Description Cost Required by Developer
OQ-A software system, user manuals, and documentation $250 per clinician (full-time equivalent) per year for three measures Yes
Additional measures $40 per clinician (full-time equivalent) per year for additional measures No
Technical manuals for additional measures $25 each No
Training and orientation video Free No
Webinars and other training videos Varies depending on site needs No
Half-day to 1-day, on-site initial and follow-up trainings Varies depending on site needs No
Technical assistance and installation support $150 per site Yes

Additional Information