Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Granholm, E., Holden, J., Link, P. C., McQuaid, J. R., & Jeste, D. V. (in press). Randomized controlled trial of Cognitive Behavioral Social Skills Training for older consumers with schizophrenia: Defeatist performance attitudes and functional outcome. American Journal of Geriatric Psychiatry.
Supplementary Materials Implementation of CBSST
Psychometric information for CBSST
Outcomes
| Outcome 1: Psychosocial functioning |
|
Description of Measures
|
Psychosocial functioning was assessed using the Independent Living Skills Survey (ILSS), a 51-item self-report measure of 10 domains of functioning (e.g., hygiene, cooking, cleaning, health maintenance, money management, socialization, leisure activities, work, school). The ILSS was administered in an interview format to participants, who responded to each item by indicating whether it was performed (score of 1) or not performed (score of 0) or by selecting "not able to demonstrate" (e.g., for food preparation items when meals were provided by assisted living staff). The average score of available items was computed for each domain, and the average score (ranging from 0 to 1) of available domains was used in analyses. Domain scores were not computed if more than half of the items were missing or had "not able to demonstrate" as the response. If a participant's domain scores could not be computed for at least 7 of the 10 domains, those scores were omitted from analyses.
|
|
Key Findings
|
A study was conducted with veterans and nonveterans with schizophrenia or schizoaffective disorder who were recruited from an outpatient clinic at a university-affiliated Veterans Affairs (VA) hospital. Participants were randomly assigned to the intervention group, which received CBSST, or the comparison group, which received supportive group therapy focused on helping participants achieve functioning goals. Data were collected at baseline, at the midtreatment point (4.5 months), and at the end of treatment (9 months). Results indicated that over time, participants in the intervention group had a greater increase in psychosocial functioning relative to those in the comparison group (p = .021).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 2: Acquisition of communication, problem-solving, and thought-challenging skills |
|
Description of Measures
|
The acquisition of communication, problem-solving, and thought-challenging skills was assessed using a version of the Comprehensive Module Test (CMT) that was adapted for use in this study. The adapted version of the CMT assessed skills as taught through CBSST.
|
|
Key Findings
|
A study was conducted with veterans and nonveterans with schizophrenia or schizoaffective disorder who were recruited from an outpatient clinic at a university-affiliated VA hospital. Participants were randomly assigned to the intervention group, which received CBSST, or the comparison group, which received supportive group therapy focused on helping participants achieve functioning goals. Data were collected at baseline, at the midtreatment point (4.5 months), and at the end of treatment (9 months). Results indicated that over time, participants in the intervention group had a greater increase in the acquisition of communication, problem-solving, and thought-challenging skills relative to those in the comparison group (p = .001).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 3: Severity of symptoms |
|
Description of Measures
|
Severity of symptoms was assessed using the following measures:
- Scale for the Assessment of Negative Symptoms (SANS). SANS assesses five symptom complexes in patients with schizophrenia: affective flattening (little or no emotional expression), alogia (impoverished thinking), avolition (lack of motivation)-apathy, anhedonia (inability to experience pleasure)-asociality, and disturbance of attention. On the basis of factor analytic studies of the SANS, two negative symptom factors were derived: Diminished Expression, defined as the average of affective flattening and alogia global ratings, and Diminished Motivation, defined as the average of avolition-apathy and anhedonia-asociality global ratings. (The disturbance of attention symptom complex was not included in analyses.)
- Beck Depression Inventory, second edition (BDI-II). The BDI-II is a 21-item self-report instrument that assesses the severity of depression symptoms. Each item includes a list of four statements about a particular symptom of depression, and respondents select the statement that best reflects their feelings or behavior.
- Beck Anxiety Inventory (BAI). The BAI is a 21-item scale that assesses the severity of self-reported anxiety. Using a 4-point scale ranging from 0 ("not at all") to 3 ("severely; I could barely stand it"), respondents rate each descriptive statement of anxiety symptoms.
|
|
Key Findings
|
A study was conducted with veterans and nonveterans with schizophrenia or schizoaffective disorder who were recruited from an outpatient clinic at a university-affiliated VA hospital. Participants were randomly assigned to the intervention group, which received CBSST, or the comparison group, which received supportive group therapy focused on helping participants achieve functioning goals. Data were collected at baseline, at the midtreatment point (4.5 months), and at the end of treatment (9 months). Results indicated that over time, participants in the intervention and comparison groups had improvements in negative symptoms (Diminished Motivation symptom factor of the SANS; p = .005), depression (BDI-II; p = .01), and anxiety (BAI; p = .007); however, there were no significant differences between groups in regard to these improvements.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.8
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
55+ (Older adult)
|
55% Male
45% Female
|
66% White
34% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Psychosocial functioning
|
3.6
|
3.5
|
4.0
|
3.3
|
3.4
|
4.0
|
3.6
|
|
2: Acquisition of communication, problem-solving, and thought-challenging skills
|
3.5
|
3.4
|
4.0
|
3.3
|
3.4
|
4.0
|
3.6
|
|
3: Severity of symptoms
|
4.0
|
4.0
|
4.0
|
3.3
|
3.4
|
4.0
|
3.8
|
Study Strengths The outcome measures used in the study are well known and have moderate to strong psychometric properties. The study employed a formal assessment process to determine the extent to which both CBSST and the comparison intervention were implemented with fidelity. All therapy sessions were videotaped, and 24 randomly selected sessions were rated by experts using the Cognitive Therapy Scale for Psychosis (CTS-Psy). Therapists were blind to which sessions would be rated. In addition, two therapists who provided CBSST received initial training and weekly supervision that included review of videotaped sessions. The study used a randomized experimental design, which minimized some of the confounding variables. Sophisticated and appropriate analyses were used in the study.
Study Weaknesses Baseline ILSS scores indicated that the two study groups differed significantly in regard to psychosocial functioning.
|
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Granholm, E., McQuaid, J. R., McClure, F. S., Auslander, L. A., Perivoliotis, D., Pedrelli, P., et al. A randomized, controlled trial of cognitive behavioral social skills training for middle-aged and older outpatients with chronic schizophrenia. American Journal of Psychiatry, 162(3), 520-529.
Granholm, E., McQuaid, J. R., McClure, F. S., Link, P., Perivoliotis, D., Gottlieb, J., et al. Randomized clinical trial of cognitive behavioral social skills training for older people with schizophrenia: 12-month follow-up. Journal of Clinical Psychiatry, 68(5), 730-737.
Supplementary Materials Granholm, E., Auslander, L. A., Gottlieb, J. D., McQuaid, J. R., & McClure, F. S. Therapeutic factors contributing to change in cognitive-behavioral group therapy for older persons with schizophrenia. Journal of Contemporary Psychotherapy, 36(1), 31-41.
Granholm, E., McQuaid, J. R., Auslander, L. A., & McClure, F. S.. Group cognitive behavioral social skills training for older adult patients with chronic schizophrenia. Journal of Cognitive Psychotherapy. Special Issue: Cognitive Theory and Therapy of Schizophrenia, 18(3), 265-279.
Granholm, E., McQuaid, J. R., McClure, F. S., Pedrelli, P., & Jeste, D. V. A randomized, controlled pilot study of cognitive behavioral social skills training for older patients with schizophrenia. Schizophrenia Research, 53, 167-169.
McQuaid, J. R., Granholm, E., McClure, F. S., Roepke, S., Pedrelli, P., Patterson, T. L., et al. Development of an integrated cognitive-behavioral and social skills training intervention for older patients with schizophrenia. Journal of Psychotherapy Practice and Research, 9(3), 149-156.
Table of information on reliability and validity of measures used in CBSST
Outcomes
| Outcome 1: Psychosocial functioning |
|
Description of Measures
|
Psychosocial functioning was assessed using the Independent Living Skills Survey (ILSS) and the University of California, San Diego Performance-Based Skills Assessment (UPSA). The ILSS is a self-report measure of basic functional living skills performed during the past month; it was administered in an interview format. The ILSS assesses appearance and clothing, personal hygiene, health maintenance, transportation, and leisure and community, all of which are sensitive to functional impairment in older outpatients with schizophrenia. The UPSA is a performance-based measure of the extent to which patients are capable of performing specific functional living skills. The instrument uses standardized role-playing situations to assess domains of functioning such as household chores, communication, finance, transportation, and planning recreational activities.
|
|
Key Findings
|
At 1-year follow-up, participants in CBSST showed statistically significant improvement in social functioning relative to participants receiving treatment as usual (p = .05). However, no differences between CBSST and treatment as usual were found in participants' capacity to perform basic everyday activities as measured by the UPSA.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 2: Symptom severity |
|
Description of Measures
|
Symptom severity was assessed by the Positive and Negative Symptom Scale (PANSS) and the Hamilton Rating Scale for Depression (HAM-D). The PANSS is a 30-item interview that provides balanced representation of positive and negative symptoms of schizophrenia and gauges their relationship to one another and to global psychopathology; the HAM-D is a 21-item multiple choice questionnaire used to assess depressive symptoms.
|
|
Key Findings
|
No significant reduction in symptom severity was found for CBSST participants at any assessment point. However, in posthoc analysis, a greater improvement in cognitive insight with CBSST was significantly correlated with greater reduction in positive symptoms (p < .05).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 3: Cognitive insight |
|
Description of Measures
|
Cognitive insight refers to metacognitive processes involved in the evaluation and correction of distorted beliefs and misinterpretations (e.g., cognitive flexibility, objective distancing, reappraisal of symptoms). The Beck Cognitive Insight Scale (BCIS) is a self-report questionnaire that assesses metacognitive processes on two subscales: self-reflectiveness (e.g., "I have jumped to conclusions too fast," "If someone points out that my beliefs are wrong, I am willing to consider it") and self-certainty (e.g., "My interpretations of my experiences are definitely right," "I can trust my own judgement at all times").
|
|
Key Findings
|
At the end of treatment, participants in CBSST had significantly greater cognitive insight scores compared with participants receiving treatment as usual, indicating comparatively greater objectivity in reappraising psychotic symptoms (p < .01). However, this difference was not maintained at 1-year follow-up.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.9
(0.0-4.0 scale)
|
| Outcome 4: Acquisition of communication, problem-solving, and thought-challenging skills |
|
Description of Measures
|
A Comprehensive Module Test (CMT) was used to assess knowledge of the specific skills and information taught in CBSST (communication, problem-solving, and thought challenging). The CMT was originally developed by the University of California, Los Angeles Center for Research on Severe Mental Illness (now the UCLA Center for Research on Treatment & Rehabilitation of Psychosis). The measure also attempts to assess the application of this knowledge by presenting specific social situations or problems (e.g., "You want to meet someone at a senior center") and probing for the application of acquired skills (e.g., steps in problem-solving: define the problem, generate solutions, list advantages and disadvantages, etc). The developer modified the measure to also include a cognitive component, which assesses patients' knowledge of acquired cognitive skills (e.g., "What are the 3 C's?") and application of these skills to symptoms and situations (e.g., "You think people were talking about you before you entered a room; what should you do?").
|
|
Key Findings
|
At the end of treatment, CBSST participants showed significantly greater skill acquisition than participants receiving treatment as usual (p < .01). This gain was maintained at 1-year follow-up (p < .05).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.3
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
55+ (Older adult)
|
74% Male
26% Female
|
80% White
20% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Psychosocial functioning
|
3.5
|
1.5
|
4.0
|
3.0
|
2.5
|
3.5
|
3.0
|
|
2: Symptom severity
|
3.5
|
4.0
|
4.0
|
3.0
|
2.5
|
3.5
|
3.4
|
|
3: Cognitive insight
|
3.5
|
1.0
|
4.0
|
3.0
|
2.5
|
3.5
|
2.9
|
|
4: Acquisition of communication, problem-solving, and thought-challenging skills
|
0.0
|
1.0
|
4.0
|
3.0
|
2.5
|
3.5
|
2.3
|
Study Strengths The measures used for outcome 1 (increased psychosocial functioning) have evidence of reliability. Instruments used for outcome 2 (reduced symptom severity) show good evidence of validity. This was a well-designed randomized study with evaluators blinded to group assignment. The treatment incorporated evidence-based practices modified for older adults with schizophrenia, addressing the cognitive and social issues documented in this population. Training procedures was manualized and administered by adequately trained personnel who were supervised by senior personnel using videotaped reviews. Attrition and missing data were reported and handled appropriately. Analyses were thorough and appropriate. Careful attention was paid to treatment fidelity.
Study Weaknesses The Comprehensive Module Test (CMT) does not have reliability data other than repeated administrations of the tool with follow-up that appears to show knowledge was maintained over time. The sample size was modest, and no evidence was presented to show that the statistically significant outcomes are related to real-world improved function for the study population. A potential confounding variable was the use of predominantly Caucasian males in the sample, which may impact generalizability. The discussion of validity was inadequate for most of the measures.
|
|
Readiness for Dissemination
|
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Granholm, E. Cognitive Behavioral Social Skills Training (CBSST) for schizophrenia [PowerPoint presentation].
Granholm, E. Cognitive Behavioral Social Skills Training for schizophrenia: Practical syllabus.
Granholm, E., Holden, J., & McQuaid, J. Cognitive Behavioral Social Skills Training (CBSST): Cognitive skills module consumer workbook. San Diego: Regents of the University of California.
Granholm, E., Holden, J., & McQuaid, J. Cognitive Behavioral Social Skills Training (CBSST): Problem-solving skills module consumer workbook. San Diego: Regents of the University of California.
Granholm, E., Holden, J., & McQuaid, J. Cognitive Behavioral Social Skills Training (CBSST): Social skills module consumer workbook. San Diego: Regents of the University of California.
Other program materials:
- Communication Skills Test
- Comprehensive Module Tests: Instructions to Subjects
- CTS-Psy [therapist rating scale]
- Implementation and Supervision of Cognitive Behavioral Social Skills Training (CBSST)
- Problem Solving Skills Test
- Thought Challenging Skills Test
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
2.8
|
3.3
|
3.3
|
3.1
|
Dissemination Strengths The consumer workbooks are appropriate for the intended audience and focus on individual goal setting and problem solving. They include practical tools, such as worksheets and checklists, to support the recovery process. The program Web site offers an overview of the program and describes how the three modules (cognitive skills, problem-solving skills, social skills) are integrated. An on-site training workshop is available, which helps therapists develop skills to effectively work with consumers. Workshop materials offer practical guidelines for program implementation and ongoing supervision. The developer is available to provide consultation and supervision for both therapists and their supervisors, which can help build program capacity and sustainability over time. The developers are available to rate audio- or videotaped consumer sessions using the CTS-Psy, an observational fidelity tool that supports therapist competence. In addition, there are three comprehensive skills tests, which measure consumer knowledge acquisition for each module.
Dissemination Weaknesses Implementation and intervention guidance for therapists is provided through participation in an optional training workshop rather than through a treatment manual, leaving those who cannot attend training without sufficient guidance on intervention delivery. Although training for organizations is available, it is unclear how an individual can pursue training opportunities. It is also unclear if opportunities exist for therapists to develop advanced capabilities or to network with each other. Although the developer has used various standardized outcome measures in researching the program, no specific materials or guidance for evaluating program outcomes are provided.
|
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
CBSST for Schizophrenia Syllabus
CBSST Manual (Blue Module): Asking for Support
CBSST Manual (Green Module): Solving Problems
CBSST Manual (Red Module): Changing Your Thinking
Granholm, E., & McQuaid, J. Cognitive Behavioral Therapy (CBT) for Psychosis [PowerPoint handout].
Granholm, E. Demonstration Video: CBT for Psychosis [DVD].
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
2.5
|
1.5
|
0.5
|
1.5
|
Dissemination Strengths The treatment manuals give clear instructions for session-by-session implementation. The provided forms and examples facilitate the group process and increase the likelihood that clients will do homework. The DVD illustrates how one might use program concepts in practice. The manual and DVD may be a useful training tool for clinicians who are familiar with cognitive behavioral therapy and who are highly experienced in working with clients with serious mental disorders. The continual sequencing of feedback from the client can serve as a quality assurance mechanism.
Dissemination Weaknesses This is a complex curriculum with no clear integration of concepts. It is unclear how a clinician should sequence components of the intervention and whether all components are required to obtain good treatment outcomes. The manuals are for group intervention, while the DVD demonstrates individual work. No guidance is provided as to how clinicians should choose group versus individual intervention. The DVD does not fully explain the context of the examples provided, so the concepts being presented and their utility in specific client situations may not be clear to some viewers. While the slide presentation stresses the importance of therapeutic alliance, none of the implementation materials demonstrate the use of alliance in the intervention or provide training on developing this alliance. Little information is provided to support training, coaching, or ongoing technical assistance. Programmatic accountability for interpretation and ongoing quality assurance are also not specifically addressed.
|
|