Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Erfurt, J. C., Foote, A., & Heirich, M. A. Worksite wellness programs: Incremental comparison of screening and referral alone, health education, follow-up counseling, and plant organization. American Journal of Health Promotion, 5(6), 438-448. Study 2Heirich, M., & Sieck, C. J. Worksite cardiovascular wellness programs as a route to substance abuse prevention. Journal of Occupational and Environmental Medicine, 42(1), 47-56. Study 3Heirich, M., Sieck, C. J., Klykulo, K., & Bonnington-Kouri, K. Moderation counseling as a route to substance abuse prevention: M-CARE's DrinkWise and Health Risk Appraisal Programs at the University of Michigan. Final Report for grant number 6 U 1 K SPO8146-03-02, awarded by SAMHSA/CSAP to the Greater Detroit Area Health Council and the University of Michigan.
Outcomes
| Outcome 1: Alcohol consumption |
|
Description of Measures
|
During face-to-face interviews at the initial screening and the rescreening at the end of the 3-year intervention period, participants answered the following questions:
- Do you ever drink alcoholic beverages such as beer, wine, or liquor? (yes/no)
- On how many days in an average week do you drink something alcoholic? (0-7)
- On the days that you drink, how many drinks do you have? (open ended)
- How many drinks do you have in an average week? (open ended)
|
|
Key Findings
|
One study examined drinkers who were at the highest risk level of alcohol consumption (those drinking three or more drinks three or more times per week) at initial screening. At rescreening, 38% of those who received counseling lowered their drinking to levels that did not put them at risk, compared with 22% of those who did not receive counseling. Because the drinkers at highest risk represented a small percentage of the study population, this finding was not statistically significant.
In another study, among drinkers who were at risk for alcohol-related problems (men who drank more than 3 drinks per day or 12 drinks per week and women who drank more than 2 drinks per day or 9 drinks per week) at initial screening, 68% of those who received ongoing follow-up counseling had reached a "safe" level of alcohol consumption at rescreening. By comparison, 46% of those who only received one brief, end-of-screening counseling session had reached a safe level at rescreening (p < .05).
|
|
Studies Measuring Outcome
|
Study 2, Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 2: Smoking cessation |
|
Description of Measures
|
During face-to-face interviews at the initial screening and the rescreening at the end of the 3-year intervention period, participants provided a history of their cigarette smoking (e.g., ever smoked, how often).
|
|
Key Findings
|
In one study, worksites that provided follow-up monitoring and counseling along with a menu of four other service options (guided self-help, one-on-one formal consultation, mini-group interventions, and full-group classes) had higher rates of participation in worksite smoking cessation services than worksites with the regular offering of wellness-related activities (p < .001). Further, among employees identified as having smoking as a cardiovascular disease (CVD) risk factor, those who received follow-up and a menu of service options had higher rates of smoking cessation (p < .01) and lower rates of smoking recidivism (p < .01) at rescreening than those who received the regular offering of programs.
In another study, among those identified at initial screening as smokers or former smokers, 65% of those who received counseling were not smoking at rescreening, compared with 53% of those who did not receive counseling (p < .001).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 3: Overall health risks |
|
Description of Measures
|
Measures of overall health risks, obtained at the initial screening and the rescreening at the end of the 3-year intervention period, included systolic and diastolic blood pressure, weight, total cholesterol, and HDL cholesterol.
|
|
Key Findings
|
In one study, worksites that provided follow-up monitoring and counseling along with a menu of four other service options (guided self-help, one-on-one formal consultation, mini-group interventions, and full-group classes) had higher rates of participation in worksite blood pressure treatment (p < .05) and weight loss (p < .001) services than worksites with the regular offering of wellness-related activities. Further, among employees identified as having high blood pressure or overweight as CVD risk factors, those who received follow-up and a menu of service options had better blood pressure control (p < .05) and greater weight loss (p < .01), respectively, at rescreening than those who received the regular offering of programs.
In another study, overall health risks improved among all study groups--those who received counseling and those who did not--from initial screening to rescreening: (1) of participants with hypertension, the percentage with blood pressure under control increased from 29% to 53% (p < .001); (2) of participants with hypercholesterolemia, the percentage with cholesterol under control increased from 2% to 27% (p < .001); and (3) of participants 20% or more overweight, 31% lost 3 or more pounds (p < .001), and 19% lost 10 or more pounds (p < .001).
In a third study, employees who received ongoing follow-up counseling had significant outcomes related to CVD risks, with a higher proportion of participants becoming risk free (p < .01) and a lower proportion of participants developing new CVD risks (p < .05) compared with employees who only received one brief, end-of-screening counseling session.
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
89.5% Male
10.5% Female
|
70.7% White
29.3% Race/ethnicity unspecified
|
|
Study 2
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
Data not reported/available
|
63% Race/ethnicity unspecified
37% Black or African American
|
|
Study 3
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
64.5% Female
35.5% Male
|
81% White
10.8% Race/ethnicity unspecified
8.2% Black or African American
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Alcohol consumption
|
2.6
|
3.0
|
3.0
|
4.0
|
2.9
|
3.3
|
3.1
|
|
2: Smoking cessation
|
2.5
|
2.6
|
2.8
|
3.1
|
2.8
|
3.4
|
2.8
|
|
3: Overall health risks
|
3.6
|
3.6
|
2.8
|
3.5
|
3.1
|
3.5
|
3.3
|
Study Strengths The studies were well designed and improved over time with the introduction of randomization at the subject level, use of health care utilization records, and use of more reliable and valid measures. The concordance between level of cardiovascular risk and indicators of drinking severity is an indication of the validity of both measures. In one study, a very high certainty of consistent service delivery could not be documented given the dispersal and sheer scope of the work conducted at the four experimental sites; however, counselors underwent a 2-day training, and new counselors were shadowed for the first few visits. Rescreening rates were above 80% of targeted respondents, which is high for large-scale studies like these. Missing data were rare, and in two of the three studies, missing values were imputed using the conservative LOCF (last observation carried forward) method. The authors considered and addressed many potential confounds. Logistic and other more sophisticated regression techniques were sometimes used and showed fairly clean relationships between the variables tested.
Study Weaknesses The nonbiometric measures used are not well developed and have questionable reliability and validity. Some of the services provided (e.g., alcohol counseling) were complex and highly vulnerable both to inexpert usage and drift. Some important sources of potential bias were uncontrolled. For example, counselors, who could not have been blind to subjects' group assignment, may have been biased about the superiority of individual versus group follow-up counseling. In one study, workers randomized to the group condition wanted to cross over to the individual condition, and workers who were not included in the study wanted to participate in the individual follow-up. This necessitated modifications to the analytic scheme. The analytic techniques used were sometimes too simplistic. For example, partial or semipartial statistics could have been used to remove the influence of preintervention status from change scores.
|
|