Intervention Summary

Kognito At-Risk for High School Educators

Kognito At-Risk for High School Educators is a 1-hour, online, interactive gatekeeper training program that prepares high school teachers and other school personnel to identify, approach, and refer students who are exhibiting signs of psychological distress such as depression, anxiety, substance abuse, and suicidal ideation. Through a self-paced, narrative-driven experience, participants build knowledge, skills, and confidence to connect at-risk students to counseling, mental health, or crisis support services. By engaging in a role-play practice conversation with fully animated and emotionally responsive student avatars that act and respond like real humans, participants learn effective communication tactics for managing these challenging conversations. The course also works to reduce stigma around mental health issues by normalizing mental health disorders and increase protective factors by improving social connectedness. Simulations are authored and delivered using Kognito's proprietary Human Interaction Game Engine, which is based on research in social cognition, neuroscience, adult learning theory, and motivational interviewing.

The course begins by introducing the participant to a teacher who is concerned about three of his students. Assuming the role of the teacher and with the help of a virtual coach, the participant practices using evidence-based techniques to effectively broach the topic of psychological distress and motivate each student to accept appropriate support. The participant also learns to avoid common pitfalls in conducting referral conversations, such as attempting to diagnose the problem or giving unwarranted advice. By providing hands-on practice, the training increases participants' confidence and ability to handle similar challenging situations in real life. A local resources button within the course links participants to specific information about the availability of local mental health resources.

Kognito provides At-Risk for High School Educators under licensing agreements to schools, districts, statewide agencies, and nonprofit organizations. A member of the client agency serves as the course administrator. Program setup requires 1 hour of the administrator's time and is conducted via an online meeting. Depending on the institution's infrastructure, goals, and overall suicide prevention strategy, administrators can expect to spend 2-8 hours per month disseminating and promoting the course to learners. Professional development credit is often available for those completing the course.

Kognito At-Risk for High School Educators is part of a suite of online role-play training simulations offered by Kognito. Other courses are available for college students, military families, and medical professionals (see Adaptations section below).

Descriptive Information

Areas of Interest	Mental health promotion
Outcomes	1: Preparedness to recognize, approach, and refer students exhibiting signs of psychological distress 2: Likelihood of approaching and referring students exhibiting signs of psychological distress 3: Confidence in one's ability to help students exhibiting signs of psychological distress
Outcome Categories	Mental health Suicide
Ages	18-25 (Young adult) 26-55 (Adult) 55+ (Older adult)
Genders	Male Female
Races/Ethnicities	Race/ethnicity unspecified
Settings	School
Geographic Locations	Urban Suburban Rural and/or frontier
Implementation History	Since its release Kognito At-Risk for High School Educators has been adopted by State-level agencies in Alaska, Arizona, Illinois, Maryland, New York, North Dakota, Ohio, Texas, and Washington for use by more than 500,000 teachers, administrators, and staff. It has been used in public, charter, vocational, and parochial schools and teacher training programs. The course was developed to be suitable for a wide variety of settings and appropriate for and acceptable to a broad range of racial and ethnic groups.
NIH Funding/CER Studies	Partially/fully funded by National Institutes of Health: No Evaluated in comparative effectiveness research studies: No
Adaptations	Kognito At-Risk for High School Educators is part of a suite of online role-play training simulations that support large-scale, universal mental health promotion and suicide prevention, as well as early intervention of suicidal ideation and mental illness. Each course is developed for a specific group of learners. In addition to At-Risk for High School Educators, other courses are available for college students, K-12 educators, military families, and medical professionals.
Adverse Effects	No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories	Universal

Quality of Research

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Study 1

Albright, G., Eastgard, S., Goldman, R., & Shockley, K. At-Risk for High School Educators: On-line interactive gatekeeper training simulation for identification and referral of students exhibiting signs of psychological distress. Unpublished manuscript.

Study 2

Supplementary Materials

At-Risk for High School Educators Supporting Document (document created for the NREPP review)

Outcomes

Outcome 1: Preparedness to recognize, approach, and refer students exhibiting signs of psychological distress
Description of Measures	Three items from a 12-item online questionnaire were used to assess this outcome. Teachers were asked to rate their perceived preparedness to (1) recognize when a student is exhibiting signs of psychological distress, (2) approach a student exhibiting signs of distress to discuss their concerns, and (3) refer a distressed student to a counseling center. An example of one of the items is "How would you rate your preparedness to recognize when a student's behavior is an indicator of psychological distress?" Responses were on a 4-point Likert scale ranging from "low" to "very high."
Key Findings	In one study, teachers in the intervention group completed pre- and posttest assessments immediately before and after the 1-hour intervention. The results of these assessments were compared with those of a single assessment completed by teachers in a no-treatment control group. The groups were equivalent on all measures at pretest. At posttest, compared with the control group, the intervention group had higher scores for preparedness to recognize, approach, and refer a distressed student (all p values < .001). The intervention group also showed pre- to posttest increases in preparedness to recognize, approach, and refer a distressed student (all p values < .001). In another study, teachers in the intervention group completed a posttest assessment immediately after the 1-hour intervention. The results of this assessment were compared with those of an assessment completed by teachers in a no-treatment control group. At posttest, compared with the control group, the intervention group reported greater preparedness to recognize (p = .02), approach (p < .001), and refer (p < .001) a distressed student.
Studies Measuring Outcome	Study 1, Study 2
Study Designs	Quasi-experimental, Preexperimental
Quality of Research Rating	2.8 (0.0-4.0 scale)

Outcome 2: Likelihood of approaching and referring students exhibiting signs of psychological distress
Description of Measures	One item from a 12-item online questionnaire was used to assess this outcome. Teachers were asked, "How likely are you to approach and, if necessary, refer a student exhibiting signs of psychological distress?" Responses were on a 4-point Likert scale ranging from "not likely" to "very likely."
Key Findings	In one study, teachers in the intervention group completed pre- and posttest assessments immediately before and after the 1-hour intervention. The results of these assessments were compared with those of a single assessment completed by teachers in a no-treatment control group. The groups were equivalent on all measures at pretest. At posttest, compared with the control group, the intervention group reported a higher likelihood of approaching and referring a student exhibiting signs of psychological distress (p < .001). The intervention group also showed a pre- to posttest increase for this outcome (p < .001). In another study, teachers in the intervention group completed a posttest assessment immediately after the 1-hour intervention. The results of this assessment were compared with those of an assessment completed by teachers in a no-treatment control group. At posttest, compared with the control group, the intervention group reported a higher likelihood of approaching and referring a student exhibiting signs of psychological distress (p < .001).
Studies Measuring Outcome	Study 1, Study 2
Study Designs	Quasi-experimental, Preexperimental
Quality of Research Rating	2.8 (0.0-4.0 scale)

Outcome 3: Confidence in one's ability to help students exhibiting signs of psychological distress
Description of Measures	One item from a 12-item online questionnaire was used to assess this outcome. Teachers were asked to rate their agreement with the following statement: "I feel confident in my ability to help a suicidal student seek help." Responses were on a 4-point Likert scale ranging from "strongly disagree" to "strongly agree."
Key Findings	In one study, teachers in the intervention group completed pre- and posttest assessments immediately before and after the 1-hour intervention. The results of these assessments were compared with those of a single assessment completed by teachers in a no-treatment control group. The groups were equivalent on all measures at pretest. At posttest, compared with control group participants, teachers in the intervention group showed higher confidence in their ability to help a suicidal student seek help (p < .001). The change in teachers' confidence from pre- to posttest was not signficant. In another study, teachers in the intervention group completed a posttest assessment immediately after the 1-hour intervention. The results of this assessment were compared with those of an assessment completed by teachers in a no-treatment control group. At posttest, compared with control group participants, teachers in the intervention group had higher confidence in their ability to help a suicidal student seek help (p < .001).
Studies Measuring Outcome	Study 1, Study 2
Study Designs	Quasi-experimental, Preexperimental
Quality of Research Rating	2.8 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study	Age	Gender	Race/Ethnicity
Study 1	18-25 (Young adult) 26-55 (Adult) 55+ (Older adult)	79.8% Female 20.2% Male	100% Race/ethnicity unspecified
Study 2	18-25 (Young adult) 26-55 (Adult) 55+ (Older adult)	77.2% Female 22.8% Male	100% Race/ethnicity unspecified

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

Reliability of measures
Validity of measures
Intervention fidelity
Missing data and attrition
Potential confounding variables
Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome	Reliability of Measures	Validity of Measures	Fidelity	Missing Data/Attrition	Confounding Variables	Data Analysis	Overall Rating
1: Preparedness to recognize, approach, and refer students exhibiting signs of psychological distress	2.2	2.5	3.0	4.0	2.0	3.0	2.8
2: Likelihood of approaching and referring students exhibiting signs of psychological distress	2.0	2.5	3.0	4.0	2.0	3.0	2.8
3: Confidence in one's ability to help students exhibiting signs of psychological distress	2.0	2.5	3.0	4.0	2.0	3.0	2.8

Study Strengths

Evidence of adequate test-retest reliability and face and content validity was presented for the measures used. The online delivery of the program provides some assurance of fidelity. Missing data and attrition were reported to be low, such that no adjustments were needed. Appropriate statistical methods were used.

Study Weaknesses

For some of the outcome measures, the reliability coefficients were low, and reliability and validity data were based on a different population than that used in the study. A number of confounding variables were not adequately addressed, including differences in the timing of data collection for the intervention and control groups and differences between the study groups in recruitment source, teaching experience, and age. In one study, pre- and posttest assessments for the intervention group were compared with a single assessment for the control group that was used as both pre- and posttest; this study design does not control for the effects of pretesting and lowers confidence in the analyses involving comparisons of the two groups at posttest. The second study was limited by the use of a posttest-only design.

Readiness for Dissemination

Materials Reviewed

The materials below were reviewed for Readiness for Dissemination. The implementation point of contact can provide information regarding implementation of the intervention and the availability of additional, updated, or new materials.

Kognito Interactive. At-Risk for High School Educators: Implementation manual.

Kognito Interactive. Assessment tools to evaluate At-Risk for High Educators.

Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the intervention's Readiness for Dissemination using three criteria:

Availability of implementation materials
Availability of training and support resources
Availability of quality assurance procedures

For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.

Implementation Materials	Training and Support Resources	Quality Assurance Procedures	Overall Rating
4.0	4.0	4.0	4.0

Dissemination Strengths

The online gatekeeper training is accessible 24 hours per day, 7 days per week, and is attractive, easy to navigate, and culturally sensitive. The course includes interactive role-play exercises with student avatars, videos, and information on making local and national referrals for students in distress. An online administrator account includes a variety of tools and resources to promote and manage the intervention. An account manager assigned to each implementing site provides an initial orientation by phone as well as ongoing support. An optional introductory Webinar can be hosted and recorded by the developer, with the recording made available for future use. User tracking reports and participant surveys support quality assurance. Email reminders are automatically sent to participants who have not finished the course to increase the rate of completion. The developer continually updates and refines the course processes and materials using input from implementers.

Dissemination Weaknesses

No weaknesses were noted by reviewers.

Costs

The cost information below was provided by the developer. Although this cost information may have been updated by the developer since the time of review, it may not reflect the current costs or availability of items (including newly developed or discontinued items). The implementation point of contact can provide current information and discuss implementation requirements.

Item Description	Cost	Required by Developer
Program license (includes educator access to training and adminstrator access to downloadable implementation manual, promotional materials, usage reports, and participant surveys)	Institutional licenses for individual schools and districts: starting at $500 per school per year, depending on the number and size of the schools Statewide and large-district licenses: contact the developer	Yes
Consultation and technical assistance by phone or email	Included with license	Yes

Additional Information

For implementers with evaluation needs that go beyond the included participant surveys, Kognito provides analysis and reporting services starting at $2,500.