Frequently Asked Questions (FAQs)

General Information

What is NREPP?

The National Registry of Evidence-based Programs and Practices (NREPP) is a searchable online registry of interventions that support mental health promotion, substance abuse prevention, and mental health and substance abuse treatment. Through NREPP, users can identify and learn more about interventions that may meet their needs. All interventions that are included in NREPP have been voluntarily submitted, have met a set of minimum requirements, and have been assessed by independent reviewers.

What does Quality of Research mean?

Quality of Research (QOR) is one of two main categories of NREPP ratings. There are six QOR rating criteria: reliability of measures; validity of measures; intervention fidelity; missing data and attrition; potential confounding variables; and appropriateness of analysis. Together, these criteria assess the strength of evidence supporting the results or outcomes of the intervention. Because interventions may target multiple outcomes, and the evidence supporting the different outcomes may vary, each outcome is rated separately. Each intervention summary contains QOR ratings as well as a description of the study strengths and weaknesses that may have contributed to those ratings. For more information on the QOR ratings, see the Quality of Research page.

What does Readiness for Dissemination mean?

Readiness for Dissemination (RFD) is one of two main categories of NREPP ratings. There are three RFD rating criteria: availability of implementation materials; availability of training and support resources; and availability of quality assurance procedures. Together, these criteria assess the quality and availability of an intervention's training and implementation materials, that is, how easily the intervention can be implemented with fidelity using the materials and services that are currently available to the public. Each intervention summary contains RFD ratings as well as a description of the dissemination strengths and weaknesses that may have contributed to those ratings. For more information on the RFD ratings, see the Readiness for Dissemination page.

What is comparative effectiveness research?

The Federal Coordinating Council for Comparative Effectiveness Research defined comparative effectiveness research, or CER, as follows:

"Comparative effectiveness research is the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in 'real world' settings. The purpose of this research is to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances.

To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations and subgroups.
Defined interventions compared may include medications, procedures, medical and assistive devices and technologies, diagnostic testing, behavioral change, and delivery system strategies.
This research necessitates the development, expansion, and use of a variety of data sources and methods to assess comparative effectiveness and actively disseminate the results."

Each intervention summary included in NREPP indicates whether or not the intervention was evaluated in CER studies (i.e., whether or not at least one of the studies included in the NREPP review met the definition of CER given above).

Are environmental interventions/community-level interventions included in NREPP?

Yes.

What is the recommended format for referencing an NREPP intervention summary?

NREPP recommends the following format for references:

[Program name]. Intervention summary retrieved on [date] from the Substance Abuse and Mental Health Services Administration's National Registry of Evidence-based Programs and Practices, [URL of summary]

How do I become a reviewer for NREPP?

Potential reviewers are recruited, selected, and approved by SAMHSA on the basis of their experience and areas of expertise. Quality of Research reviewers and Readiness for Dissemination reviewers are required to have specific expertise. Potential reviewers also must complete training on the procedures and criteria they will use to rate interventions. For more information, see the Reviewer Selection & Training page.

How do I get more information on an intervention I found on the site?

For more information about an intervention included on this site, please contact the developer directly, using the information provided in the intervention summary.

Why are there two review dates in some intervention summaries?

Interventions may undergo a new review if they have been included in the registry for 5 years and if new research and/or updated dissemination materials are available (see Can an intervention be reviewed more than once?). The new review may include a new Quality of Research (QOR) review, a new Readiness for Dissemination (RFD) review, or both. When an intervention undergoes a new review, the summary is updated to display the results of both the original and the new QOR and RFD reviews. Review dates are included to differentiate the information displayed from the two reviews.

The NREPP Review Process

Who rates the interventions reviewed?

NREPP contract staff assign two sets of two reviewers to independently review each intervention: one set for the Quality of Research review and one set for the Readiness for Dissemination review. For more information, see the Reviewer Selection & Training page.

How are interventions reviewed?

Each intervention is reviewed by two Quality of Research reviewers and two Readiness for Dissemination reviewers. Each reviewer receives a review packet of materials and information that has been assembled by NREPP staff and includes copies of studies and supporting documents (QOR reviews) or dissemination materials (RFD reviews). Working independently, each reviewer assesses the information presented in the packet using NREPP’s rating criteria. Each reviewer then completes a form in the packet, providing numerical QOR or RFD ratings as well as written comments discussing the strengths and weaknesses of the studies or the dissemination materials and processes. Completed packets are returned to NREPP staff, who prepare an intervention summary that includes the reviewers’ ratings and comments.

Yes. At the conclusion of the review, NREPP sends the program developer a draft intervention summary and asks him or her to provide comments. NREPP's response to these comments may include making factual corrections or other adjustments to clarify the results of the review. The summary is not published until the developer has approved the final version.

Can the program developer decline publication of the intervention summary on the NREPP Web site?

Yes, program developers have the option to decline publication of the summary. However, each NREPP review represents a considerable investment of time and public funds, and developers who agree to participate in a review are expected to authorize NREPP to publish the summary. In the event that a developer decides not to provide this approval, NREPP will include a notation on the Web site, indicating the intervention was reviewed but the developer declined to authorize publication of the summary. Although reviewed, these interventions will not be considered included in the registry.

Can an intervention be reviewed more than once?

An intervention reviewed by NREPP may be eligible for a new review in 5 years if certain requirements are met. The new review may include a new Quality of Research review, a new Readiness for Dissemination review, or both.

To qualify for a new Quality of Research review, one of the following must apply:

A new study or studies has been conducted to evaluate the same intervention with the same target population or a slightly modified intervention with the same or different target population.
A study or studies previously reviewed by NREPP has been updated with results from secondary analysis of the original dataset and/or analysis of longitudinal data.

All studies used in a Quality of Research review must demonstrate significant (p ≤ .05) findings related to substance abuse and/or mental health behavioral outcomes. The studies must be described in a peer-reviewed journal article or other professional publication (e.g., a book volume) or documented in a comprehensive evaluation report.

To qualify for a new Readiness for Dissemination review, the intervention's dissemination materials must have changed markedly, or new materials must have been developed, since the previous review was conducted.

NREPP plans to periodically invite cohorts of previously reviewed interventions to participate in new reviews, subject to the availability of funding.

My intervention has been accepted for review. How can I refer to its NREPP status in my program materials?

Program materials, including Web sites, can state that the intervention has been accepted for review by NREPP. A list of interventions that have been Accepted for Review appears on the NREPP Web site, and an intervention's inclusion on this list can be noted.

Please note that the acceptance of interventions for review and the inclusion of reviewed interventions on the NREPP Web site are not intended to convey endorsement, recommendation, or approval of these interventions by SAMHSA.

Submitting an Intervention for Review

How do I find out if my intervention qualifies for submission?

Requirements for submission are determined for each individual open submission period and announced in a Federal Register notice several months prior to the start of the open submission period. These notices are posted on the NREPP Newsroom page.

Is there any cost to submit an intervention to NREPP?

There is no fee to submit an intervention to NREPP. However, submitters must provide all the materials to be used in the review, including three copies of all dissemination materials, at their own expense.

Will every intervention that is submitted be reviewed?

No. Each submission undergoes a preliminary assessment to determine whether it has met minimum requirements. Submissions that meet the minimum requirements may receive an invitation for review from SAMHSA, depending on the availability of current funding.

Will there be another open submission period?

Can I submit multiple interventions for review?

Yes. Each intervention must be submitted separately.

Can interventions developed outside the United States be reviewed by NREPP?

Yes, if they meet the minimum requirements and have implementation materials that can be disseminated in the United States.

Why aren't prevention and treatment interventions considered under separate criteria?

The developers and Federal sponsors of NREPP believe that the scientific standards for the evidence-based practice assessments contained within NREPP apply equally to the prevention and treatment components of behavioral health care.

Information for Developers of NREPP-Reviewed Interventions

My intervention is included in the registry. How can I refer to the intervention's NREPP status in program materials?

Program materials, including Web sites, can state that the intervention has been reviewed by NREPP and is included in SAMHSA's National Registry of Evidence-based Programs and Practices.

Can I update my intervention summary?

Developers may request updates to some parts of the summary after its publication on the NREPP Web site.

Updates are generally limited to the following sections:

Costs
Implementation history
Adaptations
Replications

Updates cannot be made to any of the following: ratings, description of measures, key findings, strengths and weaknesses, outcomes, outcome categories, study populations, settings, and geographic locations.