Search form



Reducing Behavior Problems in the
Elementary School Classroom

TABLE 1:
Institute of Education Sciences levels of evidence for practice guides

Back to article

Strong
In general, characterization of the evidence for a recommendation as strong requires both studies with high internal validity (i.e., studies whose designs can support causal conclusions) and studies with high external validity (i.e., studies that in total include enough of the range of participants and settings on which the recommendation is focused to support the conclusion that the results can be generalized to those participants and settings). Strong evidence for this practice guide is operationalized as:

  • A systematic review of research that generally meets the standards of the What Works Clearinghouse (WWC) (see http://ies.ed.gov/ncee/wwc/) and supports the effectiveness of a program, practice, or approach with no contradictory evidence of similar quality; OR
  • Several well-designed, randomized controlled trials or well-designed quasiexperiments that generally meet the WWC standards and support the effectiveness of a program, practice, or approach, with no contradictory evidence of similar quality; OR
  • One large, well-designed, randomized controlled, multisite trial that meets the WWC standards and supports the effectiveness of a program, practice, or approach, with no contradictory evidence of similar quality; OR
  • For assessments, evidence of reliability and validity that meets the Standards for Educational and Psychological Testing.*

Moderate
In general, characterization of the evidence for a recommendation as moderate requires studies with high internal validity but moderate external validity, or studies with high external validity but moderate internal validity. In other words, moderate evidence is derived from studies that support strong causal conclusions but where generalization is uncertain, or studies that support the generality of a relationship but where the causality is uncertain. Moderate evidence for this practice guide is operationalized as:

  • Experiments or quasiexperiments generally meeting the WWC standards and supporting the effectiveness of a program, practice, or approach with small sample sizes and/or other conditions of implementation or analysis that limit generalizability and no contrary evidence; OR
  • Comparison group studies that do not demonstrate equivalence of groups at pretest and therefore do not meet the WWC standards but that (a) consistently show enhanced outcomes for participants experiencing a particular program, practice, or approach and (b) have no major flaws related to internal validity other than lack of demonstrated equivalence at pretest (e.g., only one teacher or one class per condition, unequal amounts of instructional time, highly biased outcome measures); OR
  • Correlational research with strong statistical controls for selection bias and for discerning influence of endogenous factors and no contrary evidence; OR
  • For assessments, evidence of reliability that meets the Standards for Educational and Psychological Testing** but with evidence of validity from samples not adequately representative of the population on which the recommendation is focused.

Low
In general, characterization of the evidence for a recommendation as low means that the recommendation is based on expert opinion derived from strong findings or theories in related areas and/or expert opinion buttressed by direct evidence that does not rise to the moderate or strong level. Low evidence is operationalized as evidence not meeting the standards for the moderate or high level.

* American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999).
** b. Ibid.

Back to article



Publication posted to Education World 07/06/2009
Source: U.S. Department of Education; last accessed on 07/06/2009 at
http://ies.ed.gov/ncee/wwc/pdf/practiceguides/behavior_pg_092308.pdf