When district leaders select an intervention program, “evidence-based” is a requirement that shows up in procurement guidance, grant applications, and state compliance reviews. But the term means something specific under the Every Student Succeeds Act, and the specifics matter. ESSA establishes four distinct evidence tiers, and the criteria separating Level II from the levels above and below it are precise enough to trip up procurement teams that rely on a vendor’s say-so. This breakdown covers what Level II actually requires, how to evaluate whether a program qualifies, and where the common interpretation mistakes happen.
What the Law Actually Says
ESSA defines Level II as “moderate evidence,” and that definition is statutory, not interpretive. Under ESSA, moderate evidence requires at least one well-designed and well-implemented quasi-experimental study that demonstrates a statistically significant positive effect on a relevant outcome. For Level II, the evidence must also meet applicable sample-size, multisite, and population-and-setting requirements, without strong negative findings from well-designed experimental or quasi-experimental studies examining the same intervention and outcome.
That last clause is worth slowing down on. A single qualifying study is sufficient to clear the threshold, but only if the broader evidence base for that intervention does not undercut it. A program with one positive quasi-experimental study and two credible null findings is in a more complicated position than the single-study framing suggests.
The governing language appears in ESSA Section 8101(21), and the U.S. Department of Education’s guidance documents on evidence use have consistently reinforced this reading.
What Makes a Study “Quasi-Experimental”
The defining feature of a quasi-experimental study is that participants are not randomly assigned to treatment and control groups. This distinguishes Level II from Level I, which requires a randomized controlled trial. What quasi-experimental designs must do instead is construct a valid comparison group through other means, and do it rigorously enough that selection bias is credibly minimized.
Several design features determine whether a study qualifies as well-designed and well-implemented under ESSA:
- Matched comparison groups. Students or schools in the comparison group should be similar to those receiving the intervention on key observable characteristics, including prior achievement, demographics, school size, and so on.
- Baseline equivalence. Quasi-experimental studies must demonstrate that the treatment and comparison groups are sufficiently similar at baseline on characteristics relevant to the outcome. Under WWC standards, baseline differences are evaluated using standardized effect-size thresholds, with statistical adjustment required for certain differences.
- Appropriate analytic methods. The statistical approach must account for the non-random assignment and control for pre-existing differences. Propensity score matching, regression discontinuity, and difference-in-differences designs are among the approaches that can satisfy this criterion, depending on implementation quality.
- Outcome measurement. The study must measure outcomes relevant to the program’s intended effect and do so using valid, reliable instruments.
The What Works Clearinghouse (WWC), operated by the Institute of Education Sciences, has developed standards that translate ESSA’s Level II criteria into reviewable protocols. WWC reviews studies and rates them as meeting standards without reservations, meeting standards with reservations, or not meeting standards. For ESSA Level II purposes, a study that meets WWC standards, with or without reservations, on a relevant outcome is typically considered qualifying evidence, subject to the positive-effect and no-overriding-negative-evidence requirements.
The Statistically Significant Positive Effect Requirement
Statistical significance alone is not the full picture, but it is a necessary condition. A qualifying Level II study must find that the intervention group outperformed the comparison group on a relevant outcome measure, and that the difference is statistically significant, meaning it is unlikely to be attributable to chance variation at conventional thresholds (typically p less than 0.05).
Effect size matters too, though ESSA does not specify a minimum. When reviewing vendor evidence claims, districts can consider both statistical significance and the magnitude of the observed effect. A statistically significant finding may satisfy the evidence requirement while still warranting consideration of whether the effect is educationally meaningful. When reviewing vendor evidence claims, look at both the significance level and the effect size. A statistically significant effect that is very small in magnitude may technically clear the bar while offering limited practical value.
One area where interpretation gets complicated involves subgroup findings. A study may show a statistically significant positive effect for a particular subgroup, such as students with disabilities, English learners, or a specific grade band, but not for the broader sample. ESSA’s language refers to the student population the intervention is intended to serve. If the evidence base is built on a population that differs substantially from your district’s students, that gap in generalizability is worth flagging.
What “Well-Implemented” Adds to the Standard
The phrase “well-designed and well-implemented” appears throughout ESSA evidence guidance, and the implementation half of that requirement deserves attention. A study can be methodologically sound and still fail the standard if the program was delivered in a way that does not reflect how it would realistically operate in schools.
Federal guidance describes well-implemented studies as those where the intervention was delivered with fidelity to its design, where dosage and duration were sufficient to produce the intended effect, and where the study context is reasonably similar to the context in which a district intends to use the program. A tutoring program studied with daily 45-minute sessions over a full academic year may have a well-designed quasi-experimental evaluation, but a district planning to use the same program for twice-weekly 20-minute sessions is operating outside the conditions the evidence actually addresses.
This is one of the more underappreciated compliance considerations. Districts are not selecting a program with qualifying evidence and calling it done; they are committing to implement it in a way that the evidence supports.
How the “No Overriding Negative Evidence” Clause Works
ESSA’s Level II standard requires that a positive finding not be outweighed by evidence of no effect or negative effects from other credible studies. The law does not specify a numerical threshold for how many negative studies override a positive one. It calls for a judgment about the weight of the evidence.
In practice, this means your district should look beyond the study a vendor cites in a sales conversation. If independent evaluations of the same program exist, through WWC reviews, What Works Clearinghouse evidence snapshots, or peer-reviewed research, and those evaluations show null or negative results, the program may not meet Level II even if the vendor’s featured study is methodologically clean.
The Evidence for ESSA database, maintained by the Center for Research and Reform in Education at Johns Hopkins University, provides independent ratings of programs against ESSA evidence tiers across reading and math. It is a practical starting point for districts that want a third-party read before relying on vendor-provided documentation.
Common Misreadings Districts Should Know
A few patterns appear consistently in how Level II evidence gets misrepresented or misunderstood in procurement contexts.
Correlation studies do not qualify. A study showing that districts using a program had higher test scores than districts that did not, without a matched comparison group, baseline equivalence, or appropriate analysis, is descriptive, not quasi-experimental. This type of evidence might support Level IV (evidence of a theoretical rationale) at most.
Pre-post designs without a comparison group do not qualify. Showing that students improved from fall to spring after receiving an intervention does not establish that the intervention caused the improvement. Without a credible comparison group, the study cannot rule out maturation, regression to the mean, or concurrent instructional factors.
A program review is not a study. Third-party program audits, implementation fidelity assessments, and satisfaction surveys are valuable for program management but do not constitute quasi-experimental evidence.
WWC ratings apply to specific studies, not programs in general. A program may have one study that meets WWC standards and several that do not. The rating for the qualifying study does not extend to the program as a whole, and district compliance documentation should reflect which specific study is being cited and what that study found.
Putting It Together for Procurement and Documentation
When evaluating whether an intervention meets ESSA Level II requirements, work through a short set of questions.
Does at least one study use a qualifying quasi-experimental design and meet applicable WWC standards? Is there a statistically significant positive effect on a relevant outcome? Does it meet the large-sample and multisite requirements for Level II, and does the study population and setting overlap with those the district intends to serve? Are there strong negative findings from other well-designed experimental or quasi-experimental studies examining the same intervention and outcome? And was the program studied under conditions reasonably relevant to the district’s intended use?
Affirmative answers across those questions put a program on solid Level II ground. Gaps in any of them are worth resolving before finalizing procurement decisions or including the program in grant documentation.
State education agencies vary in how they handle ESSA evidence documentation requirements, and some Title I and Title III funding streams require districts to certify evidence tier at the point of program selection. Having the qualifying study, the WWC review or equivalent, and the baseline equivalence documentation on file before an audit is a cleaner position than assembling it afterward.
Understanding what ESSA Level II actually requires protects your district on two fronts: it helps you select programs likely to produce real academic gains, and it keeps funding decisions defensible under scrutiny. Getting the evidence standard right is what holds those two goals together.
Sources
- Evidence for ESSA Standards and Procedures, quasi-experimental design criteria and baseline equivalence requirements for ESSA evidence tier ratings.
- U.S. Department of Education, ESSA Title II Part A Guidance, statutory definition of moderate evidence and application to Title II-funded programs.
- Kentucky Department of Education, ESSA Evidence Levels, plain-language summary of ESSA Levels I through IV and qualifying study types.
- IES What Works Clearinghouse, ESSA, WWC standards for meeting ESSA evidence tiers with and without reservations.
- U.S. Department of Education, Guidance on Evidence Standards, federal guidance on using evidence to inform investment decisions under ESSA, including well-implemented study criteria.
- IES, ESSA Tiers of Evidence, current IES explanation of the four-tier evidence framework and study design requirements.
- Midwest Regional Education Laboratory, ESSA Evidence Training, practitioner-facing guidance on applying ESSA evidence standards in program selection and procurement.
