Gary Miron and the National Education Policy Center (NEPC) are back with a new report purporting to demonstrate blended learning as ineffective at improving student achievement. This new iteration, “Virtual Schools Report 2016: Directory and Performance Review,” written for NEPC by Gary Miron and Charisse Gulosino, claims to be a census of virtual and blended schools, providing data about their composition and performance.
Generally, the report claims to find that virtual and blended schools have significantly different demographics than traditional public schools and that virtual and blended schools are failing academically.
Instead of being a valuable contribution to the research base on virtual and blended schools, this report fails in three major ways:
- The report addresses only a small number of schools using blended learning, and is therefore significantly incomplete;
- By failing to use essential measures - including, but not limited to students’ academic growth over time - to review school performance the report’s findings are misleading;
- Ignoring other prominent research and evidence of successful performance produces flawed findings.
The report considers only 86 blended learning schools – a narrow sampling chosen for unclear reasons. A simple review of helpful resources like the Christensen Institute’s Blended Learning Universe school directory shows that there are over 230 traditional public and public charter schools implementing blended learning in the United States.
By excluding so many schools - many of them high performing - the report’s data and subsequent conclusions are negatively biased and misleading.
Perhaps the authors misunderstand the nature and variety of blended learning models. Even so, it is quite hard to understand the underlying assumptions used to include the 86 schools in the report, which is a ridiculously small number of schools on which to base any conclusions.
This definitional confusion and incomplete school data-set in the NEPC report leads to the very confusing inclusion of Rocketship blended learning charter schools, but the glaring exclusion of such blended learning charter schools Aspire, Carpe Diem, KIPP, and Summit. This discrepancy is not acknowledged or addressed, which is conspicuous in context of the authors' pointed criticism of Rocketship; it makes the report’s conclusions about Rocketship - and all blended schools - highly suspect.
There are very few blended learning charter schools more heavily scrutinized than Rocketship, and systematic reviews have found mostly strong impacts on student outcomes (see here, here, and here for analysis on Rocketship’s results from different perspectives).
Inaccurate & Missing Data
The NEPC report purports to show a dismal picture of student outcomes in blended schools, but another major flaw of the report is that the measures of “performance” it employs are based primarily on metrics that may reveal more about student background than the quality of the school.
The outcome measures used in the NEPC report (state accountability system ratings, the percentage of students that score proficient on state tests, and high-school graduation rates) are incomplete indicators of school quality.
Rigorous efforts to measure school quality should focus instead on the growth in individual students’ scores on standards-based assessments year-over-year. By focusing on what students learn over the course of the year, growth measures are significantly better than an average, point-in-time test score (i.e., the percentage of students scoring above/below an arbitrary “proficiency” bar).
A useful measure of the effectiveness of a blended school would compare the achievement growth of students at that school to the performance of students in the schools those students would have otherwise attended. Finally, the tyranny of averaging can mask important differences and outliers, which are not given any attention or analysis. We need to study the highly performing schools to learn more about what works and why.
Contrary Research Findings
Regarding the avoidance of any evidence that may point to radically different conclusions, the authors ignore that there are a number of high-quality blended learning schools that are producing powerful gains in student outcomes.
Indeed, the emerging evidence about blended learning is very encouraging, as academic research and case studies of specific schools show that students benefit when blended learning is used effectively. One notable meta study published in 2013 from the well-respected SRI International says, “The advantage over face-to-face classes was significant in those studies contrasting blended learning with traditional face-to-face instruction.”
When implemented comprehensively and with fidelity, personalized blended learning has been shown to produce significant learning gains for all learners, especially at-risk students in poverty and/or learning English.
There is some new, promising research from the RAND Corporation: “The longer students experience personalized learning practices, the greater their growth in achievement,” according to their new research. The report, entitled “Continued Progress: Promising Evidence on Personalized Learning,” is an important contribution to understanding whether personalized learning is producing results and how it is being implemented.
The achievement outcomes identified in the RAND study are strong and indicative of the great potential for personalized blended learning to be transformative. Other worthy contributions to the research base come from the Michael and Susan Dell Foundation and the Christensen Institute, which looked at schools and districts implementing blended learning.
Looking Past NEPC
This fourth annual report on full-time virtual and blended learning schools by NEPC and Miron is a classic case of selective reporting to support preconceived findings. The headline grabbing recommendation of the report is that policymakers should stop - or slow - the growth of virtual and blended schools until the reason for students' poor performance is found and fixed.
This is a remarkably consistent finding that has persisted in almost identical language over the last 5 years Miron has been writing about virtual and blended schools. The only difference this year seems to be the addition of blended learning schools to the report’s “analysis” and spurious recommendations.The NEPC report does get one important point right: the need for more research and better data about blended (and virtual schools). I look forward to further research that aims to better understand the successful ways schools are blending learning to produce transformative outcomes for all students.