More than One Metric Matters in EdTech
When it comes to edtech I agree with the recent Forbes magazine article that student outcomes matter. Making a difference for students is why I first became an educator, then a school leader, and why I work in an edtech start-up. As the newness and shininess of edtech tools begins to wear off a bit, it comes as no surprise that everyone is starting to focus more on the return on the investment, rather than just talking about the investment itself. As a country we’ve invested millions through Race to the Top, venture funding, and district-level purchases of both hardware and software, and the efficacy of these investments and the connection between inputs and outputs as a whole remains unclear.
I think about this challenge in two ways. The first is that outcomes do matter and we - educators and entrepreneurs - all need to focus on them. Our organization, Education Elements assists districts develop and implement personalized learning strategies. Last year, in our partnering public school districts, we saw tremendous growth in student gains across the country. Across over 5000 students and 9 districts, blended learning classrooms supported by Education Elements are seeing 25 percent growth in math on NWEA Measures of Academic Performance (MAP) above national norms , and 54 percent growth above national norms in reading. In one upstate New York district, for every 100 students, 25 more students hit their growth targets in blended learning classrooms than in the non-blended ones. As a company, these are some of the things that get us most excited - we are inspired when we see tangible improvements in achievement as a result of personalized learning.
Secondly, before starting to personalize learning (or doing any sort of edtech or other school transformation effort for that matter), districts need to know what outcomes they are hoping to achieve. It is critical that the goal of personalized learning is not simply personalized learning. Not even just “improving outcomes” is sufficient. Instead districts need to align on a vision and a theory of change. They need to know why they are doing it, what the goals are, and how they will measure success. This first step is so often skipped in the rush to buy devices or purchase content. Yet without knowing what success should look like how can anyone know when they have achieved it?
Some districts, like the Enlarged City School District of Middletown, have a theory of change around their blended learning implementation; - increased small group instruction, student reflection, regular use of data to inform instruction, and the use of digital content - will lead to better student outcomes.
This in turn leads them to focus their efforts on measuring – for instance - their progress to increase small group time and engage students in thoughtful student reflection. How are they training teachers to engage in instructional models that provide adequate small group time? How does the existing coaching and walk-through tool support these changes? How are teachers and other educators qualitatively and quantitatively measuring fidelity to this theory of change so they can iterate towards success? So, at the same time that Middletown looks at scores on standardized tests, they are also looking at data from surveys around student teacher satisfaction and effectiveness.
Similarly Uinta County School District #1 in Wyoming -- like many other districts -- is focused on improving student achievement, increasing student independence, and increasing student ownership of learning. And, yes, they have invested in new digital content products, and yes, they have bought devices. But for Uinta, a critical piece is building consensus around a clearly defined theory of change of what will lead to improved outcomes. For Uinta, consensus around their innovation configuration - a tool to assess progress - has been essential. That is to say, beyond the technology, Uinta has invested in changing mindsets as well as investing in the necessary tools and support to result in outcomes.
Middletown and Uinta are just two examples of how districts can develop a theory of change around personalized learning and then focus on addressing how to best support and measure that vision. Both demonstrate the importance of not just the student outcomes, but also the approach. The power of edtech is not the promise of increased efficiencies or about squeezing marginal outcomes for students, the real opportunity for districts is the potential for a very positive transformation for teachers and students - one the enables lasting change, one where the connections between edtech, pedagogy, and outcomes are clear.
For more results, check out our document "Results Matter": 2014-2015 District Results Report.