Knowledge is Power! Measure what Matters! If our cliches are an indicator, we all know that data collection, review, analysis, and understanding is important. We all hear of data-driven decisions, and the importance of data in education and educational systems, but we are often challenged to incorporate data review and the next steps into our everyday lives. As we’ve been exploring the Essential Elements of a Data Culture, we’ve been considering how an organizational culture can shift from a culture in which data is in the periphery, pulled to the center for high stakes discussions and decisions, to one in which data is an integral part of every day, informing the small moves that reinforce the vision, clarify decisions, and advance progress. This is where our love of habits comes in…
For more than a century, standardized testing data have been used to measure the success of students, teachers, and schools - and even to mark our global competitiveness or lack thereof. These data have driven significant education policy and funding models at the national and state levels, and school districts devote up to 15 percent of their instructional days each school year to student assessments alone, costing an estimated $1.7 billion each year. The political and financial commitment to standardized testing was born out of good intentions. The incredibly high stakes for students, teachers, and schools that were tied to these data were intended to hold us accountable for educating all children. But the return on these investments is debatable at best. We know now that standardized testing data, when viewed in isolation, represent a limited view of student success and can even mislead us into making discriminatory decisions because of their inherent biases. We know the policies we’ve enforced and the decisions we’ve made based on these data have failed to close persistent achievement disparities across income levels and between white students and students of color, even after 50 years of trying.
Get free weekly tips and advice designed for leaders like you.
As a new mom, I cared deeply about two things: my baby’s well-being and my sleep. I could go without showering or hot meals, but I was NOT well equipped to deal with the lack of sleep. And I was fairly lucky - my son slept about as normally as you can expect a newborn to sleep. However, any disruption in that pattern, and I immediately scrambled: “He didn’t sleep last night. I MUST try these five new things to get him back on track.” Sometimes they worked, and sometimes the shift in routine actually made things worse. My hyper-focus on the short run cues meant that I was super reactive to one piece of information but failed to take a look at the big picture. Conversely, there were times I didn’t have the energy or brainpower to try new things – I ignored the information my son was giving me because I simply didn’t know how to use the data or what to do.
On January 28, 1986, the space program experienced one of its most catastrophic events to date when the Space Shuttle Challenger broke apart just over a minute after launch. All seven crew members died, including Christa McAulliffe, a school teacher who would have been the first teacher in space. If you’re familiar with the event at all, you know the accident was caused by a failed O-ring seal in the solid rocket booster. What’s less widely known is that, according to the recently released Netflix documentary, Challenger: The Final Flight, NASA and the company that manufactured those O-rings had information available to them that day that could have led to a different outcome. For example: The O-rings were a known problem. In many of the successful launches using solid rocket boosters prior to the Challenger, there was evidence of damage to the O-rings during launch. The temperature the day of the launch was much colder (by at least 20 degrees) than typical launch days. More than one expert at the O-ring manufacturer voiced concern that the part had not been tested at that temperature and could fail. NASA made choices about the data they used that day. They went into their decision-making process with a bias (they were motivated to launch after a series of delays), and they failed to see how that bias motivated their choices and in turn influenced their behavior. In education, we make choices about our data, too.