The Myth of Data in Schools

In 2018, one of my high school classmates wrote Educational Inequality and School Finance: Why Money Matters for America’s Students (Baker, 2018) in which he takes a close look at the myth “we are spending more, but getting less” out of our school budgets in the United States. Looking back on more than 30 years in education, I have noticed many myths that are perpetuated on schools and educators’ active participation in perpetuating many of them, so I was looking forward to Bruce D. Baker’s take on this one.

My field is teaching and learning (technology-rich teaching and learning), so I found the chapters explaining school finance and school spending interesting (I have actively avoided certain aspects of school administration over my career). Baker’s writing made a topic in which I have only marginal experience and interest engaging and informative, and his use of, and explanation of graphs, clarified and illustrated many points. (School leaders are infamous for creating and disseminating misleading graphs and misinterpreting the ones they do share, so the data geek in me enjoyed those and they allowed me to move on quickly.) The first several chapter were a quick and easy read, but I was looking forward to the chapter on Evaluating Educational Innovations. I knew he would be treating the topics that are more interesting and relevant to my work, and I was looking forward to a closer read.

I’m not sure of the importance the author puts on one particular passage, but it surely resonated with me and captures a troubling change in education. Baker described his early work in studying school finance in the mid 1990’s, specifically the projects on which he collaborated:

These state-supported research studies included research advancing the application of conceptual models and statistical methods for studying cost-effectiveness and efficiency of local schools. They represented the highest levels of empirical and rigor and were conducted by leading researchers in the field (p. 161).

He continues by identifying efforts in Texas and New York to understand the multi-faceted and problem of paying for schools and applying “statistical modeling techniques to estimate costs” and “statistical tests of reliability and validity of estimates o school performance and efficiency (p. 162). These efforts continued into the early years of the 21^st century; notably these are the same years that “data-driven” became the buzz word of choice in education and we were all supposed to be making decisions and justifying our actions with data.

Baker then make the observation that has been highlighted in my copy of his book. (I also remember reading this section as I spoke to the book out loud; this appeared disconcerting to my seat mate on the airplane at that moment.)

These were the very types of analyses needed to inform state school finance policies and to advance the art and science of evaluating education reforms for their potential to improve equity, productivity, and efficiency. But these efforts largely disappeared over the next decade. More disconcerting, these efforts were replaced by far less rigorous, often purposely speculative policy papers free of any substantial empirical analysis and devoid of and conceptual framework (p. 162).

Baker’s observation supports my own conclusion that coincident with the demand that schools use “data” and “evidence” from politicians interested in education, the bureaucrats who formulate policy, and the school boards and school administrators who make local decisions have been grounding their decisions and actions in speculation.

I am going to avoid describing my personal experiences with pointing out to various leaders that their data and interpretations are dubious. Let me just confirm they do not care for teachers and citizens who point out their data are “speculative,” “free of substantial empirical analysis” and “devoid of conceptual framework.”

While we may be disturbed by the observation that education has taken a drastic turn away from real analysis in recent decades, and decisions regarding school operation and pedagogy are to be grounded in speculation and political preferences, it is at least understandable. Schools are highly political institutions, and all data are interpreted to support one’s political preferences.

This cannot be a surprise. We have known about wicked problems (Rittel & Weber, 1973) and the fact that the solutions to social problems cannot be objectively assessed for decades. When school leaders (at whatever level) use the language of “data” they are misappropriating objectivity, reliability, and validity in an attempt to justify their decisions and protect them from criticism or even question. Politically, this is a reasonable approach; but it cannot be justified as one who cares about research and the insight it provides.

Credible leaders must be transparent: Tell all of your stakeholders exactly why you are making the decisions you are making. If your politics don’t convince them, don’t hide behind “data.” We are on to you.

References