Data be damned

When will you learn that there isn’t a word for everything?
– Nicole Krauss, The History of Love

Consider the following. In his book, Getting It Wrong From the Beginning, author Kieran Egan tells of a study in education, conducted in the 1990s, that concluded the following: “To develop competence in an area of inquiry, students must (a) have a deep foundation of factual knowledge, (b) understand facts and ideas in the context of a conceptual framework, and (c) organize knowledge in ways that facilitate retrieval and application”. (168-9) Sound familiar? Does anything about those findings bother you?

For the record, Egan has been extremely valuable to me as I have tried to articulate the many problems I have had with reform in education as I have seen it practiced both throughout my career in the classroom and while I was a member of the provincial government. In both instances, I watched enormous resources brought to bear in ways that have, to my mind, done little more than further complicate a situation that was exceedingly complicated to begin with.

So what about the study Egan notes? If you are lost within the language employed in reporting the results, wondering what the problem is, or simply curious, consider the following: what does it mean to be “competent in an area of inquiry”, maybe botany, let’s say? I admit I don’t know much about the subject but someone judged competent would, I think, need to know many things about plants since botany, by definition, is the science of plant life. A working knowledge of biology would seem essential as well since botany is a more specialized pursuit within the larger field. Finally, if someone were judged competent, I think we would need to see proof of some kind, whether through good advice on how to grow my geranium or an explanation, maybe, of why some plants need more sunlight than others.

I’ve tried to abbreviate an example that Egan takes far more time explaining but I’m hoping my version will suffice. The study I’ve noted represents just one brand of educational research with which I have a problem. If you take a look at the “conclusions” reached through the research, the “findings” could quite properly appear in a dictionary defining “competence in an area of inquiry”. In other words, a study was undertaken to prove that competence is properly defined as competence. While I cannot point to a specific “study”, I’m pretty sure there are any number of them out there that have spent enormous sums of money and considerable time proving that young people who are read to frequently and who are raised in a home where a great deal of reading material is available are more likely to be good readers.

The fourth chapter concerns the teacher and the classroom and notes that an indisputable conclusion of research is that the quality of teaching makes a considerable difference in children’s learning.
– from the abstract for “Becoming a Nation of Readers: The Report of the Commission on Reading” (1985)

If any or all of these examples strike you as absurd, welcome to one manifestation of educational research: that which purports to lend the weight of scientific validation to an already established definition or a commonly understood reality. The fact that someone perceives a need to provide such a thing is as frustrating to me as the fact that it can gain acceptance and validation among those commonly tasked with making decisions that have an impact on so many facets of our public schools. The determination to forego any comment on educational practice unless “data” accompanies the comment leads naturally to my next point.

If, like me, you have spent any time trying to offer alternatives to certain practices and/or trends in education these days, you have, no doubt, been questioned regarding data. The thing is, I don’t really mind the question; rather, my issue is how one defines “data”.

Some 7 or 8 blogs ago I started in on this issue and I warned any potential readers that this was going to be long. Now, however, I can circle back to a point I tried to make early in the argument: avenues to understanding and to action should not be restricted to those things bounded by a model of research that is either ineffective or inapplicable. Education has been subjected to a bevy of research that imagines “learning” can be understood in much the way a physicist might determine the conditions under which water can be brought to a boil. Can it?

boiling water
In the latter example, the scientist can do something that, to my mind, is impossible in education: control the variables. Even then, the identification of the variables can be more complicated than we might assume: what vessel is being used to hold the water? At what atmospheric pressure is the experiment being conducted? What is the temperature in the surrounding environment? My guess is, a proper physicist might have a few more questions that I wouldn’t think of.

In the educational example, how can we even begin to identify the variables? The scientific method demands that a hypothesis, followed by observation, lead to some statement regarding the truth or falsity of the hypothesis. In social science research, rather than an absolute proof, trends and/or possibilities might be the result. The problem remains the same, however: supportable results, within the necessarily limiting confines of traditional scientific research methods, demand a control of variables. In something as complex as a classroom, how could that ever be deemed possible?

One technique is to narrow the focus. Consider: a special class is established to help students who did poorly on a standardized exam, specifically to prepare them for the next administration of that exam so that they might do better. Lo and behold, a year later, students rewriting the exam do better. The conclusion? The intervention (creation of this special class) worked.

I won’t even try to list the many factors which might have had an impact on the improved results. Suffice to say that, to my mind, claiming that the improved results are directly attributable to the class that was established is roughly equivalent to suggesting that a certain team won the Stanley Cup because the guy who scored the crucial goal just got married and that made him a better goal scorer.

In the quotation I provided at the outset, a character in Nicole Krauss’s novel questions whether we can really have a word for everything. Are there elements of human experience that are truly inexpressible? Personally, I think there are but that’s a topic for another time, perhaps. I chose the line because it hints at another problem: what if we give a name to something without being entirely able to say exactly what we intend, even when we have a definition? “Education” is just such a word.


2 thoughts on “Data be damned”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s