Facts, data and educational evidence

License Creative Commons

Xavier Massó

Let’s start at the beginning: “facts” do actuallty say nothing, they are simply there, that’s all. What does tell us something and makes us notice them is the criterion by which we group them into data, into contextualized information, and from which we issue statements about the domain in which we have integrated them. But as “pure” facts, they say “nothing”. Let’s not forget Kant: intuition without the concept is blind, the concept without intuition is empty. Either that or we go headlong towards intellectual intuition.

It is when we make a judgment about reality that, contextualized in it, facts acquire a meaning by becoming data. A judgment whose (formal) validity will depend not that much on the data but on the criteria by which we have proceeded ourselves when grouping them, and whose (material) truth will depend on whether such judgment is adequate to the state of things on which we are pronouncing. Whether such judgment is “true” or not in the classic sense of its adequacy to the state of things it states is, therefore, contingent and requires a duly validated contrast, always in accordance with the criteria with which we have contextualized these “facts” and in wich, as data that provide us with information, they acquire significance. Something like putting order in chaos.

But whether this “order” accounts for reality or not is something that will depend on the adequacy of its forecasts to the state of things it states, which must be verified, falsified, contrasted or whatever we want to call it, by means of some type of verification adjusted in turn to the logical procedure proper to the criterion of validity adopted. In other words, if there is no verification, all we have is noise, mere flatus vocis. A theory may be wonderful, but if what it states de iure does not correspond de facto with what is predicted in it, then it must be rejected or revised.

Contrary to what is sometimes assumed, the Scientific Revolution of the 17th century – Galileo, Descartes… – and the subsequent emergence of the modern concept of science does not mean at all the primacy of facts, but of the treatment we apply to them as data and to the extent that they are significant according to the criterion of validity imposed by a method, whose predictions must be validated by the appropriate “experiments”. And the criterion is not just the mathematization of reality, but that reality is expressible in mathematical terms. Something that may not always be plausible.

Thus, Galileo stated that all bodies fall with a uniformly accelerated movement regardless of their mass and weight. It follows then that a cannonball and a chicken feather, thrown from a rooftop at the same time, will reach the ground at the same time. But this does not seem to be the case according to common experience. Thus, those who felt reinforced in their arguments, or so they thought, against Galileo’s experiments were precisely his adversaries, the Aristotelian physicists, because the cannonball falls faster than the chicken feather. Did this mean that the theoretical Galileo’s construction was invalidated because its predictions did not come true? Only in appearance, because what it was significant for him in his experiments was not so to the Aristotelians, or it was so in another sense. Simply, although they saw the same thing, either of their approaches to the phenomenon where based on different criteria, so the significance of the data used was different too.

It is not that Galileo’ statement did not fit reality, but just that it didn’t seem to fit a certain reality, that of the Earth, which, on the other hand, did seem to coincide with the Aristotelians. Although the empirical data were ultimately the same, there were two notions in Galileo – and others that we will not go into so as not to be excessively long-winded – that led to the same “data” being interpreted in a different way, namely, the force of the Earth’s gravity and the notion of “vacuum”. The first one was far beyond the Aristotelians intelectual landscape; the second did not mean the same thing to them as it did to Galileo. Of course, Galileo was absolutely aware that the cannonball fell faster, but his explanation was very different from those adduced by the Aristotelians.

To Galileo’s point of view, the results obtained with the corrections he introduced into his experiments in order to minimize as far as possible the effects of air resistance (or friction), were proof that with no air resistance and just considering the force of gravity, all bodies in free fall would experience exactly the same acceleration. For the Aristotelians, on the other hand, the results of such corrections were irrelevant or, in any case, explainable by ad hoc hypotheses and taken as a peculiarity. Galileo understood it just the opposite: the particular case of the universal law was that of the Earth with its atmosphere. Two very different ways of seeing the same “facts/data”. Although neither of them saw the air, Galileo contemplated its effects The scholastics, on the other hand, asked him where he got this silly idea about air resistance from.

It is clear then that, with the disputed issue duly settled in Galileo’s favour, the “data” that  Aristotelians understood as “evidence” in their favour were only so to the extent that their perspective was biased by their own criteria, paradoxically much more “empirical”, insofar as it was based on sensible experience, than Galileo’s “ideal” pattern. What interests us here in all this is that the “data” that  was “evidence” to Galileo, so was it to the Aristotelians, but in the opposite sense.

So, the question now would be to what extent what we call “evidence” in education are truly so and to what extent according to which educational discourse we are speaking from? And like the Aristotelians, aren’t we perhaps leaving aside educational equivalents to air resistence or gravity because “they are not seen” even though their effects are there, and would we not be instead adducing mere ad hoc hypotheses? Actually, everything seems to indicate that this is the case on too many occasions.

In his excellent ‘Real finish lessons [1]’, Gabriel Heller Sahlgren presents us with a case that shows us to what extent it is possible that this might be really happening. That is to say, that even with such thoughtful interpretations and “metadata” analyses, we are missing something that is right in front of us, because it does not fit into the criteria under which the question is being approached, like air resistance and gravity in the case of Aristotelian physicists.

Let us take the principle “less is more”, the “motto” with which Finland’s educational success was summarized in the early days of PISA, spread by one of its most enthusiastic and conspicuous educational experts, Pashi Shalberg [2]. With the statement “less is more”, and given that the Finnish education system had fewer classroom hours than most of the other countries involved in the PISA tests, what was intended to be exemplified was that fewer teaching hours and less content, less demand, in short, produced better educational results, this being the “secret” of the Finnish model at the time when it was a mandatory reference and an example of educational success par excellence.

Let us momentarily leave aside the possibility that we are dealing with a crude fallacy such as “post hoc ergo propter hoc”. Having admitted that with fewer hours Finnish students obtained better results than those of the other countries, we can legitimately ask ourselves if there is a cause/effect relationship, but also if these better results could be due to other factors, as it can be deduced from reading Heller-Sahlgren’s book. In fact, the only thing that can be inferred from this would be that the number of teaching hours is not (always) a factor that ultimately determines academic success. Firstly, because we are not talking in linear or absolute terms, but in comparative ones; that is, compared to other countries, whose socio-economic and cultural reality may be very different from that of Finland, not only in terms of teaching hours and results. In other words, a planet that has atmosphere is not the same as one without, in terms of the acceleration in free fall of any body in its surrounding. Something that Mr. Shalberg does not seem to have taken into account.

And secondly, because, even without considering the use that is made of these teaching hours in each case, it does not seem legitimate to infer, as if it were a cooking recipe whose proportion of ingredients and cooking time is inalterable in the slightest, that increasing teaching time would result in a deterioration of the results. Nothing authorizes us to think this, nor despite Mr. Shalberg may believe it.

And as Heller-Sahlgren points out, it is possible that in all this we are leaving out some factors that are as significant and perhaps more “evident” than the simple relationship between teaching hours and results. Some absences that, far beyond the mere propaganda slogan, since being ignored would remain unnoticed as a cause of any effect. In short, we are not considering other equally concurrent variables such as, for example, motivation.

Let’s imagine a country “A” in which, no matter why, students are highly motivated and, with fewer hours of class, they obtain better results than those of a country “B” with more hours, but whose motivation is lower or much lower. What would it be then more reasonable to be attributed as the cause of the different results? What on earth would make us think that by reducing the number of hours in country B, their results would improve, or that those of country A would worsen if we increased them? What “evidence” can lead us to think something like that? Only one, the principle less (hours) is more (results). We’re not going to enter into the subject of positive and negative reinforcements, let us assume, anyway, that if someone is motivated, he will try harder and with more interest, in and out of classroom, and if he’s not, he will do so to a lesser extent. But someone might say that this is not a statement supported by evidence… Sorry, it is.

Unfortunately for Pashi Shalberg, evidence has not proved him right since this debate did not remain in the realm of mere rhetoric, but got far beyond: the “evidence” is that currently, with the same “fewer” hours of teaching as in the old days of educational glory, the Finnish results in PISA, after fifteen years of a continued and sustained decline, have noticeably worsen. And this allows us to state that the premise less is more was simply false. If something was going well before and now with the same hours it is worsing, then it was not because of the hours, but because of something else, whatever it may be, but something else.

Neither Galileo nor the Aristotelians could see the air, but the former noticed it in its effects. The same thing happens with motivation and effort. It might be very difficult to be quantitatively and objectively expressed, but not so its effects. Whatever its causes are, motivation has effort as an effect, and likely improve results. What is not worth it is to ignore it and tortuously supplant it. We don’t like to talk about effort and demand, do we? All right, no problem. Will we continue to ignore it and resort instead to alleged “evidence” that is nothing but ad hoc fixes? It never mind. Do we prefer to continue to treat students pitifully as if they were stupid, don’t we? Of course, things will go on by themselves… Let’s keep unnoticed what’s being noticed on his effects…

But then we must also agree that the educational model, the governments that promote it, the hegemonic pedagogies that inspire it and the experts who devise them are at the same level as the Aristotelian scholastics who opposed Galileo, either because they do not see or because they do not want to see. Nevertheless, as Galileo once said, Eppur si muove.

___

[1] Gabriel Heller Sahlgren, ‘Real Finnish lessons. The true story of an education superpower’. Centre for Policy Studies, London 2015. There are translations into Catalan and Spanish, carried out by the Fundació Episteme, which can be downloaded free of charge from the foundation’s website: https://fundacioepisteme.cat/2023/01/25/les-autentiques-llicons-finlandeses/  and https://es.fundacioepisteme.cat/2023/01/25/las-autenticas-lecciones-finlandesas/

[2] Pashi Shalberg. Finnish educational expert, enthusiastic advocate of the “less is more” approach,  promoter of educational reforms carried out in Finland and advisor on educational issues in countries such as England and the USA; author of ‘Finnish lessons’ (Teachers College Press, New York, 2011). Sahlgren’s book is a reply to the Finnish educational good news spread by Shalberg, as its title indicates.

Source: educational EVIDENCE

Rights: Creative Commons