Knowledge claims, questions and how to avoid egocentric inferences 

Written by Alex Black

Recently whilst creating a bridging lesson about correlation and probability, I came across two knowledge claims which I believe have led to much confusion.

The first claim was made on October 27th when Imperial College London produced a preprint of one aspect of their REACT study.

“COVID-19: Public immunity “waning quite rapidly” 

Some 365,104 adults took part in three rounds of testing for the study between late June and September to measure the prevalence of coronavirus antibodies in England.

The study found that antibody levels fell by 26.5 per cent overall during the three-month period.

Source: Plus 300 other sources using this exact phrase.

Then just a few days later came another headline announcement in the almost daily flow of optimistic vaccine announcements.

‘Absolutely remarkable’: No one who got Moderna’s vaccine in trial developed severe COVID-19 By Jon Cohen Nov. 30, 2020, 7:00 AM

“That is an efficacy of 94.1%, the company says, far above what many vaccine scientists were expecting just a few weeks ago.”

Indeed many commentators are now talking about the fact that we seem to be getting quite complex scientific and medical  knowledge claims by press release.

The bridging lesson I am working on follows on from CASE lesson 20 Treatment and Effect, which is based on correlation reasoning patterns. The lesson also makes clear how important an understanding of probability is in everyday life, especially when trying to understand media claims.

The first claim that “COVID-19: Public immunity “waning quite rapidly” was understood by many to cast some doubt on the hope that a vaccine for covid-19 could be possible. Several people I discussed this with made these kinds of inferences from this claim. “So if I lose 25% antibodies every three months by the end of the year I will have none and so will have to get a vaccine every year.”

It needs to be pointed out that the study was done with home antibody testing kits that gave a binary answer Yes, you have antibodies against Covid 19 or No, you do not. Obviously the test kits are calibrated in such a way that a certain threshold levels of antibodies is needed to give a positive. These types of tests are often explained as similar to the well known home pregnancy test. Of course pregnancy is something that lends itself to a binary description whereas antibody levels do not easily fit. The study did not actually really measure anyone’s antibody level directly, as that would have been prohibitively expensive.

A second CASE lesson 18 Tea tasting deals with some of the problems related to how such binary data is to be used and how the role of chance has to be ruled out.

The REACT study, from which came the claim “COVID-19: Public immunity “waning quite rapidly”  is being done on a massive scale which has dictated some of its methodology and their limitations.

Clearly the question that the study designers were directly asking was something like this: What is the probability that a representative sample (of some 365,104 adults) of those that get a positive Covid antibody test will be the same after 3 months as a similar representative sample (from these some 365,104 adults)

Not a great news headline but it allows experts to give some useful information for policy makers. However the claim needs to be understood as a probability based assertion that applies to a large population sample.

When interpreted in a personalised way the news broadcast shorthand could easily lead to inferences that lay unjustified doubt and a certain hopelessness.

I would like to argue that the claims made in the media are usually an answer to some scientific or societal question which is the result of expert methodology. This methodology is usually statistical and any result must be by its nature a probable answer.  However these claims are often stated in a way that often elicits an over simplistic and over personalised interpretation.

To explore these claims in a more productive way and achieve a greater deal of insight I propose these simple critical thinking questions.

  1. What was the actual question the researchers were trying to answer?
  1. How did they go about getting evidence to answer the question?

These are typical of moves to mediate student understanding that are part of the craft of being an effective Let’s Think teacher. They will also allow for a thorough set of opportunities to create a common understanding of the complexities and different intentions of communication. The two media headline examples demand difficult and complex statistical thinking to decipher what the claims are. This is the type of formal operational thinking that is challenged during Let’s Think lessons.

In conclusion I feel another strong claim that can be made is if learners habitually pose the two key questions stated previously and have opportunities to socially construct an understanding of these complex matters, then they will be less susceptible to unjustified inferences and egocentric interpretations.