Most Mask Studies Are Garbage
Most Mask Studies Are Garbage
Corona research suffers from such massive publication bias that you can't trust any on-message findings at all.
As far as masks go, there are basically two kinds of studies.
There’s the kind where they do variations upon getting people to cough without cloth over their mouths and then with cloth over their mouths, and then compare how much stuff is expelled. Invariably, they discover that if you cough through cloth, you spray less stuff. It is hard to believe that anybody could ever care about stupid research results like this, but alas they do. Until recently, most of the research that mask brigade had in their corner was of this variety. As scientific evidence to justify community mask mandates, it was an embarrassment – the equivalent of proving that wood floats when called upon to show that a ship is seaworthy.
The other kind of study, the proper kind, would be a randomised controlled trial. You compare the rates of infection in a masked cohort against rates of infection in an unmasked cohort. Here things have gone much, much worse for mask brigade. They spent months trying to prevent the publication of the Danish randomised controlled trial, which found that masks do zero. When that paper finally squeaked into print, they spent more months trying desperately to poke holes in it. You could feel their boundless relief when the Bangladesh study finally appeared to save them in early September. Every last Twitter blue-check could now proclaim that Science Shows Masks Work. Such was their hunger for any scrap of evidence to prop up their prior convictions, that none of them noticed the sad nature of the Science in question. The study found a mere 10% reduction in seroprevalence among the masked cohort, an effect so small that it fell within the confidence interval. Even the study authors couldn’t exclude the possibility that masks in fact do zero.
A lot of people asked me to comment on the Bangladesh study, but the truth is, I can’t bring myself to care. In fact, I think it’s bad mental hygiene even to spend time thinking about it.
The reason is that a pervasive publication bias weighs upon this entire area of inquiry, which is why the authors of that Danish study had such a hard time. There is enormous pressure, always and everywhere, to produce findings that Corona containment measures are effective, and enormous bias against findings that Corona containment measures are not effective. When strong biases like these emerge, they taint all of the published research to the point that you can’t trust any of it – not even the good-faith, high quality stuff.
Over the past year, you have to imagine that untold numbers of researchers around the world burned through millions and millions of dollars in grant money trying to prove that masks work, with studies of all kinds. The studies that failed to prove they work were mostly junked. The Danish researchers were rare in their persistence: Why fight battles to get unpopular results into print, just so you can earn the opprobrium of the I Love Science crowd and have your research smeared by prominent public health mandarins and epidemiologists with named chairs at major universities? It’s safer to throw everything in the trash and try again with a new study.
Keep this up for long enough, and you’ll eventually get one or two studies like the Bangladesh one, that show a small effect for masks. But because you’ve subjected all research on the topic to systematic bias, you can’t know what the results of these studies even mean. If you run experiments over and over enough times, throwing out the results you don’t like and keeping the ones you do, you can find evidence of anything. You can produce high quality studies proving that rain dances cause rain if you want to. Researchers of paranormal phenomena have been doing this for decades, and by now have built up an enormous body of literature proving the existence of things like telepathy and distance healing.
What we don’t know, is the ratio of junked failed telepathy studies to successful published telepathy studies, so in fact we know nothing at all about telepathy. The same is true for masks, and lockdowns, and all the rest of it.
For evaluating the success or failure of Corona suppression measures, there’s only one tool: You have to look out your window and see if they’re doing anything obvious. Do places with mask mandates have better results than places without them? Do places with high vaccination rates have lower Corona mortality than places with low vaccination rates? In every case, the answer to these questions is very obvious. None of this stuff has any effect on the overall trajectory of the pandemic at all. That’s one of the reasons it all has to be mandated and enforced from on high in the first place. If any of these measures did have an obvious mitigating effect on infections, people would adopt them spontaneously, in the same way they wear gloves and hats in winter and drink coffee before work.
Someday, far in the future, we will have real research that we can trust on the nature of SARS-2 and the true costs and benefits that our campaigns of virus suppression have had. But those days are a long way off, and right now it’s simply impossible to evaluate the significance or meaning of any on-message studies at all.
Never forget that most published research findings are false.
UPDATE: For more on the Bangladesh study specifically, see the work of my friend William Briggs here. Note also his posts on the Danish mask study here, and on the worthless CDC mask-mandate study here.
I am an old timer in stats in data, both academia and industry, and I completely agree. I think this dimension of bias is the least understood in science.
I used to explain it as an axiom: if a group publishes a finding, but their careers would be destroyed by finding anything contrary, then that paper contains zero informational value.
I have friends with faculty appointments who were told on day one, your job now is to bring in grant money to this university. Like a law partner who is supposed to make it rain. I don't know why anyone thinks someone with those incentives, will go against any headwinds in pursuit of truth. Some do anyway, but they are exceptional people.
And like you say, too few people understand that typical statistical methods only rule out chance to a certain degree. This always means if enough studies are run, false positives will appear. I remember reading about a drug that was finally approved after 33 trials! More evidence something is a placebo is not possible.
This is also why research is meaningless until there are independent direct replications. Basically everyone understands this since 2011 but no one does it, because, see axioms and incentives above.
I work at a working class setting where the bosses demand everyone wear a mask. Due to the nature of my job, I get to visit new such locations almost weekly. One thing I’ve noticed at all of these places is that absolutely no one ever wears a mask during breaks when only their co-workers can see them. The people know, and they know that the others know. One of the things I hate the most about these stupid mask mandates is that they require people to behave in a fundamentally dishonest way.