When Research Goes Wrong
By: Julia Daley
In recent years, the problem of “fake news” has become widely known, not just in the USA but around the world. Sensationalist headlines from dubious sources having little basis in reality are virulently shared in social media, spreading misinformation far and wide resulting in sometimes very real consequences. With this month’s issue we’re tackling a related issue, “junk science,” which often exhibits the same fundamental flaws as “fake news,” but also, fortunately, the same solutions: adopting a stance of healthy skepticism, examining the evidence, verifying the credibility of the sources, etc. In short: we need to wear our critical thinking caps when reading about science.
Our two intro videos for this month might cover two different fields of science—health and social sciences—but the conclusions they reach can most definitely be applied to educational and neuroscience research as well. Our LITE video (2016) by Food Insight is a quick examination of the problem of “junk science” as it pertains to the often-contradictory health headlines so breathlessly shared in social media. It’s a useful primer on the many issues that can plague research in any field and the way those results are cherrypicked and shared in flattering press releases.
Junk science is often malicious in its intents. With the internet age, the end goal is often to drive clicks and generate ad revenue. Adding to that is the problem of predatory publishing, where fake journals publish dubious studies with little-to-no peer review and often charge academics for the privilege of doing so. If you’ve ever presented at a major conference, you’ve likely encountered one of these predatory journals; I was hounded for months to “publish my research” in spam emails from people who had clearly not even read my presentation abstract. (And yes, fake conferences are also a thing. What a world we live in!) If academics can have difficulty telling real conferences from fake ones, and reputable journals from predatory ones, this challenge is all the more magnified for everyone else as they try to reach a full understanding of the research they are reading.
Our DEEP listen for the month is a podcast by Radiolab (2017) on the replication crisis plaguing social sciences (the focus in this podcast, though this crisis effects every branch of science). While a longer listen (there is a transcript for those who prefer a longer read), the podcast episode does a thorough job of exploring a particular study from its origins to its current replication problems in an interview with the lead researcher, Claude Steele.
Unlike “junk science,” this was legitimate research conducted in good faith by real researchers and published in peer-reviewed journals. And yet, even here we can often find studies with glaring methodological flaws and biased data collection that, while perhaps not done with malicious intent, still result in “bad ideas” proliferating across entire disciplines. Even when the research itself is sound, the way the results are communicated to the public can result in a distorted understanding of what the studies actually found. These distortions can ripple out into some very real consequences.
Let’s look at an example from neuroscience and how it left a deep imprint on education that is still felt today. I am certain that many of us have encountered the idea of Howard Gardner’s Multiple Intelligences in our careers as educators. It’s a very seductive theory, promising concrete ways that educators can target instruction towards students. Unfortunately, the Theory of Multiple Intelligences is sometimes held up today as an example of a neuromyth, or a belief about the brain that simply hasn’t been fully supported with empirical research (though Gardner argues that his theory doesn’t qualify as a neuromyth, and that it was never meant to be applied to educational settings). Yet still, the now discredited theory of teaching to VAK “learning styles” connected to multiple intelligences, continues to proliferate in classrooms around the world. Other popular ideas among educators that are losing credibility with scientists include power posing, mirror neurons, mindsets, and more.
We live today in what is sometimes called the “Information Age,” and there is just so much research available now at the tip of our fingers. It can be difficult to distinguish the good from the bad, the real from the fake. Access to original research is often blocked behind paywalls, which can prevent many from reading studies carefully, leaving access only to the often-sensationalized summaries presented in news media. Even when the original research is freely available, it is typically presented in dense academic jargon that intimidates the non-initiated and obfuscates the results for anyone unfamiliar with the language norms of a given field.
Another problem, specific to educational neuroscience, is that there is a flaw in the entire premise that neuroscience can be used to inform classroom procedure. Willingham has been shouting at us about this problem for years. We still must try to inform our classroom practice with research, of course, but we must also remember that trying to understand how students act by looking at their neurons is about the same as trying to understand how a car works by looking at its molecules, and that what may work in one context with one classroom and with one teacher may not work in another context.
Sifting out the wheat from the chaff can be time-consuming, but if anything, we hope that this month’s issue will stress to you just how important it is for us all to become savvier readers of research. We provide lots of examples and strategies that you can use to help make this process easier. Perhaps you already engage students in lessons on critical thinking to help them navigate the world of disinformation we currently live in—now, it’s time to apply those lessons to yourself as you read educational research!
Julia Daley is a lecturer at Hiroshima Bunkyo University, where she teaches English conversation and writing. She earned her MA in TESL at Northern Arizona University and is certified to teach secondary English in Arizona. Julia enjoys reading about neuroscience in all its forms, particularly as it pertains to Working Memory and Cognitive Load Theory. She appreciates everyone’s patience as she’s been learning how to build a website.