If you’re at all like me you battle your nights away with burning questions. Questions like “how do I know that study X is actually an accurate representation of the natural world?” or “how can people make policy decisions based on junk science?” or “I wonder if the herrings communicating by farts* study is replicable?” Or you’re not, then you don’t and you likely live a happy life unperturbed by the terrorizing thoughts of validity and replicability. But misery loves company – and you’re still reading. So, welcome aboard.
Not all studies are created equal. Not that anyone wants you to know that. We all get bombarded with scientific information, constantly – from your Facebook feed to four out of five dentists choosing Colgate Toothpaste on the afternoon
The truth about “facts” is, as usual, more nuanced, more dark and often more foggy. There are THOUSANDS of peer reviewed scientific journals out there and each cycle they churn out the work of the scientists. Naturally, information gets cherry picked – a little piece of fact here, another fact there – and spread throughout our collective psyche. This cherry picking leads to conflict and confusion as many findings will contradict each other. The bugger is that both are often right.
But, not all studies are created equal. There are top tier journals, and second, and third. Why? Because science isn’t created equal – some studies have better procedures (meaning less chance of holes in their logic) than others – and some studies have findings that are more robust (stronger) than others. So how do you sort through all this? The answer isn’t always as fun as quoting random statistics off the internet.
You must do a bit of work – read a bit more. For every newspaper article quoting a study they should cite the source of that information and you should go find it (I like http://scholar.google.com). And that’s the key – find the original, then read it. For a TLDR version of the article go to the end as you’ll find a well laid out summary including the limitations of the work. You don’t have to be a scientist to spot a conflict of interest – “hey this study was funded by a cereal company X and reports that their cereal prevents shingles!” or a limited population that might not apply to you “caffeine helps prevent disease Y in rats”. Use your brain – you’ve got a good one.
I’m a huge proponent of science. I freaking love it – with the intensity of pretty much any Nicholas Cage character – however, I hate how it often gets boiled down to garbage by the time it reaches most of us. In the next couple of articles we’re going to go over how you can find the wheat in the piles of chaff that float around our worlds. In the meantime – read, read, read, and question EVERY fact you hear!
*yes, it’s a real study: Wahlbert, M., & Westerberg, H. (2003). Sounds produced by herring (Clupea harengus) bubble release. Aquatic Living Resources, 16 (3), 271-275.
