The Science Of Deciphering Wild Speculation From Truth
In a world where people are arguing about facts and alternative facts, how do you understand news about scientific discoveries and new facts?
One of the big ideas coming out of research about teaching science is teaching novices to frame information and questions like experts do.
Here are some ways to read news about science like a scientist does:
1. Consider the source of the news article
Is this from a writer with sufficient understanding of and experience with the subject? Within a scientific community, people know who is an expert on what. While both are often called biologists, a neuroscientist (like me) is not the one you want to go to for answers about ecology. Generally, the best scientific reporting is going to come from publications or writers that exclusively do science and tech reporting. The New York Times science section and programs like Science Friday on NPR are far more trustworthy than, say, The Daily Mail. One of the best sources of information is interviews with the people who did the research or experts in that field. The best reporters will interview the researchers or other experts and quote them.
2. Consider the source of the research
Where was the research published? If there isn’t a link to the original research paper (or at least full title, journal, and authors) be wary. With the ubiquity of the internet, there has been a massive increase in “journals” that are not peer-reviewed that one can pay to have work published. One way a publication’s quality is measured is the impact factor . Impact factor is how often articles published in a journal are cited by other articles. A high impact factor doesn’t mean the research is better, but if the research never gets cited, it is likely an illegitimate publication.
Who paid for the research? Scientists are required to disclose conflicts of interest (real or perceived) in their papers. This will include how the research was funded. If funding is from a corporation that could benefit from or be damaged by the findings, proceed with caution.
3. Evaluate the claims
Does this finding have sufficient support? Extraordinary claims require extraordinary evidence. When scientists evaluate each other’s research, they are very critical of claims about the significance of a project. Scientific claims must be backed up with appropriate experiments and analysis (especially statistics). Wild speculation or extrapolation is a giant red flag. Likewise, a news item about a cure for every kind of cancer must be more carefully read than one about a drug that reduces mortality from melanoma by 50%. A good science journalist will explain both the limitations and possibilities of the research they’re reporting.
What questions were the researchers trying to answer? If there is not a clear question driving the research, you need to start asking some questions about the research.
Do other news outlets agree about the findings? One of the hallmarks of a solid scientific discovery is reproducibility. If another person in the field cannot reproduce the result or disagrees about the conclusion, they will likely write another paper disputing the original finding. Analogously, a big science story will be reported about in multiple places. If those journalists are not reporting very similar things, then you need to dig deeper.
4. Find the original research
If the US government funded the research, the abstract (synopsis), and often the full article, will be available through Pub Med from the National Institutes of Health. Many college libraries give alumni access to their databases and subscriptions, which will have the full research articles. More and more biologists are uploading their work on sites like biorxiv.
5. Ask an #ActualLivingScientist
Scientists talk to each other a lot. If they have a question about a procedure or analysis someone else used, they will ask that person directly for advice.
Did you know there’s a large community of scientists on Twitter? They are passionate about #scicomm. They write about their daily work and recent findings. Some of them will even identify that snake you saw (@AlongsideWild). There are accounts that rotate among interesting scientists (@neurotweeps, @biotweeps, @realscientists) who often do Q & A. Individual scientists will sometimes answer direct questions about their own research or something that is in their area of expertise.
6. Stay curious
Scientists ask questions about the things that interest them. Keep reading critically and asking more good questions.
