Maybe you have seen or heard articles on such papers which note that there is scientific evidence that stereotypes like intelligence including men’s ability to read maps or women’s ability to take care of children in the brain is real.
In a field that has produced so much literature how does one determine what is valid cutting edge research and what is merely the author’s passion for hype?
Eager reporters and scientific spin are frequently attributed to university’s public relations departments and free-lance science-journalists of popular daily papers or twitters from the academic fields.
However, the deceptive impressions occasionally can originate in the research papers themselves.
Magic: Spotting the spin
My colleagues and I recently published a set of guidelines in Nature Communications that provide precisely this help, recognizing five potential sources of misrepresentation to avoid.
To aid identification the initials combine to spell out the word Magic, which stands for magnitude, accuracy, generalizability, inflation and credibility.
For magnitude, the question is: Is it possible to say that the degree of any difference is properly specified? Look at this research done on Sex differences in the human brain conducted in 2015.
From the 34,716 different patterns of functional brain connectivity, it identified a sex difference in statistics of 178 out of them.
But since the test they used calculated that less than half of 1 percent of the differences they were measuring were actually statistically significant, they could not reasonably cite them as prominent sex differences. In this study, they weren’t.
The second question is the question of accuracy. Has specific techniques and variables been depicted and appropriately applied in the analysis of the results?
The third question again is related to how generalizable the study findings are. Is the applicability of search results of search wide enough that authors do not want to take any chances?
Here we consider the issue that many scientific investigations are conducted with groups of people who are recruited and often self-selected – sometimes students.
The fourth category, inflation, is concerning whether the authors do not use accrual language in presenting the findings. Such words as ‘profound’ and ‘fundamental’ can therefore be misappropriated for example.
Bear in mind that James Watson and Francis Crick only said that they have found some considerate biological interest: a double helix of DNA in this case.
Finally, we should consider credibility: To what extent are authors careful to state how their work does or does not relate to prior literature by using specific terms on scientific reflectivity?
For instance, they may be looking at the gross sex differences in things like, visual perception and spatial awareness.
If for instance gamers are more likely to be boys, then that could not at all mean that their brains are wired for them —it could equally well be boys doing what is culturally expected from them due to gender expectations that might make such games popular among them.
The version of the survey conducted post-lockdown has indicated that the people have a better trust in utterances of the scientists than before the pandemic.
Those in the field of science have to be more cautious that they do not reverse that trend by providing information that is as fuzzy as it is sensationalised.
Perhaps the Magic guidelines will assist them and their editors accomplish this, if not, armed with the Magic guidelines, the sharp and vigilant readers will be prepared.
Reference: Rippon G. Five simple questions can help spot exaggerated research claims over sex differences in the brain
