From small to large studies, early or highly sited studies, scientists aren't immune to cognitive biases in their work.
Credit: Petr Kratochvil/Public Domain
Researchers have been studying how science is done by asking how common biases are, how they affect different disciplines, what are the factors that cause these biases, and how to reduce bias in scientific work. The research paper was published in Proceedings of the National Academy of Sciences. Over 3,000 meta analyses with 50,000 research studies were used to map the biases in science.
These are the types of biases that were found:
- Small-study effect: when studies with small sample sizes report large effect sizes.
- Gray literature bias: the tendency of smaller or statistically insignificant effects to be reported in PhD theses, conference proceedings or personal communications rather than in peer-reviewed literature.
- Early-extremes effect: when extreme or controversial findings are published early just because they are astonishing.
- Decline effect: when reports of extreme effects are followed by subsequent reports of reduced effects.
- Citation bias: the larger the effect size, the more likely the study will be cited.
- United States-effect: when U.S. researchers overestimate effect sizes.
- Industry bias: when industry sponsorship and affiliation affect the direction and size of reported effects.
Researchers looked into these factors that may increase the risk of bias:
- early career status
- isolation of working alone without colleagues
- size of collaboration between scientists
- involvement in misconduct
- gender of researchers- pressure to publish
The most biases come from small studies. Studies that made it into peer-reviewed journals or became highly cited also were more likely to overestimate the effects of their research. When a small sample size of a population is taken, the results gained may not actually represent the reality overall, which is why it's an overestimation effect. Early studies without verification, especially U.S. studies, tended to report more extreme effects about the matter they were studying. The frequency of publishing, the gender of the researcher, and the incentive to publish the not have more bias than other studies.
The influence of different biases changes in different disciplines, yet seems to be a constant with respect to individual scientists. Senior author John Ioannidis said:
"This is particularly driven by the social sciences, so if you broke scientific fields into big bins of biology, medicine, physical sciences and social sciences, it seems that the social sciences are seeing the more prominent worsening of these biases over time."
Bias in behind some areas, but nonexistent and many others. Therefore, bias does not undermine all of the scientific work. A one-size-fits-all solution is not likely, as interventions to resolve biases is likely required to be tailored to the specific problems found in individual disciplines or fields of scientific research.
Making science better can happen on its own, as was done with physics where they decided to replace tiny studies done by small teams with multi-team collaborative models. Each field of science is different with different bias profiles, and it's probably best for each to choose the best way to reduce biases affecting their particular field. Ioannidis said "this has to be a grass-roots movement". Scientists need to understand and want to do this for the good of their science. Institutions and funding agencies should not enforce these changes.
References:
- Studies of scientific bias targeting the right problems, study finds
- Daniele Fanelli, Rodrigo Costas, and John P. A. Ioannidis, Meta-assessment of bias in science, PNAS 2017; doi:10.1073/pnas.1618569114
If you appreciate and value the content, please consider:
Upvoting 
, Sharing 
or Reblogging 
below.
me for more content to come!
Looking to contact me? Find me on Discord or send me a message on SteemKURE.
Please consider supporting me as a Steem Witness by voting for me at the bottom of the Witness page; or just click on the upvote button if I am in the top 50:
If you are unsure how to vote for witnesses, you can put my name in the "SET PROXY" section at the bottom of the Witness Voting page which will use my witness votes.
2017-03-27, 10:10am