More and more papers are retracted each year. In a reproducibility study Bayer was able to repeat only 14 out of 67 drug target research experiments from landmark papers. Amgen set out to repeat 53 key discoveries in the field of cancer and succeeded in only 6.
About a year ago, a big reproducibility crisis broke out in the scientific community. It all began with a group of Japanese and American scientists who published a provocative paper describing the creation of a new type of induced pluripotent stem cells in a leading scientific journal. It soon became clear that their results could not be replicated and the verdict of the official investigation was that the data were falsified.
More and more papers are retracted each year. The reasons behind these retractions were investigated in a few publications (listed at the bottom). Some argue that honest mistakes are the most prevalent reason behind the retractions, but if you read Retraction Watch it is pretty obvious that is not true as many recent retractions stem from fraudulent, rather than sloppy, science. The retracted papers usually remain online because they still offer an insight into the scientific process. A big problem is that the retractions are often not well publicized. There are also significant variations when it comes to handling retractions. Many editors strive for transparency in their retraction notices, offering detailed explanations for what went wrong, but others do not provide any answers or tell curious readers to ask the retracted authors to supply them.
The highest-profile retraction last year was of two papers published in Nature at the beginning of the year. As already mentioned above the papers described a technique called STAP (Stimulus-Triggered Acquisition of Pluripotency), which is a new method for creating pluripotent stem cells from adult cells. The published method was unbelievably simple – placing blood cells in a mild acid for a short time – but no one was able to reproduce it. Nature retracted the papers in July, the leading author Haruko Obokata was found guilty of research misconduct and one of the co-authors committed suicide.
Another resounding case of fraud refers to the study claiming development of an AIDS vaccine. Laboratory Manager Dong-Pyou Han was sentenced to nearly 5 years in prison and has to pay back $7 million to the NIH. He confessed to spiking rabbit blood samples with human antibodies to make an experimental HIV vaccine more promising. This case is interesting because it also resulted in criminal charges against Han which, to say the least, is not a common practice in cases of scientific misconduct.
A few years ago in 2009, researchers were asked in an anonymous poll if they had ever falsified data. 2% confessed to falsifying data and 34% to manipulating results to confirm their initial hypothesis. Key data can often not be reproduced and this has the greatest consequences for pharmaceutical and other such companies. To mitigate the potential loss of money many companies carry out in-house validation programs. In a reproducibility study Bayer was able to repeat only 14 out of 67 drug target research experiments from landmark papers. Amgen set out to repeat 53 key discoveries in the field of cancer and succeeded in only 6. Even publications in prestigious journals cannot be excluded from this trend.
In 2013 an excellent paper on an experiment with open-access journals was published in Science. The author intentionally wrote a falsified paper on a drug for cancer treatment. He submitted the paper to more than 300 open-access journals over a 10 month period, and more than half of them accepted it. A number this high should raise alarms!
...databases listing the retractions and lists of predatory journals will promote awareness against fraud...
It is interesting that scientists are aware of this problem but until recently nobody addressed the issue properly. Recently the NIH finally recognized the problem regarding data reproducibility in preclinical research and issued so-called “Reproducibility Guidelines” with which they plan to enhance reproducibility. They claim that a complex array of other factors and not fraud seem to have contributed to a lack of reproducibility. Factors include poor training of researchers in experimental design, an increased emphasis on making provocative statements rather than presenting technical details and publications that do not report the basic elements of experimental design. Some scientists reputedly use a secret ingredient or technique to make their experiments work — and withhold details from publication or describe them only vaguely to retain a competitive edge. Additionally there is the problem of publication bias which means journals prefer to publish positive results. There are few forums in which researchers can publish negative data or papers that point out scientific flaws in previously published work.
Papers publishing negative results and confirmation studies will help to raise the excellence of scientific publications to a higher level.
The NIH held a joint workshop in June 2014 with the Nature Publishing Group and Science on the issue of reproducibility and rigor of research findings, with journal editors representing various basic/preclinical science journals. The journal editors at that workshop came to consensus on a set of principles to facilitate these goals. These principles included: rigorous statistical analysis, transparency in reporting (reporting replicates, statistic methods, randomization, blinding, sample-size estimation, inclusion and exclusion criteria), data and material sharing, consideration of refutations, establishing best practice guidelines for image based data and descriptions of biological material (antibodies (report source, characteristics, dilutions and how they were validated), cell lines (report source, authentication and mycoplasma contamination status) and animals (report source, species, strain, sex, age, husbandry, inbred and strain characteristics of transgenic animals)).
As long as the whole process of acquiring research funding remains based on the number of publications, researchers will still strive for quantity and not quality.
These reproducibility guidelines are just half the answer to these recent events in science. In my opinion, one of the main reasons for falsifying research data is the pressure to publish to get more funding and competition among laboratories. As long as the whole process of acquiring research funding remains based on the number of publications, researchers will still strive for quantity and not quality. Double blind review process, where a reviewer’s comments would be made public after acceptance of a paper, would further improve the whole process of peer review. Papers publishing negative results and confirmation studies will help to raise the excellence of scientific publications to a higher level. On the other hand, databases listing the retractions and lists of predatory journals will promote awareness against fraud and will be a reminder of what is the basic purpose of our research: to discover new and exciting things that will improve our quality of life.
By Mojca Jez, PhD, Researcher at the Blood Transfusion Centre SI
Publications about Retractions and Data Falsification:
[tw_callout size="waves-shortcode" text="" callout_style="style2" thumb="" btn_text="Republish the article" color="#37a0d9" btn_url="https://scinote.net/blog/republish/" btn_target="_blank"]
