Reproducibility, the ability to replicate or reproduce experimental results, is one of the major tenets of the scientific method.
However, in the case of academic preclinical research, reproducibility (or more accurately the lack of reproducibility) has become a significant problem. An increasing number of reports have found discrepancies in published preclinical studies across scientific disciplines. For instance.
– Amgen found that 47 of 53 “landmark” oncology publications could not be reproduced.
– Bayer found that 43 of 67 oncology & cardiovascular projects were based on contradictory results from academic publications.
– Dr. John Ioannidis and his colleagues found that of 432 publications purporting sex differences in hypertension, multiple sclerosis, or lung cancer, only one data set was reproducible.
These studies, and the many others that report similar results, highlight a significant problem in the development of new therapies to treat disease. The identification of potential drug candidates typically happens in academic research labs. Pharmaceutical companies then use these new drug candidates as the basis for their drug development efforts. With increasing reports of discrepancies in preclinical publications, pharmaceutical companies are being forced to re-evaluate their reliance on academic research (see Bayer’s decision to halt nearly two-thirds of target-validation projects).
So why do so many preclinical publications contain research that can’t be reproduced? Join our panel of distinguished guests from academia, publishing, and the startup community to hear about the latest approaches to dealing with this critically important issue.
Presentations are followed by a panel discussion and audience Q&A. Continue the conversation at a nearby bar (to be announced at the event). *Presentations and discussion livestreamed. Follow the event on Twitter (@sciobayarea and #sobay), and like SOBA on Facebook.
Chris Mentzel is a program officer in the Science Program at the Gordon and Betty Moore Foundation, focused currently on developing a strategy for long term investment in “data-driven discovery” that will enable scientists to turn the scientific data deluge into opportunities to address some of today’s most important research questions. Chris identifies the people, advanced instrumentation, and information technologies that help solve important data-rich science questions.
Elizabeth Bartmess is part of the Reproducibility Project, a large-scale open science collaboration to estimate the reproducibility of a large sample of studies from the scientific literature, focusing on three major psychology journals. She develops and maintains policy, procedures, and documentation, connects researchers to information and volunteers, and coordinates volunteers. Her background is in research psychology and in user experience design.
Michael Cohn is a health psychology researcher at UCSF, where he designs web-based emotion skill programs for people with a variety of chronic illnesses. He has a PhD in social psychology and is highly interested in efforts to determine the overall reliability and robustness of the field’s findings. As a volunteer administrator with the Reproducibility Project, he keeps track of the pool of studies targeted for replication, matches contributors with appropriate projects, and helps build collaborations across institutions to facilitate larger replication studies.
Liz Silva graduated from Simon Fraser University with a degree in Biology, then the University of Alberta with a Masters in Molecular Biology and Genetics. She completed her Ph.D. at the LRI, Cancer Research UK in Developmental Biology and followed that with two post-docs, first at the MRC-LMCB in London, and another at the University of California San Francisco. Drosophila has featured heavily in her research career and she has covered a range of fields: cell cycle, immunity and regulation of organ size to name a few. She has a firm appreciation of basic research, the beauty of genetically-tractable model organisms, and a good confocal image. She’s been an Associate Editor for PLOS ONE since 2011, cultivating a passion for finding solutions to issues in scientific publishing and research policy: funding of basic research, data sharing, problems with the impact factor and, of course, open access.
Alexander Berger is a Senior Research Analyst at GiveWell, a nonprofit dedicated to finding outstanding giving opportunities and publishing the full details of their research to help donors decide where to give. He has a BA in philosophy and an MA in education policy, both from Stanford, and leads GiveWell’s research on opportunities to improve the reliability and efficiency of biomedical research.
Christin Chong and Jonathan Russell
Christin Chong, Ph.D is a postdoctoral scholar at the University of California, San Francisco. She is co-founder of MicroPub.org, a platform for helping and rewarding scientists for reproducible data. She is an avid cellular/molecular neuroscience writer, and her research focuses on the genetic basis of when and how much we sleep.
Karin is responsible for technology management, licensing and outgoing material transfer agreements at the Office of technology Management at UCSF. Previously at the Office of Technology Transfer, Karin managed a large life-sciences portfolio consisting of technologies from seven of the UC campuses. Her work involves managing patent interferences, infringement issues and other licensing-related disputes. Karin conducted her doctoral work at UCSF in the laboratory of Dr. J. Michael Bishop, in collaboration with her Swiss mentor, Dr. Ernst Hafen of the University of Zurich. Her research focused on elucidating the developmental roles of a cancer-related tyrosine kinase gene in Drosophila melanogaster. She received a M.S.-equivalent degree in biochemistry from the University of Zurich in Switzerland, where she began her research in Drosophila genetics.
Photo: Myleen Hollero