28 March–1 April 2015

Boston Convention and Exhibition Center

Boston, MA

Attendees: >14,000

Experimental Biology is an annual meeting that brings together researchers from dozens of life-science disciplines, from laboratory to translational to clinical investigation. Six professional societies take part in the 'superconference': American Association of Anatomists, American Physiological Society (APS), American Society for Biochemistry and Molecular Biology, American Society for Investigative Pathology, American Society for Nutrition (ASN) and American Society for Pharmacology and Experimental Therapeutics (ASPET).

'Reproducibility' seemed to be the keyword of the conference. Reproducibility is a cornerstone of the scientific method. It holds that independent researchers should be able to repeat a reported experiment, under the same conditions, and obtain the same results. Reproducibility confirms that a result is reliable. But in recent years, reproducibility in published research has faltered. Some reports suggest that as little as 10–25% of published findings can be reproduced. Non-reproducible results represent a waste of time, labor and money; delay medical progress; damage scientists' careers; and erode public support for research. These costs are staggering, leading many facets of the research community to respond. Investigators, funding agencies, pharmacology companies, academic institutions and journal publishers are all working to identify and address factors contributing to the reproducibility crisis.

During the conference, three of the six participating societies held sessions dedicated to the problem of reproducibility. The APS symposium on “Reproducibility in Research” featured Shai Silberberg (US National Institute of Neurological Disorders and Stroke, Bethesda, MD), Malcolm MacLeod (The University of Edinburgh, UK) and Richard Nakamura (US National Institutes of Health, Bethesda, MD). Silberberg outlined potential causes for poor reproducibility, including incorrect identification of resources (such as cells or antibodies) and lack of transparency in reporting, two issues that resurfaced in presentations throughout the conference. He suggested that authors' submission of unclear or incomplete methodological descriptions and publishers' convention of abridging methods sections (or omitting them altogether) in print versions of papers are two reporting practices that contribute to poor reproducibility. MacLeod added that investigator bias is a key problem in preclinical research that hinders translation of findings into clinical applications. Nakamura highlighted the ability of peer review to improve reproducibility by applying more rigorous standards. Though he primarily focused on review of grant proposals, he also commented that the science community needs a change in attitude toward publication, a thought echoed by speakers in other sessions.

ASN addressed reproducibility and scientific publication in a symposium titled “Research Reporting in the 21st Century.” Jeffrey Drazen recapped the history of reporting ethics from his perspective as editor in chief of New England Journal of Medicine (Waltham, MA). He reminded attendees that the goal of research is to gain new knowledge that others can use, a premise that should be driving publication of results. Ivan Oransky, co-founder of the blog Retraction Watch, pointed out that the number of retractions of published papers is rising, owing in part to better detection of fabrication, falsification and plagiarism in published reports. Debra Parrish (Parrish Law Offices, Pittsburgh, PA) closed the session by discussing the legal implications of misconduct in research reporting.

ASPET's symposium “Reproducibility in the Pharmacological Sciences” presented strategies for addressing the reproducibility crisis. Darrell Abernethy (US Food and Drug Administration, Silver Spring, MD) noted the need to publish studies with negative results as well as those that report confirmation of or failure to confirm a published result. Michael Williams (Feinberg School of Medicine, Chicago, IL) observed that the current science publishing system is willing to overlook a lack of rigor in favor of 'flashy' results and that the core scientific method has been replaced by technological advances, disenfranchising both science and scientists. He called on the community to raise the standards of publication and on journal staff to prioritize enforcement of appropriate standards. Adena Schachner (Boston University, MA) suggested that calculating and reporting a 'replicability' score for journals could remove a barrier to reporting replication studies by rewarding journals that apply rigorous reporting standards and encourage publication of negative, confirmatory or contradictory results. Leonard Freedman of the Global Biological Standards Institute (Washington, DC) made an appeal for standards in life-science research to facilitate the development and use of consensus-based best practices. He cited the absence of a unifying quality-control framework as one factor contributing to the lack of reproducibility in research.