16 Feb

III Arnold Workshop: Modeling and Code

16 Feb 2016 through 17 Feb 2016
9:00 am to 2:00 pm
AAAS Headquarters

Thanks to generous funding from the Laura and John Arnold Foundation, Science Magazine/AAAS is carrying out a series of three workshops aimed at addressing reproducibility and transparency in science. The first workshop resulted in the publication of the “TOP” Guidelines, to which more than 500 journals are now signatory. Results of a second workshop, which focused on open standards for data and samples, are currently being prepared for publication. The third workshop will take place in Washington D.C. on February 16-17 2016 and will address modeling and code. 

The computational steps that produce scientific findings are increasingly considered a crucial part of the scholarly record, permitting transparency, reproducibility, and re-use. Important information about data preparation and model implementation, such as parameter settings or the treatment of outliers and missing values, is often expressed only in code. Such decisions can have substantial impacts on research outcomes, yet such details are rarely available with scientific findings.
Issues that present in facilitating access to code and modeling details vary from DOIs and linking to publications, standards for dissemination and citation, benchmarking and testing, patenting and licensing, through to workflow development and mentorship. This workshop will consider ways to make code and modeling informatin more readily available, and include a variety of stakeholders. We hope to produce a document at this workshop with recommendations that researchers, journals, and funding agencies could use to raise standards for reproducibility in the computational sciences.

If you have questions about workshop goals or agenda, please email Marcia McNutt (mmcnutt@aaas.org) or Victoria Stodden (vcs@stodden.net). If you have questions about workshop logistics please email Anna Bashkirova at abashkir@aaas.org.

We look forward to seeing you in February at the AAAS for a productive discussion and concrete actions!


Tuesday, February 16th

09:00 am-09:30 am

Breakfast and mingle

09:30 am – 10:00 am

Welcome and summary of previous workshops

10:00 am – 11:00 am

Keynote Presentations

Marcia McNutt

Victoria Stodden

11:00 am – 12:00 pm

Panel 1: Integration with the scholarly record: Case Studies and Lessons Learned

Michela Taufer

12:00 pm – 1:00 pm


1:00 pm – 2:00 pm

Panel 2: Interoperability standards, proprietary codes, and verification/testing

Michael Heroux

2:00 pm – 3:00 pm

Panel 3: Licensing and facilitating re-use

Victoria Stodden

3:00 – 3:30 pm


3:30 pm – 4:30 pm

Panel 4: Credit and citation standards, persistence (i.e. DOIs, repositories, embargo periods)

Kate Keahey

4:30 pm – 5:30 pm

Panel 5: Edge cases, specialized hardware, large or exceptionally complex code bases

Lorena Barba

5:30 pm

Adjourn (Day 1)


Wednesday, February 17th

09:00 am- 09:30 am

Breakfast and mingle

09:30 am – 10:30 am

Panel 6: Minimal sharing requirements, workflows

Ewa Deelman

10:30 am – 12:00 pm


12:00 pm – 1:00 pm


1:00 pm – 2:00 pm

Finalize Recommendations

2:00 pm



Organizing Committee: Marcia McNutt (AAAS), Victoria Stodden (University of Illinois at Urbana-Champaign), Michela Taufer (University of Delaware), Bertram Ludaescher (University of Illinois at Urbana-Champaign), Jelena Kovacevic (Carnegie Mellon University), Barbara Jasny (AAAS), Andrew Sugden (AAAS Science International), Valda Vinson (AAAS), Jake Yeston (AAAS)

Confirmed participants: 

Almadena Y.Chtchelkanova (CISE, National Science Foundation) 
Barbara Lerner (Mount Holyoke College)
Brian Nosek (Center for Open Science)
Brooks Hanson (AGU)
Bruce Childers (University of Pittsburgh)
Carl Boettiger (UC Berkeley)
Chaitanya Baru (National Science Foundation)
David H. Bailey  (Lawrence Berkeley National Laboratory) 
Deborah Olster (National Science Foundation)
Fernando Rios (John Hopkins University)
Greg Farber (National Institute of Mental Health, NIH)
Irene Qualters  (CISE, National Science Foundation)
Jeff Spies (Center for Open Science)
John Ioannidis (Stanford University)
Jonathan Petters (Johns Hopkins University)
Juliana Freire (New York University)
Kate Keahey (Argonne National Laboratory)
Kyle Bocinsky (Washington State University)
Lisa Kempler (Mathworks)
Lorena Barba (George Washington University )
Mike Heroux (Sandia National Laboratories)
Miriam Leeser (Northeastern University) 
Nicholas D'Imperio (Brookhaven National Laboratory)
Phil Bourne (National Institutes of Health)
Philip Stark (UC Berkeley)
Robert Hanisch (National Institute of Standards and Technology)
Ron Boisvert (National Institute of Standards and Technology)
Sallie Keller (Virginia Tech)
Susan Gregurick (National Institute of General Medical Sciences, NIH)
Stephen Feinberg
Roger Wakimoto (National Science Foundation)
Ewa Deelman (Information Sciences Institute)
Yolanda Gil (Information Sciences Institute, University of Southern California)