A casual conversation about fellowship projects at a networking event revealed that Todd Haim, who was serving his fellowship at the National Institutes of Health (NIH), and Tiffany Sargent, a Fellow at the National Science Foundation (NSF), were involved in similar evaluation efforts at their agencies.
“We were being asked to assess and evaluate the impact of federal funding at our agencies, and how the funding was contributing to new knowledge, to societal impacts, or generating new areas of focus,” says Todd.
They soon discovered several other S&T Policy Fellows who were engaged in assessment activities; their common interests launched the FIRE (Federal Innovation and Research Evaluation) Affinity Group. Tiffany and Todd were named co-chairs to convene and coordinate the 30 participants, which included both Fellows and government employees from several agencies. They began meeting monthly to discuss challenges and to share best practices and success stories.
”Evaluation and assessment are so broad and can cut across a variety of fields, which is why so many people were interested and got so much out of our group discussions,” recalls Tiffany.
Erin Fitzgerald, an S&T Policy Fellow at the Department of Defense, and the FIRE Affinity Group’s technology officer, noted “with the economic stimulus package there was money for many research investments but no one was sure how to evaluate the return on investments. I wanted to learn how others were approaching the issue.”
From the monthly discussions the idea for a broaderbased symposium emerged, and the FIRE Affinity Group set as its main objective for 2010 to convene a conference to highlight the latest developments, techniques, and technologies being used to evaluate the impact of federal programs. The aim was to showcase success stories and innovative methods and tools.
“Agencies are applying different strategies and metrics,” said Peter Wu, an S&T Policy Fellow at NSF and FIRE Affinity Group member. “We recognized the value of gathering individuals across government agencies to highlight best practices and share resources.”
Over the next several months the group collaborated to develop the symposium program, recruit speakers, and secure funding. The symposium, Novel Methods for the Evaluation of Federal Research Programs, held in April 2010, was a rousing success. It attracted nearly 200 participants from federal agencies and non-government organizations.
The keynote presentation was delivered by Stephanie Shipman, assistant director of the Center for Evaluation Methods and Issues at the U.S. Government Accountability Office. She stressed the importance of bringing together leaders in the field and bridging the gap between the different federal agencies.
The program included two panel discussions: Capturing the Economic Impact of Federal Investments in Research and Tools and Instruments to Evaluate the Impact of Federal Investments in Research, which gave attendees the chance to hear perspectives from a range of experts in the field. Panelists included:
- Stephen Campbell, Group Leader for Impact Analysis for the Technology Innovation Program at National Institute of Standards and Technology
- Richard Clinch, Director of Economic Development at the Jacob France Institute
- Robert Harbick, Principal at Booz Allen Hamilton
- John L. King, Resource, Environmental, and Science Policy Branch at the U.S. Department of Agriculture Economics Research Service
- Steven McKnight, Division Director of Civil, Mechanical and Manufacturing Innovation at the National Science Foundation
- Stephen Merrill, Executive Director of the National Academy of Science’s Board on Science, Technology, and Economic Research
- Andrew Toole, Assistant Professor of Economics at Rutgers University
- Alan Porter, Professor at Georgia Institute of Technology
- John Stasko, Professor and Director of the Information Interfaces Research Group at Georgia Institute of Technology
They discussed useful tools, from IT management and new software solutions to knowledge management strategies and practices. “The symposium helped me understand the most effective data to collect in order to make an informed decision that I know I will use at NIH and in the future,” said Todd.
At the start of the 2010-11 fellowship year, the initial FIRE Affinity Group leaders passed the baton to new officers and the group continues to flourish and meet monthly to discuss ideas, share evaluation methods and review best practices.