PART 4

Evaluating Investments and Performance in Research

The most important issue in evaluating our national investment in science today is the Government Performance and Results Act (GPRA), which codifies the responsibilities of scientists that were discussed in previous chapters. This section examines how the GPRA is being implemented. Because it is a new process, growing pains are inevitable and each agency must work to refine its approach. But the GPRA will be with us for the foreseeable future and we must find a way to implement it so we accomplish the true intent of the Act. We also see in this section how the United Kingdom is dealing with assessment and accountability.

Andrew J. Vogelsang from the General Accounting Office starts us off with updates on recent developments in GPRA implementation. He details the three essential requirements of GPRA. To address concerns from scientists that they were asked to "measure the unmeasurable," the statute provided for an alternative form, which is flexible to meet the needs of the agencies. Vogelsang details what happened when the agencies submitted drafts to Congress. When GAO issued a report that said these agencies' plans appear to provide a workable foundation for Congress to use, "this broke with some of the commonly held views on the Hill."

Joshua Gotbaum, from the Office of Management and Budget, deals with a "painful and difficult" topicfundamental misunderstandings about GPRA. He stresses that research and public investments in science and technology are important and central to the public agenda. The discussion is about how we allocate and decide how federal research dollars are spent. Although it is hard to measure scientific gain, we must measure its performance. But what you ought to measure varies agency by agency and process by process. The next test, he says, will be how to incorporate GPRA into budget decisions prospectively.

Joseph Bordogna provides us with a snapshot of how GPRA is being implemented at NSF. He focuses on one of five outcome goals identified in the NSF strategic plan (discovery at and across the frontiers of science and engineering). NSF used the "alternative format" authorized by GPRA. NSF's experience with GPRA has been positive in its ability to encourage thoughtful planning and invest in the most creative and innovative endeavors in research and education. For this reason, Bordogna is optimistic about what lies ahead.

Ben Martin discusses the key features of the UK research system as well as recent science policy in that country. He also discusses experiences with evaluation in three sets of agencies: the Higher Education Funding Council (and the university research assessment exercises), the research councils, and government department research. He details how the assessment exercises have worked in academia and in Institutes.

Susan E. Cozzens focuses on the issue of confidence. She says the first purpose of the GPRA is to increase the confidence of the American people in government in general and in science and technology specifically (or at least in the federally funded portions). The question she explores is: Will GPRA as it is currently being implemented make scientists and engineers better at winning the confidence of the American public as well as more socially responsible? Directing attention toward defining outcomes (that is, what research delivers to the public) and asking agencies whether they are moving toward those outcomes are fundamentally healthy processes for science.