14

The Here and Now of NSF and GPRA

Joseph Bordogna

These are exciting and optimistic times at the National Science Foundation (NSF), as they bring the possibility of record increases for investments in research and education.

There are also some notable personnel changes at NSF, with Rita Colwell's arrival as Director, once her confirmation by the Senate is complete. Her long-time relationship with AAAS will clearly serve both organizations very well. Neal Lane's move to the White House underscores how his leadership has strengthened the Foundation and positioned it to take on new challenges.

One of those challenges is the topic of this chapter: implementation of the Government Performance and Results Act (GPRA) at the National Science Foundation. The issue of performance measurement for research is continuously evolving, and the GPRA process is far from complete. The one thing (and perhaps the only thing) beyond dispute is that GPRA is for real and in force.

It is appropriate to start with a point of comparison. The GPRA process bears a striking similarity to Winston Churchill's legendary advice on public speaking. Churchill said all speeches should have three parts: First, tell the audience what you are going to say. Second, say it. Third, tell them what you said. The GPRA process transfers the essence of Churchill's wisdom to the arena of budgeting and planning.

Strategic planning is the prospective first step. It covers what we intend to do. Then comes the "just do it" stage, namely actual budgets and performance plans. These are the plans of action that guide decisions throughout the year. Finally, we take a retrospective look by crafting Performance Reports that examine what we actually did.

There are, of course, some notable differences between Churchill's speeches and the GPRA process. For starters, a good speech usually unfolds over a half-hour or so. The GPRA process will take years to unfold. It can be difficult to hold your audience's attention for that length of time (especially when your audience's membership changes periodically). Furthermore, no amount of Churchillian rhetoric can compensate for substantive shortcomings when implementing GPRA. Success vis-a-vis the GPRA process is not derived from inspiring language, but rather from documented results and solid empirical evidence.

This provides the backdrop for a snapshot of the "here and now" of GPRA at NSF. This snapshot will focus on only one of the five outcome goals identified in the NSF strategic plan, and it will pull out key pieces of our FY 1999 budget request and performance plan related to that particular goal. Finally, I will comment on how we are planning to gauge our success at meeting that goal.

Outcome Goals

If you follow the Foundation, you may have come across the five outcome goals identified in our GPRA strategic plan. This chapter will focus on the first goalthat of discovery at and across the frontiers of science and engineering. The four other outcome goals speak to other dimensions of our research and education mission: promoting connections between discoveries and their use in service to society, education and training, and data collection and dissemination efforts.

Without question, the first of these goals relates. It has also attracted the most concern and apprehension, as is appropriate for any discussion of performance measurement in the context of fundamental research activities.

Definition of Success

The implementation NSF's performance is:

• "successful" when NSF awards lead to important discoveries, new knowledge and techniques (both expected and unexpected) within and across traditional disciplinary boundaries, and high-potential links across these boundaries;
• "minimally effective" when there is a steady stream of outputs of good scientific quality.

NSF's response to what is known as the "alternative format" authorized by GPRA. The law specifically allows agencies to work with the Office of Management and Budget to develop qualitative descriptions (as opposed to quantitative targets) in defining certain performance goals.

This supports the idea that NSF should not ask researchers to tell us on what day they expect to make a major discovery, or when they will be (for example) 75 percent of the way toward that discovery. It is worth noting that there has been some argument about the categories "successful" and "minimally effective." While it might be desirable to use a term such as "excellent" to exemplify the true intent of NSF's work, the Act itself, however, specifies the terms to use when employing the alternative format.

Priorities and Performance Goals

This brings us to step 2 in the process: the plan of action as expressed in our FY 1999 budget request and performance plan. When Neal Lane joined Vice President Al Gore and other agency heads at the White House to unveil the 1999 budget request, he listed NSF's priorities as follows:

• Advancing science and engineering at and across the frontiers;
• Improving grant size and duration; and
• Focusing on key multidisciplinary prioritiesparticularly in such areas as Knowledge and Distributed Intelligence, Life and Earth's Environment, and Educating for the Future.

Each of these priorities is put into operational terms by activities identified in our performance plan. A few examples are:

• We will have an independent assessment panel examine the balance across our portfolio.
• We have adopted a performance goal of increasing average grant duration from 2.3 to 2.7 years.
• A number of performance goals speak to our emphasis on multidisciplinary priorities. Some of these relate to the specific themes identified in the budget, while others will examine more generally our support for emerging opportunities.

Of course, the real test will come during FY 1999, through the investment decisions that lead to the outcomes expressed in the five goals.

Factors in Assessing Success

Step 3 encompasses the part of the GPRA process that is yet to come. To repeat steps 1 and 2 establish intent and outline a plan of action. The next stage requires developing performance reports that assess and document our progress. Since the performance plan covers FY 1999, we will not begin writing our first "official" performance report until the end of that fiscal year. It will therefore not be completed until early in the year 2000 (right around the time we submit our FY 2001 budget to Congress).

It is nevertheless appropriate to begin examining what a performance report might look like. Success for discovery-related activities can come in many forms. The general frameworks could include:

• "Important Discoveries." Despite what you may read in some circles, these never seem to run out. Examples include Bose-Einstein Condensate, planets orbiting distant stars, and advances emerging from plant genome research (such as isolating the genes that control flowering and self-fertilization in plants).
• "Contributions to the Knowledge Base." These are the puzzle pieces that lead to advances across broad areas. An excellent example can be found in the recent Science article on the connection between climate and the fate of early colonial settlements. Tree ring data proved to be the key piece of this puzzle.
• Another set of important outcomes is related to "Opening New Directions." The emerging field of nano-scale science and engineering speaks to this. It has opened new avenues of exploration in the physical sciences, the life sciences, and numerous areas of engineering. Nano-scale science and engineering links these fields in new ways.
• "Linkages Across Areas of Knowledge." The emerging field of functional genomics (which has given us DNA on a chip) is a prime example of this type of advance.

In addition, NSF's performance report will likely contain a brief discussion of the importance of failure. There is great value in findings that reveal dead ends, paths to avoid, and weak spots in underlying theories. NSF's performance plan specifically requires examining how well we cultivate risk-taking in our investments.

Conclusion

By necessity, this snapshot has covered just a small sampling of the here and now of GPRA at NSF. This chapter has focused on just one of NSF's five outcome goals. In addition, for that one goal, it has examined have glimpsed only a small fraction of the various investment strategies and priorities outlined in the agency budget request and performance plan.

What this snapshot should nevertheless make clear is that each part of this process requires walking a tightrope. It is a tightrope that runs between terms like accountability and independence, planning and flexibility, and risk and confidenceall of which must be balanced to attain the highest return on the taxpayer's investment.

To conclude, it is appropriate to juxtapose two quotes. The first is a line about the very notion of planning in relation to research. It reads: "The best planning in the world is no substitute for plain, dumb luck." The second comes from the writings of Louis Pasteur: "Chance favors the prepared mind."

Our experience to date with GPRA has made clear that thoughtful planning can enhance, and in no way hinder, NSF's ability to invest in the most creative and innovative endeavors in research and education. For this reason, the here and now of GPRA at NSF should leave all of us feeling appropriately optimistic about what lies ahead.

Joseph Bordogna is acting deputy director of the National Science Foundation. This article is based on remarks delivered at the 23rd Annual AAAS Colloquium on Science and Technology Policy, held April 29­May 1, 1998, in Washington, DC.