While it is generally agreed that federal investment in scientific research is crucial for American prosperity and innovation, specialists lack rigorous tools and methods to fully assess the impacts of such investment, a top AAAS official told a congressional panel recently.
Albert H. Teich, director of Science and Policy Programs at AAAS, told the panel that policy research on the impact of science and technology has proved to be “just as unpredictable as basic research in physics, chemistry, or life sciences.” Decision-makers must take into account the fact that some studies may yield unanticipated results or be useful only over the longer term, Teich said.
The impact of science investment. Albert H. Teich, right, testifies at a hearing before the House Subcommittee on Research and Science Education. Also testifying were, from right: Julia Lane, director of the SciSIP program for the National Science Foundation; Daniel Sarewitz, co-director of the Consortium for Science, Policy & Outcomes at Arizona State University; and Fiona Murray, associate professor of management at MIT.
Just as vexing, policy researchers from different disciplines can have difficulty talking to one another, much less to their “customers” in government, Teich said. He and other specialists on science policy testified at a 23 September hearing by the House Subcommittee on Research and Science Education. The topic was The Science of Science and Innovation Policy, or SciSIP, a program of the U.S. National Science Foundation aimed at finding better ways to understand and measure the impact of the billions spent each year on federal R&D.
“I view science and innovation policy as critical for maintaining our international competitiveness and creating jobs,” said U.S. Representative Daniel Lipinski (D-Illinois), chairman of the subcommittee. “But the best policies are not self-evident. As someone who was trained as an engineer and a social scientist, I believe we need data and proper analysis of this data to be able to determine—as best we can—the optimal policy to implement.”
Teich said research on science and innovation policy has been underway at least since the 1960s, but “there is a feeling that the results of this work are not widely known or used among those who actually make science and innovation policy.”
John Marburger, who served as White House science adviser under President George W. Bush, called in a 2005 speech at the AAAS Forum on Science & Technology Policy for the establishment of a “science of science policy” that would spur development of theories, analytical tools and rigorous data sets to assist policymakers in their decisions. In 2007, NSF allocated $6.8 million to start the SciSIP program. It has requested $14.25 million for the program in the 2011 fiscal year.
Teich said one goal is to foster a cadre of specialists who can produce a more systematic, coherent body of knowledge on the science and engineering enterprise—knowledge that can be brought directly to bear on national policy decisions. Teich noted the SciSIP program’s focus on building a “community of practice” among science policy researchers from diverse disciplines.
AAAS has played an active role in building this community through a 2009 workshop that brought such researchers together to learn from one another. AAAS will convene another invitational workshop on 19 October to help connect the researchers with their government customers. “We hope that the workshop will serve to allow these two communities to better understand each other’s needs and expectations,” Teich said.
Taking stock of science. U.S. Representative Vernon J. Ehlers (left), the ranking Republican on House Subcommittee on Research and Science Education, and Committee Chairman Daniel Lipinski, heard testimony on 23 September.
Building a “community of practice” includes more exchange of ideas on curricula and nomenclature among about 25 universities that offer graduate education in science, engineering and public policy. It would be particularly useful, Teich said, for policy officials to become involved in curriculum reviews at the institutions and to become more engaged in teaching as adjunct professors or guest lecturers.
He also suggested the creation of a fellowship program—not unlike the well-regarded AAAS Science & Technology Policy Fellowships—that would give science policy researchers an opportunity to work in a federal agency or congressional office. Such a program “would allow them an opportunity to learn first-hand the language, needs, and priorities on an agency, department, or congressional committee,” Teich said.
Daniel Sarewitz, co-director of the Consortium for Science, Policy & Outcomes and a professor of science and society at Arizona State University, said it is essential for researchers to do a better job of measuring the real benefits of R&D spending. It is easier to measure outputs, such as number of scientific research publications, than the outcomes, he said. The 40-year war on cancer “has yielded remarkable new scientific knowledge, yet remarkably modest public health benefits for the tens of billions spent,” Sarewitz said.
Julia Lane, director of the SciSIP program for the National Science Foundation, said better tools to assess the impact of science also should bring practical benefits for federal program officers. “In a nutshell,” Lane told the panel, “you can’t manage what you can’t measure.”
In one federal initiative, called STAR METRICS, jointly led by NSF, the National Institutes of Health and the White House Office of Science and Technology Policy, researchers will collect data on the number of jobs created, retained or lost under the federal economic stimulus program. More than 100 research universities have expressed an interest in participating. The STAR METRICS approach has drawn the interest of European, Brazilian, and Japanese science agencies as well, Lane said.
Fiona Murray, an associate professor of management at the Massachusetts Institute of Technology, said the importance of the SciSIP program goes beyond its emphasis on data collection. It also encourages more searching analyses, she said, that can shed light on “whether and how particular policy interventions have an impact.”
In the end, the key is whether new findings are adequately communicated to policy-makers and integrated into workings of Congress that can seem remote and arcane to scientists, Teich said. “In some respects, physics is a lot easier,” he told the panel. In freshman physics, he said, you can be asked to assume a frictionless plane and it works for the problem at hand. But “assume a frictionless Congress,” Teich said, “and you’ve got nothing… Politics isn’t neat, and data doesn’t always trump a lot of other factors that go into decisions.”
Links
View written testimony and the webcast of the hearing.