Sharp-eyed budget observers may have noticed something unusual of late in federal R&D expenditures: official data from the White House Office of Management and Budget (OMB) indicates federal R&D, as of the 2017 fiscal year, is roughly $20 billion less than it had been previously. (This drop can be readily seen in the AAAS historical data dashboard and in certain graphs and figures reliant on data from OMB and other official sources). The drop occurred in spite of the fact that Congress provided some fairly generous increases for science agencies that year. So what happened?
As it turns out, it’s neither a data error nor nefarious accounting: rather, it’s the result of recent changes by OMB to the official guidelines that define categories of R&D. These changes – which have taken effect after some years of deliberation – mean agencies now count R&D and non-R&D activities slightly differently, compared to what they counted pre-2017.
For more info on the changes, AAAS reached out to John Jankowski and Christopher Pece, experts with the National Science Foundation’s National Center for Science and Engineering and Statistics (NCSES). Jankowski and Pece have been deeply involved in the discussion over R&D definitions for several years within the U.S. government and through the Organisation for Economic Cooperation and Development (OECD), the international body that sets global standards for tallying R&D. Their consolidated email responses are below, and have been edited for length and clarity.
Under these recent changes, what is now being excluded? What’s still included as R&D?
The primary change was to exclude “pre-production development,” which is defined as non-experimental work on a product or system before it goes into full production, including activities such as tooling and development of production facilities. These are usually synonymous with the “operational systems development” account in DOD’s budget activity structure, but are not necessarily exclusive to the DOD category [AAAS note: see figure 1].
Research, both basic research and applied research, remain unchanged in the reported totals. Experimental development is [still] included and is defined as “creative and systematic work, drawing on knowledge gained from research and practical experience, which is directed at producing new products or processes or improving existing products or processes.” Consequently, experimental development still includes the production of materials, devices, and systems or methods, including the design, construction and testing of experimental prototypes. Experimental development also continues to include technology demonstrations, in cases where a system of components is being demonstrated at scale for the first time, and it is realistic to expect additional refinements (feedback R&D) following the demonstration.
Beyond DOD, what are the major agencies effected by the change, as far as you’re aware?
Other than the DOD, we know that NASA was affected by the change, but not nearly to the extent as DOD. The change reduced the DOD R&D total by more than $20 billion [AAAS note: see, for instance, figure 2], whereas NASA estimates that the revised definition reduced its R&D total by about $2.7 billion in FY 2017. It was also possible that some Department of Energy obligations would be affected, but DOE has informed us they had already been excluding these types of activities from previous reports.
What motivated the changes?
In October 2015 the Organisation for Economic Cooperation and Development (OECD) issued the seventh edition of the Frascati Manual: Guidelines for Collecting and Reporting Data on Research and Experimental Development.[i] Although the international standards always referred to the “D” in R&D as experimental development, the issue of pre-production development as being outside the scope of R&D was made considerably more explicit.
The Office of Management and Budget updates Circular A-11 each year as part of the President’s budget formulation process.[ii] As such, the 2015 edition of the Frascati Manual presented an ideal opportunity for OMB to update the guidance for federal agencies and explicitly call for the exclusion of operational systems development from the R&D totals.
What are the advantages of adopting a narrower definition of R&D? How does the new definition fit with common practice elsewhere?
The underlying definition of R&D has not changed (creative work undertaken to expand the stock of knowledge…). What has changed is how that definition is translated into language that is meaningful to the organizations from whom the data are collected. By more explicitly defining “development” as “experimental development,” the data are more consistent across federal agencies, are more consistent with the R&D totals reported by other sectors of the economy, and are more consistent with international standards used in reporting R&D totals by other countries.
Based on discussions with several private sector businesses, they generally do not consider much of the operational systems development-type activities to be R&D, and therefore would not have reported such activities as R&D on the NCSES/Census Bureau Business Research and Development and Innovation Survey (BRDIS). One result was that comparison of federal R&D obligations to industry (reported to the NCSES Survey of Federal Funds for Research and Development) were consistently overstated compared to what industry reported as R&D funds received from the federal government. This change is likely to result in more consistent reporting across sectors within the U.S.
The change should make the U.S. data more comparable with budget data from other advanced economies in the OECD, all of whom follow the guidelines in the Frascati Manual for collecting and reporting R&D statistics. These countries have traditionally excluded pre-production development-type activities from their R&D statistics. Many countries have for years explicitly used the term “experimental development” on their survey forms.
NCSES conducts extensive surveys of R&D performers beyond just the federal agencies. Do you expect these changes to be reflected in the data you get back from non-federal sources?
NCSES has updated all of our surveys of both R&D performers and funders to reflect the latest definitions and type of R&D (basic research, applied research, and experimental development), as defined by the 2015 Frascati Manual. We have not (at least yet) observed a change in business sector reports on R&D stemming from this definition clarification, which is consistent with our understanding that many businesses do not consider pre-production development to be part of their R&D, and therefore such funds would not be reported as R&D funded by the Department of Defense. We are also making adjustments to our imputation methods to ensure their exclusion from the national business R&D totals. There does not seem to be any change in R&D totals reported for or by other non-federal performers as a result of the definition change.
[i] The Frascati Manual is essentially the international standard for assembling R&D spending statistics in the industrialized world. - AAAS
[ii] Circular A-11 is the annual circular published by OMB that guides federal agency budget preparation. The whole thing – clocking in at 948 pages – is online; R&D-specific guidance is in Section 84. -AAAS