Good Science Plus Bad Management
Equals Bad Science
Jon M. Veigel
SunRunner Associates, LLC
Surprise, AZ
1. Setting the Scene for Multi-institutional Research Collaborations
a. Overview
Thinking about multi-institutional research collaborations ought to begin at the beginning. In Merriam Webster's Collegiate Dictionary, the first two definitions of the word "collaborate" are:
1. "To work jointly with others or together esp. in an intellectual endeavor."
2. "To cooperate with or willingly assist an enemy of one's country and esp. an occupying force."
The first definition satisfies our expectations of how a multi-institutional research collaboration would work to meet its goals and objectives. The second definition reminds us to be cautious, to recognize that the ‘soft’ issues of managing joint efforts are always important in meeting those same research goals and objectives, especially if friends become enemies.
Multi-institutional research collaborations involving universities, companies, and/or government have become more common in recent years. They offer, they are coming to be seen as effective organizational structures to cope with the seemingly geometric escalation in the scale, complexity, and resource intensity of research opportunities. They can be especially productive when the skills and talents brought into the collaboration by each institutional member creates a whole more powerful than the sum of the individual elements. Research collaborations are particularly valuable when the outcomes produced would otherwise have been unthinkable.
American industry has been more active in establishing consortia than has academia. The collaborative forms and purposes they have tried—from those between individual companies crisply focused on a particular product, process, or material to industry-wide efforts addressing fundamental issues of broad interest—have, in effect, been a test bed to discover what works and what doesn’t. In many cases, a key insight has been to recognize the fundamental importance of applying good management practices to the collaborative endeavor. They have come to recognize that treating the management of multi-institutional collaborations with benign neglect puts the whole enterprise at risk. The "best practice" techniques for operating collaborations harvested from the breadth of experience in the private sector are often perfectly transferable to research collaborations managed by universities.
The undeniable appeal of consortia at the right time and for the right purposes cannot be generalized to consortia at any time for all purposes. Pollyanna notwithstanding, collaborations can be enormously wasteful of time, talents, and resources. Failures of the first kind are those in which the research done was flawed in concept or execution. Failures of the second kind are those in which the key management of the collaboration through their sins of commission did the wrong thing or through their sins of omission did not do the right thing. Regardless of prime cause, the result is the same: the collaboration, whatever it produces, produces significantly less of value than it could have. Failures can be obvious and undeniable; failures can also be more subtle and more deniable—management declares the collaboration a success and moves on.
Every collaboration begins with the expectation of success; no one willingly chooses to be in a consortium certain to fail. Yet fail they do. This paper is designed to help the managers of prospective or existing research collaborations identify and apply ‘best practice’ techniques of modern management. Used with wit and care, these practices can help to minimize their chances of failure. While such an outcome is obviously devoutly hoped for by the collaboration’s management, these techniques serve more optimistic purposes by maximizing the chances that the collaboration meets or exceeds the expectations that motivated its establishment.
The craft of managing science, however, seems to be viewed by many scientists with about the same respect as the public has sometimes viewed the craft of novel writing. Alexander Theroux has noted to that "Tolstoy, to amuse himself, would ask people in passing, critics of his own work especially, if they thought they could play the violin, to which with horror virtually everyone would cry, "No!" But when asked whether they could write fiction, they would reply without so much as a pause, "Yes."
Practicing scientists have a version of these sentiments if they believe that management is a skill more akin to the skill of writing fiction than to the skill of playing the violin. The assumption that, having ourselves been managed we can be a manager, is an understandable human conceit. However, having been managed is no more the single necessary qualification to manage well than is having read a novel the single qualification needed to write a good novel.
In fact, good management, like good science (or novel writing), is an artful blend of wit and wisdom, of tools and processes, and of insights and serendipity. The managerial skills and sensitivities of senior research managers of multi-institutional collaborations can increase the chance that their research will be done well, on time, and on budget. Creating effective multi-institutional research collaborations often requires management skills that are different both in kind and degree from the managerial skills usually expected of researchers in the past. However unfamiliar academic researchers are with these skills, they have become part and parcel of research management in industry and, to some extent, in federal laboratories. Much like good ideas in any field, they have the undeniable attraction of being obvious once realized, and low cost or no cost in their application.
The willingness and the ease with which researchers believe in the value of these skills is partly a function of the culture existing both in their particular discipline and in the agencies on which they depend for funding. Because, for example, of their inability to do cutting-edge research without access to a very limited array of very expensive machines, experimentalists in high-energy physics have long been accustomed to participating in multi-institutional and multi-researcher collaborations. Such participation has necessarily meant that successful senior researchers have developed and applied a broader range of management skills than required of equally successful researchers in fields where single-investigator research is still possible. The American Institute of Physics "Study of Multi-institutional Collaborations" explores these issues in more depth.
My intentions are to begin with a broad review of the context that provides the rationale, the opportunities, and the necessities driving the growing attractiveness of serious collaborations in university research. The issues selected represent neither a comprehensive, nor a complete, list of all issues of possible managerial relevance. They have been selected because of the likelihood of their having a major impact on the trends in and results from university research. While a commonplace observation, it is nonetheless true that the trends that will impact research in the next few years were shaped in the past and are beyond our ability either to change significantly or to do much other than react to. Watching the appearance and developments of trends in society today, particularly in technology- and science-dependent businesses, may, by analogy to our current experiences, pay dividends to those willing and able to position their research in anticipation of trends not yet widely acknowledged for their likely impact on the research environment.
The next two sections then deal more directly with the management issues critical to two distinctly different stages in the life cycle of a multi-institutional consortium. The processes used to design the new collaboration providing the organizational backbone of a research proposal meant to be nationally competitive are substantially different from the skills and techniques that will sustain and improve the operation of a multi-institutional collaboration once an award is made. In neither case do I (nor could I) present a management recipe universally guaranteed to produce the desired results. Would that it were possible!
Rather, my intention is that the reader's managerial intuition will be validated and informed, and that the ideas presented will provoke insights about how new or different management approaches can help realize his or her particular research goals. The references to the management literature cited provide entry points to a rich resource of papers based on actual managerial experiences organized to illuminate the fundamental meaning of those experiences. Since that meaning and those experiences may be directly relevant to improving the processes used in a multi-institutional consortium I have included a larger number of papers than I actually discuss.
b. The context for change
The economy of the United States over the last few decades has demonstrated both the promise and the trauma of rapid change. Innovators who recognized and exploited opportunities that lay just out of sight to the rest of us have dazzled us with their successes. In contrast, optimizing what already exists no longer brings the economic success it once did.
It belabors the obvious to observe that the climate for university research has been impacted by changes and the rates of those changes in the rest of society. In some cases, the university is being driven to change, in others it is being drawn to change. The late Congressman George E. Brown, Jr. pungently described the ways in which science was serving, and not serving, the needs and expectations of society in a paper he called "The Objectivity Crisis."
It took time before university research began to experience the kinds of changes in research norms, expectations, and practices that industrial research had been forced to accommodate to.
A good measure of this ferment is to note the variety of the groups that have examined multi-institutional consortia. For example, the National Science and Technology Council issued a report in 1999 that called for the renewal of the federal government-university research partnership for the 21st century. The Business-Higher Education Forum established the University-Industry Research Collaboration Initiative to "... review issues such as ownership of intellectual property, freedom to publish research findings, patent and license policies, and organizational culture and communication." The "Industry-University Research Collaborations: Report Of A Workshop" co-sponsored by the Industrial Research Institute, the Government-University-Industry Research Roundtable, and the Council on Competitiveness is another token of broad national interest in research collaborations at policy levels in the private and public sectors.
Predicting the future of research has been compared to the difficulty of crossing a chasm in two leaps. Thinking through their strategic choices about research will help a campus to define the areas for which a multi-institutional collaboration is the option of choice. There are a number of developments that could have a significant impact on university-based research.
Taken to its logical conclusion and generalized as a drive for zero latency, the instant availability of whatever is needed to complete the next step of any process. Zero latency has become a goal in processes and organizations where minimizing capital used for facilities, equipment, material, or staff frees that capital for more productive investment elsewhere.
While an exaggeration meant to emphasize the point, the drive to eliminate latency can help explain the source of some of the confusion and stress experienced by those parts of the research enterprise of interest to profit-making companies. The tension between an industry sponsoring a project and the investigator over how quickly the research data will be available for company use is but one example.
Recognizing the possibility that university research can lead to economically valuable products, the normal development pattern for research can usefully be focused and expanded. Collecting the data, analyzing it to produce information, and synthesizing that information to produce knowledge is a standard description of the research process. Noting that integrating that knowledge produces intelligence, and finally that intelligence can be applied to produce economic value, extends the concept of research as historically practiced on American campuses. In "normal" research, each step is separated from the next by a period of latency. Any pressure to reduce this latency, an essentially invisible part of the cascade of the research process, will inevitably constrain or change the research process itself. Zero latency is a concept most easily supported when it is a requirement placed on others; good and sufficient reasons proving the impossibility of zero latency when applied as a requirement on our own work are always instantly obvious.
Though researchers may well see zero latency and research as mutually exclusive, there is no bright line that separates them. Zero latency is a good example of how the research community finds itself both resisting and accommodating new ideas that seem so straight forwardly appealing elsewhere in the economy, yet are so anti-thetical to historic patterns of research practice. Multi-institutional collaborations, particularly when there is no tradition of common practices, are unfortunately fertile grounds for new enthusiasms. In the circumstance, the only nearly certain outcome is the need to deal with unanticipated consequences.
While industrial experiences provide analogies for what may be future experiences of research in the American university system, there are examples from higher education itself. Nationally ranked researchers, their graduate students, and their equipment moving en mass from one university to another illustrate a clear version of an acquisition by one university of a resource from another university. Regardless of the usually diplomatic language of higher education used to describe the move, few doubt that an ‘unfriendly takeover’ has occurred. In effect, the movement some years ago of the magnet lab from the Massachusetts Institute of Technology to Florida State University represented a bold example of an acquisition, albeit through the vehicle of competition under a National Science Foundation Request for Proposal.
I believe that the factors have begun to fall into place in our university system for what, in industry, would be called value-chain integration. Words such as outsourcing, acquisitions, alliances, and collaborations are almost code words for the fundamental transition we are undergoing. Depending on its own imperatives and on its place in the value chain, universities, for example, may benefit from outsourcing elements of its research enterprise to others and/or from accepting work that is outsourced from other organizations. Looked at through this lens, a multi-institutional collaboration works only by outsourcing. Each university in a collaboration outsources tasks to its other institutional members and simultaneously benefits from the outsourcing of research to it by those same members.
Alliances and collaborations among and between universities range from the merest pro forma public assertion of a joint future to serious attempts to build value-added joint ventures. While now floundering, the alliance between hospitals associated with medical schools at Stanford University and the University of California, San Francisco was a joint venture unthinkable only a few years ago. That it happened at all was due to the recognition by both institutions that such an alliance offered the least objectionable response to federal and public pressures to contain medical costs.
If these now isolated examples become trends, major efforts by universities to acquire resources already existing at other institutions could roil research in universities as nothing before it.
The act of knowing the unknown has long been a fundamental bedrock of university research. The use to which those research results were applied, if any, were usually separated in time and in place from the actual act of the research.
Such separations were usually viewed as a virtue of the research. No longer. The primacy of knowing as an intellectual process ongoing throughout a professional academic career has been supplanted in many fields by the recognition that knowledge in and of itself is an economic commodity. The free market economy is necessarily biased toward promptly realizing the economic value of such commodities. Technology by technology, discovery by discovery, as soon as the free market becomes capable of harvesting the economic value of that technology or discovery, knowing becomes knowledge and zero-latency begins to exert its demands.
Even the use of the word commodity imposes a particular interpretation of events. Premium, high-value products compete on a wide variety of value-added characteristics; price itself may have little or no impact on the decision to purchase. Commodity products compete primarily on price alone. When it first appeared in American grocery store produce aisles, for example, the kiwi fruit from New Zealand was viewed as a premium product because of its novelty value and its limited supply. Quite naturally, it had a premium price. Now that the United States produces much of its own kiwi fruit, its price as a commodity has dropped.
Researchers have been the beneficiaries of a long-held public view of research as a premium product. For these many years and in many fields, the cost of research was not the central issue, especially when the research could be cast, even tenuously, as satisfying broad national purposes such as defense or health. The substantial external pressures imposed on research practice in recent years have shown that research, while not being viewed as a commodity like cotton or gravel, can still be quite productive even when subject to many of the same commodity-like cost considerations that discipline other parts of the economy.
Increasing pressures on university researchers to avoid, reduce, and eliminate costs have brought a discipline to the conduct of research not previously seen. Part of the appeal of alliances lies in their capacity to argue their effectiveness, even in face of such economic pressures.
The visibility of university research to the public has increased substantially. With it has come demands to improve research competitiveness both relatively and absolutely. Particularly in public universities, citizens have begun to view research in terms that are essentially opportunity-cost oriented. Parents and legislators see professorial time and resources devoted to research as diminishing the time and resources available for teaching taxpayer’s children.
It has become a common expectation that American companies will attempt to influence governments to enact policies or regulations that either favor their own company and/or act to hinder their competitors. In general, American universities have acted in concert through their national organizations to influence policy development. Such actions, by their very nature, must represent a consensus view of what is best for the higher education community across the country. Not unexpectedly, the nature of the policy imperative will change depending on the particular higher education community being served. Community colleges and Carnegie Research I institutions have markedly different views about the nature and scale of federal support for research -- differences that show quite clearly in the attempts by these communities to affect the decisions about the allocations of federal dollars.
The country’s next economic recession, and its attendant pressures to reduce federal support of research, may well lead to universities attempting more aggressively than to date to use laws and regulations as policy tools of choice to protect themselves, to earmark new research areas, or to hobble their most serious competitors. The application of the collective power of university members of a collaboration to accomplish these same ends may be even more tempting as real or perceived threats to research escalate. As Oliver Wilde once noted, he could resist anything but temptation.
The degree of federal support for research has led to the constant examination of the future actions and intentions of federal agencies. Even without their intending to do so, federal agencies with programs to support research may stifle the possibilities for collaborations. They may, for example have research programs too weak or too underfunded to adequately support collaborative work. For historic or political reasons, an agency research program may be overly focused on areas no longer as cutting edge as they used to be. They may pay little attention to the complementary aspects of collaborative research. Completely new or cross cutting initiatives may fail to find support because of agency missions as defined by law, policy, or habit. And finally, actions by the Congress that add or eliminate research support are often the result of factors unrelated to perceived research value. Funding for new collaborative efforts will be viewed simultaneously as a long overdue recognition of research potential and as the cruel denial of support owed to non-consortial research.
2. Managing the managing of research: Thinking about organizing multi-institutional research alliances
Multi-institutional collaborations offer the possibility of outcomes otherwise unavailable. The research itself may have been impossible to do because no single campus could organize the scale of resources needed. Successful collaborations may offer benefits collateral to the research, such as an increased visibility for the university and for its faculty. The institution may find that the problem of faculty migrating to universities with better facilities is reduced. The university's portfolio of research may broaden as new funding opportunities are identified. A university may find itself able to pioneer and/or dominates in new research areas that would previously have been beyond its grasp. And finally, successful collaborations may help buffer the university against changes that are rapid and/or unexpected.
There is a wide variety of papers and books about collaborations. Individually, the focus of each ranges from an interest in international collaborations, to academic research collaborations, and in purely industrial collaborations. Collectively they map the possibilities, the problems, and texture, feel, taste and smell of collaborative activities. Representative papers are those by Haskins, Senker, Inkpen, and Marshall.
The backdrop for action
University Chief Research Officers seldom have resources sufficient to satisfy their current needs, much less than what is needed to position themselves adequately for whatever future they eventually must deal with. That future faces them with the challenge of coping with new factors even less under their direct control than the research of their own faculty ever was. Multi-university collaborations, value-chain integration, and other organizational arrangements yet to be devised will demand very different organizational responses on the part of universities.
One example already confronts them. The changes in the form, function, and expectations of industrial research have directly impacted university research. Industrial support for research that had become customary and expected disappeared, sometimes almost overnight. In parallel, industries have begun to provide support that formerly would never have been even imagined.
Feller , Teich, and other researchers have analyzed factors that correlate with universities recognized for their national success in research. There is no general agreement as to the factors that are most important. Universities interested in raising their research competitiveness can measure their progress by tracking their improvement in the factors they have selected as most useful in their situation. Bearing in mind that correlation is not necessarily causation, interested universities could, for example, work to improve their research competitiveness by improving some or all of the six factors listed by Feller in reference 12:
The university supports a goal to meet national standards of research performance.
The university must have a faculty that was recruited nationally, is productive in its research, and recognized for that productivity in the relevant national research communities.
The cadre of graduate students must be of high quality.
The university must have faculty salaries that are competitive nationally.
Faculty teaching loads must be competitive on the national scene.
The university itself must have an organization and policies that are flexible enough to support an active and productive research environment.
Each of these factors lends itself to factual analysis. While definitional disagreements may create fuzzy boundaries, unambiguous data exists to support a conclusion whether or not a given factor is present on a campus. Much less quantitative in nature, but as equally important a factor in my view, is the "white space" that will determine how effective any list of factors will be in producing a competitive national research portfolio. Most easily visualized by analogy to the literal white spaces seen around the college and department boxes found on a university organizational chart, the white space in the sense intended here is composed of all the those characteristics that actually animate an organization. The campus culture, the openness of communication, the morale, and the leadership styles all illustrate the components of the ‘white space’ that make it possible for the more measurable factors usually cited to work together to produce the research results observed.
The balance between classic and collaborative research is illustrated in Figure 1. Assuming that the center of this figure represents the present day, moving up on the vertical axis reflects an increasing number of collaborations. Going down on the same axis indicates a decrease in the number of collaborations. The horizontal axis starting from the same center point can show either an increase in the amount of classic research moving to the right, or a decrease moving to the right. Each of the cells so defined suggests a story line, a metaphor that instantly suggests the circumstance produced by that particular permutation of factors.
In the upper right hand corner, The Pie Expands, the simultaneous increase of collaborative activity and classic research activity might easily account for the renaissance of that university. The lower right hand quadrant where classic research is seen to be expanding at the same time as collaborations decrease, the story line could be could be labeled The Past Prevails. In direct contrast, the diagonal cell, labeled Corporatizatation Of Research, with the university experiencing a drawdown on its classic research activity while experiencing a substantial increase in multi-institutional research activities. In the final quadrant on the lower left, collaborative research and classic research both decrease. The story line metaphor here could be entitled The Famine Descends.
b. Planning under uncertainty
Though harder to put into practice than to describe, each university has three strategic options about its future research pathway. Any option chosen will reflect, among other things, and a valuation of the uncertainties to be faced over the time being considered and the likely ability of the university to control those uncertainties. Experience and competence in multi-institutional research collaborations increases a university’s flexibility and resilience.
Courtney offers a convenient framework for thinking about strategic choices and uncertainty. The first of three strategic choices he discusses is the boldest. In this option, a university will choose to try to shape the research future within which it will operate. Doing so obviously implies both a conscious intent to be a national leader in research and the sustainable intent to invest the resources needed to make it happen. A second strategic choice is adaptation. In this case, the university chooses to believe that its best option lies in the flexible, rapid, and agile application of its existing research resources toward recognizing and capturing the opportunities that are presented to it.
The third option, selected either by necessity or choice, is to wait and see. This option does not propose no action; but rather requires the action of careful and watchful attention so that when circumstances change university administrators can review whether choosing a different strategic posture is now warranted. Clearly, consortia can be powerful tools of choice for universities intent on either shaping their future or actively accommodating to it. At least from the perspective of a university choosing to wait and see, collaborations seem a less attractive option, though faculty may find them quite appropriate in given circumstances.
On its way to choosing among these strategic alternatives a university necessarily works to understand the kind of uncertainties they face, to judge the level of those uncertainties, and to assess their abilities to control those uncertainties. The work of blank lays out four levels of uncertainty. The first is a future thought to be well enough understood so that today’s actions can straightforwardly determine tomorrow’s actions. A second level of uncertainty is only slightly more complicated. In this case, a decision tree can visually represent the limited number of futures that are thought to be possible. Techniques developed in the last decades can be applied here to sort through the probabilities that various actions will produce the desired results.
Addressing five questions blending both strategic and tactical factors can help a university Chief Research Officer intent on maintaining, improving, or reaching national research recognition begin to map the future.
Pursuing a strategic research direction, forestalling a competitor’s initiative, or establishing a research consortium all require making choices among alternatives. Most managers prefer to see themselves, and be seen, as making decisions based on logic and available evidence. Scientists with a managerial role play, given their rigorous training in the scientific method, certainly sure this preference. Yet, despite its seductive appeal, experienced managers recognize that decision-making is more often than not profoundly shaped by factors lying completely outside the envelope of any description of the scientific method.
The work by Hammond explores in some detail what they describe as hidden psychological traps that can play an overly important role in decision-making. They describe three traps. They call the first the Anchoring Trap in which "... the mind gives disproportionate weight to the first information it receives", the second trap is the Status-Quo Trap in which decision-makers display "... a strong bias toward alternatives that perpetuate the status quo". They call their final trap the Framing Trap, in which ultimate decisions are made likely or important just by the way the problem or the opportunity is originally framed.
Other useful papers in this general area are those by Shaw, Sahlman, Sharpe, Williamson, Kelly, Miller, and Quinn .
3. Managing the Managing of Research Collaborations: Producing Powerful Proposals
Developing a good multi-institutional research proposal is hard work compared to the work of preparing the ‘usual’ kind of research proposal. Even when proposal team members have some experience in preparing collaborative proposals, the complexities inherent in building a collaboration involving different institutions and/or disciplines can be daunting. When team members have not previously developed a consortium, learning by doing only increases the difficulties.
a. Managing risk
Any collaboration is by its very nature risky. University Chief Research Officers and Principal Investigators for a proposed collaboration are key players for the due diligence analysis of the risks of the proposed collaboration. Many of these risks have long been recognized and addressed, such as the seemingly endless battles over intellectual property where the interests of the project funder and the researcher may look orthogonal to one another.
Collaborations bring the imperative for a comprehensive risk analysis into sharp focus. Some of these risks will be common to all proposals, while others are unique to collaborations. The risk management analysis will address issues such as intellectual property, conflicts of interest, collaboration-specific assets, partners as competitors, intangibles such as goodwill, joint liability, disruption of ongoing research, key staff departures, and unmet obligations. Only a careful inventory will identify research risks inherent in the money, capital equipment, personnel, and process to be committed to an alliance. Individual risk exposures must be reduced or buffered. Future risks must be minimized or eliminated. The collective risks to a campus from their joining a collaboration must be assessed and prioritized.
Analyzing the risks of possible collaborations can also be viewed somewhat more broadly than a focus on risks primarily involving money or legal exposures. Here the perspective of the potential Principal Investigator may differ from those of the university research officer. Perhaps for that reason alone, questions such as the following will be useful to answer:
b. Thinking about the types of collaborations.
Collaborations can very widely in their scale and in the type of management necessary for their successful operation. Large research centers, such as the Engineering Research Centers funded by the National Science Foundation in recent years, represent a significant capital investment on the part of the federal government and others. The level of this investment and the high profile that such centers have on a given campus lead quite naturally to the recognition that professional management techniques are required.
At the opposite end of the spectrum are small collaborations that may more closely resemble a classic research model. Perhaps, for example, a few professors see the advantages of combining their efforts. In contrast to the large Centers just discussed, available equipment may shape the research that is possible. The Principal Investigators may well view their management responsibilities as penance for their past sins and/or as the price they must pay to do the research itself. It is therefore unlikely that much attention will be paid to using optimal management techniques. This may be the most sensible option in view of the likely costs to be incurred for the benefits realized.
What lie between these two extremes are multi-university collaborations that share few of the characteristics of either large Centers or small-scale alliances. Yet it is just this very class of collaborations that could easily become the dominant force in university research in the future. Good management for these consortia will pay dividends evident either to the researchers themselves or to accountable university or funding agency administrators.
There are at least five different ways to visualize collaborative research joint ventures:
Collaboration here might be an arrangement between two faculty who see such a collaboration as a way to meet some externally imposed need or to satisfy some individual want. It is a marriage of convenience that is never consummated, never annulled. Without being overly cynical, perhaps two faculty agree to cite each other as participants on their individual research proposals in order to create the gloss of apparent multi-institutional research.
This type of collaboration is actually better described as a quid-pro-quo exchange, a service offered for a service rendered. Such a service might be provided by one faculty to another that meets both their needs. For example, the agreement by a researcher with a unique piece of equipment to carry out an analysis for another researcher that could lead to a jointly authored paper.
Much as in the exchange-of-services collaboration just described, each party in this type of limited collaboration has a need to be met and a contribution to offer. However, in contrast, the essence of a true collaboration is first seen here: creating new values together. Groups with similar research interests could, for example, pool their unique resources for a limited time and/or a limited purpose.
In this collaboration, similar groups pool complementary resources for ongoing work that creates new value. Despite the complementarity of their skills and interests, each group comes to the collaboration precisely because they recognize that working outside the collaboration it would be unlikely or impossible for them to produce the results that entices them to work together.
This collaboration is built on the premise of benefits to be realized from value-chain integration. Teams may share interests only in the product of the consortium and/or may owe their allegiance to very different disciplines. The motivation for all members to join the consortium is the possibility that groups so different can produce unique research values and pioneer completely new research areas. When coupled with the likely absence, at least for a while, of any similar concatenation of skills elsewhere, their monopoly position can give them an immense competitive advantage.
Kanter brings life to the concept of a consortium by her use of personal relationships as a metaphor. She organizes her examples about the process is needed to develop a collaboration around stories of selection, courtship, getting engaged, and setting up housekeeping. In her view, collaborations work best when meaningful integration between members occurs at five levels: the strategic, the tactical, the operational, the interpersonal, and cultural.
c. The agreement to collaborate.
Like any successful common endeavor, there is a set of truisms, almost a set of homilies that will shape and measure the collaboration. In the best of circumstances, members of the collaborative team will recognize, nurture, and live by them. In other circumstances, teams may deny, ignore, or be oblivious to one or more these factors.
Investing the time to write and fine tune an agreement that defines major issues for the collaboration is time well spent. Perhaps perversely, a formal consortial agreement may be most useful when the impulse is to claim it is unnecessary, Memories that can be fallible and/or self-serving and unexamined assumptions that can be wrong are slender reeds on which to lean for the effective operation of a serious collaboration. Careful agreements will:
-identify first among equals for each member
-designate specific individual responsibilities, authorities, and accountabilities
-establish functional inter-organizational and interpersonal communication channels.
-empower the internal Executive Committee
A consortial agreement that has been negotiated with openness, care, and luck can resemble a Nash equilibrium. At the signing, all parties recognize that further negotiation will not improve the results for all collaborative members. To the extent that the agreement is the best possible, incentives to sabotage the agreement by independent action are reduced.
The agreement permits, favors, or prohibits specific activities in order realized the outcomes desired. Seen from one perspective, successful collaborations, like successful companies, "... have explicitly defined and rewarded roles that facilitate knowledge capture, refinement, retrieval, interpretation, and use." Through their University Relations Committee, the Industrial Research Institute has issued a report on enhancing industry-university cooperative research agreements.
d. The value of governance and advice.
All collaborations, especially those that that are visible on campus and/or in the outside world or are composed of multiple institutions, will benefit from a governance structure that is a source of focused and unbiased insights and advice that would otherwise be difficult or impossible to obtain. Internally, an Executive Committee composed of principals from each consortium member reviews plans, budgets, processes, and results. The goal is to establish a comfortable way to reach consensus on issues likely to be divisive if they appear to be decided by the dictate of the Principal Investigator. Done right, a delicate balance is struck in which the Committee neither has, nor needs, actual decision making authority.
An External Advisory Committee made up of senior decision-makers from the constituencies served by the collaboration has proven invaluable where they have been established and employed effectively. They have at least two unique roles.. Because of their affiliations and the breadth of their personal experiences they can monitor plans, budgets and results in ways that consortial staff cannot. Their annual review of the performance of the Principal Investigator is meant to be useful input to the PI and to provide reassurance to members of the consortium and to the funding sources.
Properly constituted, members of the Committee can also provide entree for the leaders of the consortium to individuals at levels not usually accessible to the Principal Investigator and/or to organizations outside the boundaries of the common experience of the collaborative team.
The value to a collaboration of having one or more institutional champions is often overlooked or ignored. They are valuable precisely because they have no formal role to play in the collaboration. Such an individual or individuals can pay unanticipated dividends over the life of the consortium. Carefully recruited for their level of authority in the institution, their position, and their self-interest in the success of collaboration, such champions can defend and enhance the collaboration at policy and decision-making levels that would never be open to the Principal Investigator.
Once motivated and informed, such champions can build and sustain relationships with their peers at institutions that are also consortial members. Their assistance will aid in meeting objectives and commitments by increasing the likelihood that commitments made by the university will be met as promised. They can help monitor performance, resolve critical path issues, and serve as an early alert both for opportunities and for threats to the consortium.
In the best of all situations, they are coaches to the collaboration’s leader. In no sense is this mentoring role meant to imply that the head of the collaboration lacks some critical skill. Rather, the "champion" can act as a sounding board to the leadership, offering counsel from a one-step remove, yet from a base of knowledge and experience that can contribute to the overall chances of success.
Once again, the balance of power and authority is a delicate one. An institutional champion has no direct command authority, nor should any be needed. Conversely, the alliance leadership would be willfully obtuse to ignore the observations of such individuals. They don’t have to be obeyed, they just need to be taken seriously. And, like Caesar’s wife, they must be seen to be taken seriously.
e. The Special Wisdom and Tactics Team (SWAT)
Many of the issues dealt with in preparing a strong multi-institutional proposal will be one-time issues in the eyes of the proposed Principal Investigator, yet be issues faced repetitively by the research office. The research office is the perfect home for a mechanism that collects lessons learned from these repeated activities, organizes it to enable that information to be retained rather than constantly reinvented, and finds a mechanism to deliver that knowledge to proposal teams when they need it and in a form matching their needs.
One way to provide this service reliably, consistently, and accurately is to create a resource in the research office charged with these responsibilities. Instead of trying to provide these services on an ad hoc, added-duty basis, the Chief Research Officer could consider chartering a Special Wisdom and Tactics Team.
Such a SWAT team can help ensure that the management practices used by consortium are the best possible, not just the minimally sufficient practices. The SWAT team involvement in the development of competitive new collaborative proposals can help establish a competitive advantage for the university. By bringing common practice to the development of proposals across the campus its actions can help reduce risks and uncertainties otherwise faced by proposals prepared only by the researchers themselves. Properly used, it can contribute to both the efficiency and the effectiveness of a university research operation.
As it builds experience, part of the SWAT team’s task can be to insure that the lessons it has learned are applied to ease the preparation of future proposals elsewhere on campus. In effect, a SWAT team acts as a research-process incubator for the entire campus. It seems clear that the university research office must play the key coordinating and oversight role.
A SWAT team is a virtual resource until constituted and brought to bear upon the preparation of a given proposal. The resources, skills, time, and money invested in the team will determine its effectiveness. SWAT team members may be members of the university staff or they may be brought in from the outside.
Internal SWAT team members may otherwise have duties as members of the research office staff itself, may be faculty members with skills appropriate to the needs, or may be on-campus practitioners recognized for their best-practices skills. In some cases, it may be necessary to engage individuals outside the university as SWAT team members to fill gaps in a SWAT team for which no skills are available on campus and/or whose skills complement and strengthen skills that do exist on campus. Teams of consultants or companies may offer the right blend of services. In any case, the more regular their services on the SWAT team, the higher the quality of service delivered.
The activities possible for such a SWAT team will be determined by many factors. In some cases, the potential Principal Investigator and the proposal team will have the skills and experience to require only minimal assistance and proposal review. In other cases, a SWAT team that is both broader and deeper in its skills and experience may provide a larger proportion of the work required to prepare the proposal.
SWAT team activities may include helping to develop the management plan, identifying and recruiting high-level on-campus champions, helping to structure the internal Executive Committee, helping to design and recruit the External Advisory Committee, drafting the management plan, and reviewing the entire proposal. The SWAT team has no role to play in developing the research agenda, other than possibly assisting in the strategic planning process.
4. Managing the Managing of Research: Succeeding after success
Learning their proposal has been successful ought to remind researchers of the Chinese saying "May you get what you wish for". Its implied double message should give pause to their enthusiasm. Impeccable science and management are the means by which the researchers realize the blessings of this saying, rather than experiencing it as a curse.
Many scientists have views of management that are uninformed by experience or interest. There are two polar opposite views of the management of research. In one, management is seen as a cost to be minimized; a necessity to be endured. In the other, management is recognized as an investment intended to create research competitive value; an opportunity to be seized. These two views define the strategic landscape -- individuals and teams acting on their beliefs about management, whether recognized or not, will have fundamental impacts on the productivity of their consortium.
a. Thinking about process
Managing the collaboration means managing process. To do it right, managing process means the consistent and fair use of a comprehensive management framework that captures individual issues and requisite actions regarding them. The framework also arrays the issues in relationship to one another as an aid to setting priorities. The work of Garvin is particularly effective at framing these issues. His paper is the primary resource for this and the next two sections.
Even casual readers of the popular press have been exposed currently fashionable management descriptors such as Total Quality Management, re-engineering, and benchmarking. Each concept has its true believers, its consultants as priests of that faith, and books about them that are almost theologically prescriptive.
Their very existence demonstrates that management techniques are by no means static, but rather a ferment of attempts to improve management practice. The work by Malone and his colleagues at the MIT Center for Coordination Science reviews of their attempts to go behind the veil of buzzwords to understand management process at a fundamental level.
An interesting management initiative with good potential to be of use to university multi-institutional collaborations is the web-based "Best Practices Manual". It has been prepared by staff from various of the Engineering Research Centers at American University supported by the National Science Foundation. Though carefully noted as not being an NSF publication, has been created as a "how-to" manual for staff of existing or prospective university-industry research centers.
A selection of papers exploring management and organizational issues include Schafferand Dodgson.
To be effective, the research collaborations must pay serious attention to process. Some processes are going to be the responsibility of the managers. Other processes will involve the entire organization. Managers will be involved in direction setting, negotiating and selling, and monitoring and control. The entire organization will be involved in the various work processes, behavioral processes, and change processes. Table 3, taken from the paper by Blank, constructs the matrix defined by these two axes. Each cell in the matrix asks questions that well-managed collaborations should take seriously.
Organizational process
The same authors proposed a more detailed matrix for organizational processes. Here the processes for work, behavior and change are described in terms of their
definition, role, major categories and examples. A similar matrix at a similar disaggregated level of attention can be found in table ex for managerial processes.
b. Managerial process
The Principal Investigator and the staff of the collaboration will work together within the universe bounded by their abilities and their styles. Figure why shows the possible combinations of leadership styles. The vertical axis tracks the degree to which a given manager is supportive of staff. The horizontal axis defines whether the directive behavior of the manager is low or high.
Managers whose style is highly directive and who pay little attention to offering support to their staff are practicing a top-down form of management. While much less fashionable that it used to be, this hierarchical style is still perfectly appropriate in particular circumstances. For example, when a staff member new to the collaboration does not have the skills are experience necessary carry out a particular activity, strongly directive behavior on the part of manager may well be the only option. A managerial style characterized by low directive behavior and low supported behavior is a good definition of a manager who delegates. Low directive behavior and high supported behavior characterized a manager whose style is to share in support with his or her staff. And finally a manager that is simultaneously highly supportive and highly directive can be described as 8 selling or coaching style.
In chapter 8 of her book "The Change Masters", Kanter discusses the critical role of an entrepreneur within the organization, a role today likely to be labeled intrapreneur.
In my experience I found that people who have managed my activities throughout my career all too often ended up complicating, disrupting, delaying, or discouraging either me personally or the work I was involved with. Whatever the stimulus, the result was to create what amounted to a cost measured in lost opportunities. I'm equally convinced that through my own inattention, obliviousness, or clumsiness as a manager, my own staff could quite justifiably have concluded that I, too, had created these reactions in them .
In my view also, those managers for whom I worked all too seldom supported me, did little to enable me to get my work done effectively, failed to address critical path issues that only they could resolve, did not encourage me, or championed me or my work. Here, too, I believe that my personal record is a manager is decidedly mixed. In the perfect world, every individual in all organizations would practice a managerial golden rule: managing others as they would be managed themselves.
c. Inter-team Communication
Building strong and effective relationships among and between research teams unused to working together is a considerable challenge even in the best of circumstances. Alliances among different institutions further complicate the challenge because of their physical separation, whether on different campuses or at an industrial or governmental laboratory site.
The difficulties that can be encountered do not seem to increase either gracefully or arithmetically, especially when further complicated by a concatenation of teams from distinctly different disciplinary or experience backgrounds. Especially when driven by their eagerness finally to begin the research so long anticipated, it will be difficult for the teams in a collaboration to pay sufficient attention to the seemingly off-purpose, non-research activities that are fundamental, necessary, and pivotal for eventual research success.
Collaborating slowly but surely is a behavior to be admired, but probably seldom seen in new collaborations. Visiting other teams early and often can be prompted by the needs to discuss and refine the research agenda. At least as importantly, such visits help to tease out the compatibilities and disconnects the are sure to exist when research teams, each used to their own rhythm of working together, begin to build mutually satisfying relationships with one another.
The goal of such visits is effective communication and teamwork. Getting to that goal is no more a straightforward and linear process than is research. False starts, fumbles, and stumbles must be experienced both to be believed and to reach the point where research progress isn’t hobbled by talking past one another. Whether separated by distance, organization, culture, or discipline, each collaborative team operates with assumptions that may even be unrecognized or unexamined within an individual team, much less within other teams with which they must cooperate. Given this reality, it is folly to assume that any team will make the same assumptions as another team, much less that they will necessarily act in a similar fashion when confronted by similar facts.
In the summer 1999, NASA provided a classic example of how acting on unexamined assumptions can lead to project disaster. In calculating the engine burn required for a satellite to enter orbit around Mars, one of the two teams responsible for the calculations used the metric system, the other team used English units. Neither team noticed this discrepancy until after the satellite burned up entering the Martian atmosphere.
As the different teams in the collaboration begin the long process of learning to work together they must relentlessly spotlight commitments, risks, and promises. Disagreements must be noticed, acknowledged and resolved. One industrial staff member I spoke with had personally participated in nine different collaborations among industries that sometimes also involved universities. He had seen the problems that arise in collaborations when disagreements are neither acknowledged nor dealt with. His unforgettably vivid advice to fellow team members who find themselves disagreeing with actions being taken by the rest of the group is to "Vomit on the table!" to forestall sweeping potentially fatal problems under the rug.
Managers and their staff can evaluate the success of their relationship by evaluating themselves against seven categories. As displayed in table 3 the characterizations from stronger to weaker may allow individuals to judge the status of their relationships. The very least, discussions among team members and/or among senior management team of the phrases contained in this table should lead to productive discussions of the status of relationships in the collaboration and the things that can be done to improve those relationships.
d. Inter-personal communication
The strength of any collaboration lies in the skills and experience of its individual members and in their ability to work productively together. The individual members of any collaboration must be selected with attention to their ability to tolerate ambiguity, negotiate fairly, communicate openly, resolve all problems promptly, do what they say, fail faster, and learn eagerly.
Managers are responsible to evaluate the performance of their staff. Almost without effort managers find themselves slotting individuals in a performance range from outstanding to dismal. Managers can pleasantly and accurately view their successful staff as resulting from the individual’s own efforts and the support of the manager. Less pleasant to contemplate are the ways in which a manager contributes to the performance of the least successful staff. The paper by Manzoni reviews with what ease, and with what self justification, managers can create the conditions that lead to the failure of one or more of their staff, even when that is the last of their intentions. The authors review the nature of the relationship between a manager and a staff member and show ways to establish a relationship that can benefit both parties.
A mixed message from a manager may be delivered in a statement such as "Your decision was a good one and I'm overruling it." The manager can pretend that the message is mixed by sang that "You can be proud of your contribution". Some statements act to make the topic undiscussable "I feel good about this, and I'm sure you do, too." In some cases, statements can make the undiscussable undiscussable: "Now that I've discussed everything, is there anything else you'd like to talk about?"
Organizations act through their policies, their practices, their actions, and the culture they support to shield individuals from accountability and to deny itself the benefit of experience. The papers by Argyris very concisely illustrates the dangers in defensive communications. There are four types. We managed to hobble ourselves by our defensive this and even by good management practice.
Defensive behavior on the part of individual team members or, for that matter, teams themselves can be one of the elements that undermines the potential success of a research collaboration. Most team members, being human, act almost automatically to minimize their vulnerability, their risk, the possibility of embarrassment or humiliation, and any appearance of incompetence. Every researcher acts to avoid these results by, as Argyris has noted, trying to maintain unilateral control, trying to maximize winning and minimizing losing, trying to suppress negative feelings, and trying to be as rational as possible.
President Reagan's maxim when dealing with the Soviet Union before and after its collapse is perfectly applicable here: trust, but verify.
e. Productivity
Measuring the productivity of university research projects and the accountability for the results has often been done, if done at all, by noting after the fact the number of papers published and the students graduated. If these results are then retrospectively defined as the intended product of the research any reference to what might have been or possibly even what was promised is lost.
Difficult as they are to construct usefully, brighter lights are now beginning to shine on the issue of productivity and accountability. The Virginia Center for Innovative Technology, for example, issued an RFP that made a serious attempt to define the factors that it was going to use to determine accountability of particular research projects they supported. For their purposes, accountability was defined in three categories. The first was rapid response and education. Here the desired outcomes were to provide rapid response to business technological problems and opportunities, contribute to education and to the workforce, to provide student assistance to a company, and to provide formal education programs for students and/or businesses and industries. The metrics in this category were whether the companies felt satisfied with the student assistance they got and whether the new courses and/or workshops delivered to the students and the company met their needs.
Their second accountability category was technology development. The single desired outcome here was to facilitate and lead proactive development. The metrics they used looked at funding multipliers from both private and/or federal sources, examined research disclosures, patents, and papers; and to judge resulting research competencies.
The third and final accountability category was technology application and commercialization. Here, the desired outcome was to strengthen the academic and industrial infrastructure in Virginia. The metrics used examined the companies created, the assistance rendered, or retained in Virginia. They also tracked whether Virginia's products, prototypes, or processes had improved and/or been developed and commercialized. They checked whether jobs had been created and/or retained in state; judged the extent to which safety and/or environmental issues had improved; and worked to determine whether companies were satisfied by the services they received from the collaboration.
f. Outcomes
Collaborations are never undertaken in anticipation of failure. But not believing in the possibility of failure can by itself almost insure a self-fulfilling prophecy. When we say failure, most of us would imagine a collaboration that collapses. In fact, collaborations that fail can fail more subtly. Failures can be much like a cup that is described low as either half empty or half full: the collaboration prides itself on its accomplishments with out acknowledging the opportunities that were lost.
While lost opportunities may be seen in terms of missed research choices, less obvious are losses in opportunities and results that occurred because of management that was inexperienced, inattentive, or willfully blind to the possibilities. Collaborations can be threatened when, or if, one or more of the following situations arises:
This laundry list of problems may provide a checklist for the historical analysis of a consortium. More importantly, it motivates the synthesis of a list of "Commandments" that if implemented by a consortium help to increase the probability of its success. They are:
g. The SWAT team
The SWAT team described earlier in its function assisting in the preparation of better proposals, also has an important role to play in the successful in a of the time for hungry or in implementation of projects after they are awarded. No longer constrained by only providing the services needed to produce a winning proposal that meets the stringent requirements imposed on it, SWAT teams in this circumstance can provide a greater range of services.
Sample services that can be provided routinely, at the request of Principal Investigator, or at the instigation of the campus research office are:
provide process team assistance for planning, organizational development, and so forth
Universities face important choices in choosing how to use their SWAT team. Will SWAT team assistance be offered to consortium taking proactive actions or will SWAT team primarily carry out activities that are primarily reactive in nature -- for example, assisting consortium management that is under threat for loss of funding by its agency.
Independent of direct SWAT team activities, carefully designed policies can provide invaluable assistance to the consortium, particularly in its early start-up phase. General Electric, for example, gives its new start-ups a 100-day exemption from normal administrative requirements, with the exception of those policies and practices that are judged to be absolutely fundamental to the operation of the company.
Collaborations that succeed become what in business terms are described as "going concerns." Such a consortium will be comprised of institutional members and staff who recognize that the value of their mutual needs to work together are still significant and outweigh the costs associated with the consortium. They have a shared history of surviving conflicts. They share the desire to press forward with common objectives that are recognized by all. Commitments made by members to provide funds, staff, or other resources are routinely met. Trust within the consortium widens and deepens in ways that are visible to the staff. Common recognition of these factors encourages actions that would not otherwise have been taken. Finally, it will be recognized internally and externally that individual responsibility, authority, and accountability are the threads that have been woven together to become the very fabric of the consortium.
The situation to avoid is that described by Argyris(Reference 38, page 85): "… it is possible to achieve quite respectable productivity with middling commitment and morale. … In such a system, superficial answers to critical questions produce adequate results, and no one demands more."
References:
Theroux, Alexander (1999). "Tolstoy need not apply." in The Wall Street Journal, October 22nd, p. W15.
2 See http://www.aip.org/history/pubslst.htm#collabs as of 24 August 1999.
3 Brown, George E. (1993). "Guest Comment: The Objectivity Crisis" in the American Journal of Physics, Volume, No. 9, pp. 779-781.
4 See http://www.whitehouse.gov/WH/EOP/OSTP/html as of 6 October 1999.
5 See http://www.acenet.edu/about/programs/Programs&Analysis/BHEF/initiatives/uni-ind/home.html as of 9 September 1999.
6 "Industry-University Research Collaborations: Report Of A Workshop" (1995) co-sponsored by the Industrial Research Institute, the Government-University-Industry Research Roundtable, and the Council on Competitiveness. Duke University, 28-30 November 1995.
7 Scarbrough, Harry (1999). "Knowledge as Work: Conflicts in the Management of Knowledge Workers" in Technology Analysis & Strategic Management, volume 11, No. 1, pp. 5-16.
8 Elfring, Tom and Ard-Pieter De Man (1998). "Theories of the Firm, Competitive Advantage and Government Policy" in Technology Analysis & Strategic Management, volume 10, No. 3, pp. 283-293.
9 Haskins, Mark, Jeanne Liedtka, and John Rosenblum (1998). "Beyond Teams: Toward an Ethic of Collaboration" in Organizational Dynamics, Spring, pp. 34-50.
10 Senker, Jacqueline and Margaret Sharp (1997). "Organizational Learning in Cooperative Alliances: Some Case Studies in Biotechnology" in Technology Analysis & Strategic Management, volume 9, No. 1, pp. 35-51.
11Inkpen, Andrew C. and Kou-Qing Li (1999). "Joint Venture Formation: Planning and Knowledge-Gathering for Success" in Organizational Dynamics, Spring pp. 33-47.
12Marshall, Edward M. Transforming The Way We Work: The Power of the Collaborative Workplace, American Management Association, New York, 1995.
13Feller, Irwin (1999). "Strategic Options to Enhance the Research Competitiveness of EPSCoR Universities" prepared for the American Association for the Advancement of Science Workshop on Academic Research Competitiveness, Coeur D’ Alene, Idaho, September 30-October 2.
14Feller, Irwin (1996). "The Determinants Of Research Competitiveness Among Universities" In Competitiveness and Academic Research, edited by Albert Teich (Washington, DC: American Association for the Advancement of Science), pp. 35-72.
15Teich, Albert H. and Kathleen M. Gramp (1996). In Competitiveness and Academic Research, edited by Albert Teich (Washington, DC: American Association for the Advancement of Science), pp. 73-112.
16Courtney, Hugh, Jane Kirkland, and Patrick Viguerie (1997). "Strategy Under Uncertainty". In Harvard Business Review, N-December pp. 67-79.
17Hammond, John S., Ralph L. Keeney, and Howard Raiffa (1998). "The Hidden Traps In Decision Making" in Harvard Business Review, September-October, pp. 47-58.
18Shaw, Gordon, Robert Brown, and PhilipBromiley (1998). "Strategic Stories: How 3M Is Rewriting Business Planning" in Harvard Business Review, May-June pp. 41-50.
19Sahlman, William A. (1997). "How to Write a Great Business Plan" in Harvard Business Review, July-August pp. 98-108.
20Sharpe, Paul and Tom Keelin (1998). "How Smithkline Beecham Makes Better Resource-Allocation Decisions" in Harvard Business Review, March-April pp. 45-57.
21Williamson, Peter J (1999). "Strategy as Options on the Future" in Sloan Management Review, Spring pp. 117-126.
22Kelly, Kevin (1997). "New Rules for the New Economy" in Wired, September, pp. 140 et seq.
23Miller, J. A., P.M. Norling, and J.W. Collette (1997). "Research/Technology Management" in Encyclopedia of Chemical Technology, Fourth Edition, Volume 21, pp. 264-291.
24Quinn, James Brian (1978). "Strategic Change: Logical Incrementalism" in Sloan Management Review, Fall, Volume 20, No. 1, pp. 7-21.
25Quinn, James Brian (1980). "Managing Strategic Change" in Sloan Management Review, Summer, Volume 21, No. 4, pp. 3-20.
26Simons, Robert (1999). "How Risky Is Your Company?" in Harvard Business Review, May-June pp. 85-94.
27Kanter, Rosa Beth (1994). "Collaborative Advantage: The Art of Alliances" in Harvard Business Review, July-August, pp. 96-108.
28Zack, Michael H. (1999). "Managing Codified Knowledge" in Sloan Management Review, Summer, pp. 45-58.
29The University Relations Committee of the Industrial Research Institute (1995). "A Report On Enhancing Industry-University Cooperative Research Agreements", the Industrial Research Institute, Washington, D. C., 8 pp.
30Garvin, David A. (1998). "The Processes of Organization and Management" in Sloan Management Review, Summer pp. 33-50.
31Malone, Thomas W. et al (1999). "Tools for inventing organizations: Toward a handbook of organizational processes" in Management Science, Volume 45, No. 3, pp. 4 25-443.
32See http://www.erc-assoc.org/manual/index.htm as of 22 September 1999.
33Shaffer, Robert H. and Harvey A. Thomson (1992). "Successful Change Programs Began with Results" in Harvard Business Review, January-February, pp. 80-89.
34Dodgson, Mark (1999). "What Role for Management in Science?" in Technology Analysis & Strategic Management, Volume 11, No. 2, pp. 133-141.
35Kanter, Rosabeth Moss (19 -- --). "The Change Masters"
36Manzoni, Jean-Francois and Jean-Louis Barsoux (1998). "The Set-Up-To-Fail Syndrome" in Harvard Business Review, March-April, p. 101-1 13.
37Argyris, Chris (1991). "Teaching Smart People How to Learn" in Harvard Business Review, May-June, pp. 99-109.
38Argyris, Chris (1994). "Good Communication That Blocks Learning" in Harvard Business Review, July-August, pp. 77-85.