3/31/2014
The following is adapted from Dr. Harvey’s October remarks to the Congressional Advisory Panel on NNSA Governance. Dr. Harvey updated his presentation for NS&D Monitor this week. The draft recommendations from the Panel are expected to be released next week.
I am pleased to offer a perspective on the issues facing the Congressional Advisory Panel on NNSA Governance. First, I summarize my bottom line:
— Some things at NNSA are working well—e.g., science-based certification of stockpile safety and reliability;
— But some are not—i.e., delivery of warhead LEPs and related infrastructure on time/cost;
— Problem has two components: NNSA-specific, external to NNSA; and
— Multiple studies over the past 20 years agree on what needs to be done, but recommendations either have not been implemented, or implementation has failed.
Therefore, the Panel must:
— Understand why previous implementation has failed;
— Focus on effective and practical implementation of existing recommendations; and
— Carefully consider the unintended consequences of proposed legislation to fix problems.
Why Should You Listen to Me?
Since receiving a Ph.D. in experimental elementary particle physics, I spent 13 years at DOE’s Lawrence Livermore National Laboratory working on nuclear weapons and related programs and then five years at Stanford University, with Bill Perry and then Mike May, addressing problems that span the intersection of science and technology and national security. I have had two tours at the Department of Defense—six years in OSD Policy in the Clinton Administration as the Deputy Assistant Secretary in charge of nuclear weapons and missile defense policy. I recently completed four years in the Obama Administration in OSD’s AT&L organization focusing on, among other things, joint DoD-DOE oversight of the nuclear weapons stockpile. In between the DoD tours was an eight-year stint at NNSA under three outstanding leaders—John Gordon, Linton Brooks and Tom D’Agostino—directing their Policy Planning Staff. As a result, I am well positioned, I believe, to comment on important matters before this Panel. What Does the Nation Need from NNSA?
NNSA must ensure a nuclear weapon’s R&D and manufacturing infrastructure, and sufficient highly qualified personnel with capabilities to advance three core missions:
— Sustain today’s stockpile and ensure that it remains safe, secure and effective;
— Certify the safety/reliability of the future stockpile; and
— Provide a quick, effective response to technical or geopolitical surprise.
In addition, the nation needs an enterprise that will leverage nuclear capabilities and infrastructure to support national security needs beyond U.S. nuclear weapons.
What’s the Good News?
The NNSA enterprise has done a good job developing the tools, the scientific facilities and the expertise to certify a safe and reliable stockpile—i.e., the “science” part of stockpile stewardship. It has also been effective in leveraging core capabilities in support of the national security missions of other DoD and interagency components, including support to nonproliferation, arms control and global threat reduction activities. The NNSA’s weapons labs, plants and the Nevada Test Site are crown jewels of U.S. science, technology and engineering.
What’s Broken?
NNSA has, in large part, been unable to plan, manage, oversee, and hold accountable a nuclear weapons enterprise responsible for delivering on time and cost: (1) warhead life extension programs (LEPs) to sustain the nation’s nuclear stockpile, and (2) large infrastructure projects that provide capabilities to respond to future stockpile needs. The problem can be divided into elements that are NNSA-specific, and those external to NNSA:
NNSA-Specific Problems
— Poor program management/oversight: The NNSA federal workforce lacks project management experience and systems integration skills; poor cost estimation for large programs; design changes at late stages (e.g., HEUMF, UPF) drive high costs; multiple lines of authority promote lack of accountability;
— Ineffective relationships between NNSA and the national labs: Lack of trust between federal officials and lab managers; micro-management of contracts; NNSA directs not just “what” but “how”; unclear lines and multiple levels of authority within NNSA (HQ, Service Center, Site Offices, labs, plants) often in conflict with each other; focus on contract compliance, not outcomes; labs/plants need funding stability, flexibility, and predictability to function effectively and efficiently—none of these is present;
— Risk-adverse safety/security culture: Results in increased costs; less productivity/innovation; creates “work-free safe zone” mentality; unwillingness to “push back” on Defense Nuclear Facilities Safety Board recommendations that do not buy down risk commensurate with their costs;
• Seeking to eliminate all risks—big and small—diverts attention from managing the most important ones. This can lead, somewhat perversely, to a less safe and secure work place.
— Structural: Unlike DoD, a small number of projects (e.g., major construction, LEPs) represent a large component of NNSA’s budget. Also unlike DoD, funds for construction and operations are commingled. Given fixed budgets, therefore, cost escalation on a major project puts enormous fiscal pressures on other activities. Oftentimes, construction projects are short-changed to pay for other activities having shortfalls. Inefficient budget profiles for construction both delay completion and increase costs;
— NNSA Culture: An organization’s culture is extremely important in defining how it views itself and its mission. For example, the military culture has evolved over decades, indeed centuries, and permeates the entire Department of Defense. It is characterized by:
• Core Principle: Defend the nation;
• Motivated by outcomes, objectives; “can do” attitude, habitually impatient;
• Defined by uniformity (and uniforms!);
• Organized by training;
• Characterized by discipline;
• Responsive to the chain of command;
• Accepts and manages risk every day;
• Conveys enormous responsibility to junior officers/enlisted and empowers them to achieve objectives; and
• Advances star performers and weeds out others. The organizational culture of nuclear weapons R&D and manufacturing, evolving from the Manhattan Project, was different and had several notable characteristics:
• Unique partnership between government, the University of California, and private sector to bring advanced S&T to bear on a singular threat to the nation’s survival;
• Excellence in science and technology above all else;
• Scientific not military culture;
• Enterprise responsive to DoD requirements, but independent of DoD;
• Less organizational discipline, OK for labs to disagree with federal officials;
• Intense competition of ideas/concepts between lab-led product teams;
• Competition generally not cost-constrained;
• Strong, highly competent federal organization in Albuquerque to drive product.
An Enormous Culture Change
This culture has undergone enormous change over the past two decades resulting from the end of the Cold War and the end of nuclear testing. Perhaps the most significant change, however, resulted when the mission evolved from a balanced program of S&T, engineering development and production of new nuclear warheads on roughly a five-year cycle, to much less frequent programs to extend the life of existing warheads. Because of the relative lack of production work over most of the past two decades, and from heightened sensitivity to safety resulting in part from the Challenger Accident, the NNSA has lost its way on a production “culture” and has become exceedingly risk averse.
The military culture is driven from the top, and is reflected at the very junior levels of the Services and DoD. Changing NNSA will require senior leaders that embody both a culture of achievement and a singular commitment to the substantial time and effort required to drive that culture down through all levels of the organization.
Problems External to NNSA
— DOE: NNSA autonomy from DOE never achieved; most DOE orders and directives create inefficiencies/costs and should be replaced with commercial best practices; NNSA vs. DOE roles and responsibilities are confused and overlapping; DOE safety/security oversight both excessive and ineffective—focus on compliance with DOE orders and directives diverts management attention from mitigating most important risks;
— Congress: Energy and Water Appropriations Subcommittees are not structurally well-suited for oversight of nuclear weapons enterprise or aligned with defense mission; too many “bins” of money add to inefficiency and impedes management flexibility; endless series of continuing resolutions create program instability; funding construction projects year-to-year (vs. MILCON commitment) also drives program instability; willingness of Congress to “punish” NNSA more readily than it does DoD for comparable offenses;
— DNFSB safety oversight: The DNFSB sees its job as simply assuring that NNSA is adhering to relevant DOE safety orders and directives. It is the NNSA program manager’s job to evaluate DFNSB recommendations in connection with program scope, cost and schedule and manage overall risk accordingly. Why, the DNFSB might argue, is it the Board’s fault if a risk adverse culture at DOE discourages “push back” when the Board’s recommendations do not reduce risk commensurate with cost/schedule impacts?
• While independent safety oversight is critical, the DNFSB model is broken. Problems arise when NNSA and DNFSB argue about the interpretation and application of DOE safety rules. The “sunshine laws” under which DNFSB operates create corrosive impacts when safety issues play out in the public arena and press before the dialogue or choices are mature. The end result is that the DNFSB, an advisory body, in effect exercises directive authority over safety with no accountability for impacts on cost and schedule. As a result, there are increasingly negative effects on R&D, experimentation, and innovation;
— DoD: Lack of coherence amongst Services, CoCOMs, Joint Staff and OSD in DoD’s requirements process contributes to schedule delays and increased costs. Perception that DoD is too willing to run risks in favoring “stockpile” funding over “science” and that it does not understand the role that science plays in ensuring safe and reliable warheads;
The various components of the problem are inextricably linked—a comprehensive solution must address all of them. Said another way: Changing NNSA’s reporting chain, if everything else stays the same, is insufficient to fix the problem. Indeed, it may not even be necessary if practical solutions can be identified, and implemented, to address other key shortfalls.
What Do We Do About the Problem?
We know what needs to be fixed. Multiple panels and commissions over the past two decades—among them commissions led by the PFIAB, the National Academy of Sciences, the Defense Science Board, Johnny Foster (multiple times), Larry Welch, Hank Chiles, the Stimson Center and most recently the work of the Bipartisan Congressional Commission on the U.S. Strategic Posture—have developed coherent, self-consistent recommendations to address the problem.
What the Congressional Advisory Panel must do is:
— Synthesize, integrate and prioritize those recommendations;
— Understand root causes why they have not yet been effectively implemented; and
— Focus efforts on means for practical and effective implementation.
General Welch once argued that a “good” DSB study is 10 percent ideas and 90 percent implementation. Successful implementation requires: (1) creation of champions within the organization who are empowered to, and held accountable for, effecting real change, and (2) means to monitor progress on implementation on at least an annual basis. This second item suggests a continuing role for the Advisory Panel, or a follow-on effort, once it accomplishes its near term mission.
A Scalpel, Not a Cleaver, Sometimes Necessary
The Panel must carefully consider the unintended consequences of any proposed legislation; some solutions will require a scalpel not a meat cleaver. An overarching principle should be “do no further harm.” Recall that the NNSA Act resulted from a 1999 PFIAB study—“Science at its Best, Security at its Worst”—that examined security shortfalls at DOE in light of the Wen Ho Lee spy incident. Although it had flaws, the legislation enacted by Congress establishing a semi-autonomous NNSA within DOE was a thoughtful and prudent approach. But ineffective implementation short-circuited any desirable outcome and, indeed, resulted in making some things worse.
Finally, if the Panel succeeds in leading to resolution of these matters, it will result in a nuclear weapons enterprise that is much more efficient and thus more able to deliver its products within existing budgets.
APPENDIX
Fixing Program Management in NNSA’s Major Acquisition Programs
NNSA has, in large part, been unable to plan, manage, oversee, and hold accountable a nuclear weapons enterprise responsible for delivering on time and cost: (1) warhead LEPs to sustain the nuclear stockpile, and (2) large infrastructure projects that provide capabilities to respond to future stockpile needs. Among the root causes for this situation: The NNSA federal workforce, while technically competent, does not possess the depth and breadth of program management, systems engineering, and cost estimation skills to execute large and complex projects. If the enterprise is to succeed, other problems—atomization of budgets, ineffective safety/security oversight, lack of integration among LEPs, irregular exercise of needed capabilities—must be addressed in parallel.
How Did We Get Here?
In addressing this problem it is useful to recall how we got to where we are today. For decades, evolving from the Manhattan Project, nuclear warhead acquisition has been seen as an inherently governmental function and, unlike most DoD acquisition programs, are not competed in the private sector. As a result, NNSA spends about $7 billion a year to sustain and operate eight government-owned and contractor-operated facilities—three national labs, four production plants, and a test facility in Nevada. This entire enterprise supports the U.S. nuclear weapons stockpile from cradle to grave. Unlike private sector companies doing business with DoD, facilities, personnel and infrastructure are maintained year after year to provide a set of capabilities that can be drawn from as needed in support of the nuclear stockpile.
During the Cold War, the U.S. nuclear weapons enterprise carried out an intensive program to field new types of nuclear warheads on roughly a five-year cycle. Vigorous competition between two lab product teams—one headed up by Lawrence Livermore and one by Los Alamos, each with (fire-walled) Sandia partners—resulted in the winning team being awarded the development program for the next warhead. Although cost was a factor, the competition was mainly on design innovations to achieve ever better system performance—more “bang” in smaller, lighter packages—as well as providing enhanced safety and security.
A highly-competent federal team—the DOE’s Albuquerque Operations Office or ALOO—drove product by ensuring that the lab team fully engaged the production plants and delivery system experts. To a large degree, ALOO was the systems integrator for warhead development and production. This process worked well during the Cold War when the enterprise was turning out new warheads on a scale that sometimes exceeded a thousand per year.
Major Changes Drive Transformation
Over the past two decades, major changes have transformed warhead acquisition—the end of the Cold War, the end of nuclear testing, a much more tightly constrained fiscal environment and, most importantly, the evolution of the mission from a continuous 5-year cycle of new warhead development to much less frequent programs to extend the life of existing warheads.
By the early 1990s, the U.S. had just completed its last modernization cycle and there was not much life extension work on the horizon. Most effort in the 1990’s and early 2000 timeframe was devoted to advancing the “science” of stockpile stewardship; the production complex was allowed to wither away and ALOO was disestablished. The argument went—money’s tight, we aren’t producing anything and the big scientific challenge is to develop the experimental and analytical tools to certify safety and reliability under a test ban.
Today, NNSA faces a series of LEPs involving every warhead in the stockpile with needed plutonium infrastructure modernization off track, with uranium infrastructure that is falling apart, and with an enterprise that has lost management and systems integration skills. The challenge is akin to training a mechanic to overhaul a car while racing 80 mph down the freeway.
Who “Owns” the B61-12 Bomb LEP?
Consider a real world example. I posed a question to several knowledgeable individuals: “Who is the program manager for the B-61 bomb LEP?” There were several responses:
— “Its Sandia.”
— “It’s Los Alamos.”
— “It’s the B61 Project Officers Group (POG).”
— “It’s the B61 Executive Steering Group (ESG).”
— “It’s the NNSA feds in Albuquerque.”
— “It’s the Nuclear Weapons Council Standing and Safety Committee.”
— “No one manages the program.”
To state the obvious, we have a problem. Notwithstanding these responses, there is a well defined set of responsibilities and accountability for managing the pieces of individual LEPs, to the extent it is allowed to function effectively, and a fairly well defined process guiding development and production. The nuclear design lab, in this case Los Alamos, generates the engineering design information for the nuclear explosive package. Sandia is responsible for managing the non-nuclear component design, development, and testing and for integrating the nuclear and non-nuclear components into the bomb. Sandia is also responsible for integrating the bomb with the DoD bomb tailkit assembly. All activities require close coordination with production activities at Sandia, Kansas City, Y-12 and Pantex, and with the Air Force.
The problem is that there is no driving entity, as ALOO was, to coordinate activities at (up to) eight sites on tight and interdependent schedules; to ensure the labs and plants engage productively; or to pull funds from one activity at one site that is on track to augment another activity at another site that isn’t. This is the problem that needs solving.
How Can We Improve Program Management?
In managing NNSA’s large acquisition programs, there are two generic functions. Program managers organize the set of activities involving RDT&E and systems integration leading to a manufactured product. Federal oversight defends and advances the program in the Washington community, engages Congress on required resources, and holds the program manager accountable for meeting cost and schedule milestones.
In the traditional approach, DOE/NNSA federal officials both oversee and manage acquisition programs for warhead life extension and infrastructure recapitalization. This acquisition model has been successfully demonstrated by, in addition to ALOO, the Navy Strategic Systems Program (SSP) and the Naval Reactors (NR) program. Both SSP and NR employ several hundred federal acquisition experts and engages companies in the private sector as support contractors but not as lead systems integrators. Both have been in business for decades and have been successful in achieving safe, secure and cost effective operations across their respective systems’ life cycles.
The Navy SSP’s intensive engagement with NNSA on all aspects of the W76 SLBM LEP, the program management expertise that it brought to bear, and the respect it has engendered over decades as a technically-competent customer of NNSA’s services, were major factors in the relatively successful execution of that program.
For NNSA, this approach would necessarily entail a multi-year effort to transform the workforce by hiring systems engineers from the outside, by training from within, or some combination of the two. It would exploit lessons learned from the success of DOE’s Office of Science in bringing certain major construction projects to completion. A variation of this approach would have the SSP take on program management and system integration responsibilities for Navy warheads until comparable capabilities at NNSA are restored.
How Do We Fix Cost Estimation
If NNSA is to succeed in managing large acquisition programs, it must establish a capability for independent cost estimates and a disciplined cost reporting system. Independent cost estimation facilitates an informed dialogue between the government and the M&O contractors regarding what the government wants, what it thinks it will cost, what it can afford, and what adjustments in specific requirements (i.e., cost/performance trades) are needed to meet cost targets. That’s a very different approach from “here’s what I want, tell me what it will cost.”
As an example of the problem, initial costs estimates for the B61 LEP, and for the CMRR-NF and UPF construction projects, were low not by 20-30 percent but by factors of two and three. For the B61, when the scope of work was not fully understood, initial estimates came in at a couple of billion. A follow up estimate assumed that the B61 cost would be comparable to that of the W76 LEP—in the range of $4B. Lab experts, when subsequently engaged by NNSA, concluded that the B61 LEP would be more complex than the W76 and thus would cost significantly more. When Sandia issued the final cost report, the estimate rose to approximately $8B, where it is today.
In 2010, DoD committed to transfer about $8B over six years to support activities that were under funded in NNSA’s budget including specific LEPs, warhead surveillance, and CMRR and UPF construction. Within a few months, cost escalation by factors of 2-3 for these projects rendered DoD’s investment wholly insufficient. Poor cost estimation by NNSA had the consequence of decreasing trust between the two departments.
As many know, DoD’s Cost Assessment and Program Evaluation (CAPE) organization carries out independent cost estimates and reviews investment alternatives. It maintains a store of cost data, and estimating relationships, from previous major acquisition programs to inform “should cost” analyses for current programs. NNSA does not have adequate cost estimation capabilities, and must move out to acquire them. Until it does, NNSA could establish a long-term strategic partnership with CAPE to provide assistance in this area and to help train its people.
Necessary but not Sufficient Conditions for Success
Competent program management, systems integration, and cost estimation are necessary but not sufficient conditions for successful execution of LEPs and major construction projects. Other obstacles that prevent or impede work from being done must also be addressed in an overall solution. Specifically:
— Budget Atomization: Program managers cannot manage if they cannot readily move funds across the eight sites to address LEP program contingencies as they arise. Budgets are submitted to Congress with dollars allocated in small “buckets” to fund the workload anticipated at each site, and Congress funds it that way. To move money often requires a ponderous process to gain NNSA and DOE approval, and Congressional reprogramming authority. This does not facilitate a program’s agility and responsiveness to challenges;
— LEP Program Office: There are interactions between multiple ongoing LEPs—in the activities at the nuclear labs and plants, and in the integration and test activities with the relevant Services. An LEP program office, with knowledgeable direction like that previously provided by ALOO, is needed to manage and de-conflict these interactions; — Safety Oversight: Independent safety oversight is a critical component of all programs. That said, examples abound where multiple layers of oversight coupled with a risk adverse DOE and NNSA culture has led to substantial cost growth in major projects without increasing safety. Nor has the oversight process involving the DNFSB been effective. The NRC could probably do a better job and we should consider that option;
— Regular Exercise of LEP Capabilities: The government’s overall plan for LEPs—both near and long term—must ensure that the complex exercises the full set of required capabilities on a stable, year-to-year basis that actually builds competence and confidence. Capabilities not regularly exercised will be lost—we have experience with that. For example, I am concerned that further delay of the interoperable warhead LEP may impact LLNL’s efforts to maintain key capabilities. Absent such a plan, the quality of the program manager will not make a difference.
