Space Projects: Astrophysics Facility Program Contains Cost and Technical
Risks (Letter Report, 01/28/94, GAO/NSIAD-94-80).
The Advanced X-ray Astrophysics Facility (AXAF) is to be the third in
the National Aeronautics and Space Administration's (NASA) series of
four "great observatories," which are expected to provide new data and
insights into the age, evolution, and composition of the universe. By
redesigning and dividing the AXAF into two satellites, NASA has been
able to make the program more affordable. The restructuring cut program
costs by 54 percent without compromising the AXAF's scientific mission.
Affordability for the AXAF-I--the satellite devoted to high-resolution
x-ray imaging--may still be a concern, however, because funding reserves
for it may fall short. Also, NASA may not have fully accounted for the
increased technical risks inherent in the new design. As now designed,
AXAF-I cannot be repaired in orbit, which means that hardware failures
could scuttle a mission. Further minimizing the risk of hardware
failures demands the highest quality parts and rigorous testing, which
would boost costs significantly. NASA expects the restructure program
to achieve most of the original science objectives set for AXAF. The
objectives should be achieved because, owing to a higher orbit for
AXAF-1 and greater individual use of instruments, the satellites would
operate more efficiently than the originally planned single satellite.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: NSIAD-94-80
TITLE: Space Projects: Astrophysics Facility Program Contains Cost
and Technical Risks
DATE: 01/28/94
SUBJECT: Aerospace research
Testing
Cost control
Satellites
Space exploration
Product performance evaluation
Research and development costs
Aerospace engineering
Mission budgeting
Scientific research
IDENTIFIER: NASA Advanced X-Ray Astrophysics Facility Program
NASA Hubble Space Telescope Program
NASA Compton Gamma Ray Observatory
NASA Space Infrared Telescope Facility
NASA AXAF Charge Coupled Device Imaging Spectrometer
NASA High Resolution Camera
NASA High Energy Transmission Grating Spectrometer
NASA Low Energy Transmission Grating Spectrometer
Astro-E Mission (Japan)
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
January 1994
SPACE PROJECTS - ASTROPHYSICS
FACILITY PROGRAM CONTAINS COST AND
TECHNICAL RISKS
GAO/NSIAD-94-80
Space Projects
Abbreviations
=============================================================== ABBREV
AXAF - Advanced X-ray Astrophysics Facility
NASA - National Aeronautics and Space Administration
Letter
=============================================================== LETTER
B-255776
January 28, 1994
The Honorable Barbara A. Mikulski
Chair, Subcommittee on VA, HUD, and
Independent Agencies
Committee on Appropriations
United States Senate
The Honorable John D. Rockefeller, IV
Chairman, Subcommittee on Science, Technology,
and Space
Committee on Commerce, Science, and Transportation
United States Senate
The Honorable Louis Stokes
Chairman, Subcommittee on VA, HUD, and
Independent Agencies
Committee on Appropriations
House of Representatives
The Honorable Ralph M. Hall
Chairman, Subcommittee on Space
Committee on Science, Space, and Technology
House of Representatives
The National Aeronautics and Space Administration (NASA) is
developing the Advanced X-ray Astrophysics Facility (AXAF) to be one
of the mainstays of this nation's space science program during the
next decade. It is to be used to investigate important questions
such as the age and origin of the universe.
We reviewed the status of the 1992 redesigned AXAF program to (1)
assess the reasonableness of NASA's estimate of program costs and (2)
determine the extent to which the redesign will provide scientific
returns comparable to the original program. We recognize that a
recent congressional direction may affect a portion of this program,
but the cost and technical impacts of that decision are uncertain at
this time. However, we believe the results of our work in other
segments of the program are still relevant.
This report was prepared pursuant to our statutory authority, not at
the committees' request. It is intended to provide a better
understanding of the implications of NASA's decision to restructure
the program.
BACKGROUND
------------------------------------------------------------ Letter :1
AXAF is to be the third in NASA's series of four "great
observatories." It is to complement the Hubble Space Telescope, which
views objects in the visible and ultraviolet bandwidth of the
electromagnetic spectrum,\1 the Compton Gamma Ray Observatory, and
the yet-to-be-developed Space Infrared Telescope Facility. These
observatories are intended to provide new data and insights for
studies of the age, evolution, and composition of the universe and
its objects. The Congress authorized NASA to begin developing AXAF
in fiscal year 1989.
Our 1992 report\2
on the program stated that (1) the program's estimated cost had
increased by about 23 percent since it was first approved, (2) the
funding reserves had been used to offset additional increases, and
(3) the possibility existed that costs would increase further since
NASA might decide to launch AXAF on an expendable launch vehicle
rather than on the shuttle. We also reported that, while test
results on the AXAF's outer mirrors were encouraging, a number of
significant challenges remained for NASA to successfully launch and
operate the observatory.
In 1992, NASA concluded that AXAF, as then designed, was not
affordable. Budget cutbacks and the prospect of continued cost
growth caused NASA to restructure the program. To reduce cost, NASA
divided the program into two separate satellites or missions. One
satellite, AXAF-I (imaging), will emphasize high-resolution x-ray
imaging. Its instruments will include the AXAF Charge Coupled Device
Imaging Spectrometer, the High Resolution Camera, the High Energy
Transmission Grating\3 Spectrometer, and the Low Energy Transmission
Grating Spectrometer. AXAF-I is currently expected to be launched on
the shuttle in September 1998. The other satellite, AXAF-S, was to
concentrate on high-energy, high-resolution spectroscopy using the
X-ray Spectrometer. NASA planned to develop AXAF-S in-house at
Marshall Space Flight Center by using civil service personnel and
limited contractor support. However, the Conference Report
accompanying NASA's Fiscal Year 1994 Appropriations Act indicates a
$19-million reduction in the spacecraft portion of AXAF-S development
funding and directs NASA to include the spectrometer on the future
Japanese Astro-E mission. At the completion of our current review,
NASA did not have a plan to implement this change.
--------------------
\1 The electromagnetic spectrum is the entire array of energy
wavelengths as a continuum, from gamma rays to radio waves. The four
observatories cover the spectrum from gamma rays to infrared.
\2 Space Projects: Status and Remaining Challenges of the Advanced
X-ray Astrophysics Facility (GAO/NSIAD-92-77, Feb. 28, 1992).
\3 Transmission gratings are devices that are used to bend x-rays in
a predictable way, thereby enabling scientists to determine the
wavelength of the energy.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
By redesigning and dividing AXAF into two satellites, NASA was able
to make the program more affordable. The restructuring reduced
estimated program development and operating costs by 54 percent
without sacrificing much of the program's science content. Revising
the AXAF-S program in accordance with the recent congressional action
will further reduce program costs by an undetermined amount.
Affordability for the AXAF-I portion of the program, however, may
still be a concern because funding reserves for this portion may be
inadequate. Also, NASA may not have fully accounted for the
increased technical risks inherent in the new design. As currently
designed, AXAF-I cannot be repaired in orbit, which means that
hardware failures could seriously degrade or destroy the mission.
Further reducing the risk of hardware failures would require the
highest quality parts and rigorous testing, which would increase
costs significantly.
NASA expects the restructured program to achieve most of the original
science objectives established for AXAF. The objectives should be
achieved because, owing to a higher orbit for AXAF-I and greater
individual use of instruments, the satellites would operate more
efficiently than the originally planned single satellite.
PROGRAM COST IS UNCERTAIN
------------------------------------------------------------ Letter :3
In February 1992, before the program was restructured, NASA estimated
that AXAF would cost about $5.6 billion to develop and operate. In
September 1993, NASA estimated the total cost to be about $2.6
billion--a 54-percent reduction. (See table 1 for a comparison of
the estimates before and after NASA restructured the program.)
Officials told us that the fiscal year 1994 appropriations conferees
directed NASA to terminate the spacecraft part of AXAF-S, and that
action will further reduce the program's cost, but the officials have
not yet determined the amount of the reduction. Consequently, the
amounts shown in the table do not include changes that may result
from the fiscal year 1994 congressional action. For example, one
change would eliminate the planned Delta II launch for AXAF-S (about
$58 million).
Table 1
February 1992 and September 1993 Cost
Estimates
(Dollars in millions)
Feb. 1992 Sept. 1993
Cost element estimate\a estimate\a Difference
------------------ ------------ ------------ ------------
Advanced $54 $54 0
technology
development
Development 2,000 1,773 $(227)
Mission operations 3,325 529 (2,796)
and
data analysis
Launch support 145 226 81
Tracking and data 55 2 (53)
support
Construction of 18 18 0
facilities
============================================================
Total $5,597 $2,602 $(2,995)
------------------------------------------------------------
\a Neither estimate includes costs for civil service personnel.
Most of the $2,995 million decrease (about $2.8 billion) is
attributed to a change in operating concepts. Under the original
design, NASA planned to operate AXAF for 15 years and to maintain and
service it in orbit using the shuttle. Under the new design, the
life expectancy of the satellites was reduced to about 5 years and
NASA does not plan to service the satellites. Design changes such as
reducing the number of mirrors and science instruments account for
most of the remaining reduction.
FUNDING RESERVES MAY BE
INADEQUATE
------------------------------------------------------------ Letter :4
While the 1992 redesign decision and recent congressional action will
significantly reduce estimated costs, affordability may still be a
concern in AXAF development. Our analysis of funding focuses on
AXAF-I because, given the recent congressional action, NASA has no
current estimate of the AXAF-S's cost.
NASA's cost estimate to develop AXAF-I includes reserves to cover
contingencies and programmatic changes that may occur during the
development program. The level of reserves that NASA includes in
program cost estimates varies from program to program and depends on
the amount of uncertainty and risk of the particular program.
However, in past reviews, NASA cost estimating officials have stated
that in complex programs, prior to the critical design review,
funding reserves should equal about 30 percent of development
costs.\4 As of October 1993, NASA's estimate for AXAF-I included
$197.3 million in reserves--about 21 percent of the remaining
estimated development cost.
Some of the AXAF-I reserves have already been earmarked to fund
certain activities. For example, NASA forecasts a high probability
(60 to 90 percent) that contractor overhead and general and
administrative rates will increase and estimates that the increase
could consume over 10 percent of the total available reserves.
In addition, most of the reserves are allocated toward the end of the
redesigned observatory's 7-year development effort. About 70 percent
of AXAF-I's remaining reserves are earmarked for use in the last 3
years of the development effort. Consequently, the program must be
managed using minimal amounts of reserve funding in the early years
of development. During NASA's fiscal year 1993 oversight hearings,
the NASA Administrator testified that the greatest demand for reserve
funds normally occurs early in a program's development. If adequate
reserves are not available when needed, schedule slips and higher
costs will likely result.
--------------------
\4 See Space Transportation: NASA Has No Firm Need for Increasingly
Costly Orbital Maneuvering Vehicle (GAO/NSIAD-90-192, July 31, 1990);
Space Projects: Status and Remaining Challenges of the Advanced
X-ray Astrophysics Facility (GAO/NSIAD-92-77, Feb. 28, 1992); and
NASA Program Costs: Space Missions Require Substantially More
Funding Than Initially Estimated (GAO/NSIAD-93-97, Dec. 31, 1992).
TECHNICAL RISK REMAINS A
QUESTION
------------------------------------------------------------ Letter :5
Under NASA's redesigned concept, neither of the satellites would be
serviceable in orbit, meaning technical problems that cannot be
corrected through ground communications could degrade or destroy the
mission. NASA may not have fully accounted for this risk.\5
The risks associated with the inability to service AXAF-I are clearly
illustrated by NASA's experience with the first two observatory
missions. The Hubble Space Telescope required at least one servicing
mission to correct problems discovered after it was deployed, and the
Compton Gamma Ray Observatory experienced problems both during and
after deployment. It required an unscheduled spacewalk by two
shuttle astronauts to properly deploy an antenna, and it has since
experienced failures in two data recorders and degradation in onboard
batteries.
NASA categorizes its payloads in terms of the risk it is willing to
assume. All payloads are assigned to one of four categories, ranging
from A to D, based on assessments of the consequences of mission
failure. Factors such as cost, complexity, mission priority, and
degree of acceptable risk are weighed in deciding upon a design
approach. Payloads assigned the highest classification are
considered to have the highest priority and cost.
Despite its high priority and cost, NASA classified the original AXAF
as a class B payload. In making this determination, NASA noted that
AXAF would operate in a low-earth orbit where it could be serviced by
the shuttle. The classification permits the use of less costly
components and a lower level of testing than is required for a class
A payload. NASA has not changed the payload classification for the
imaging mission even though it will no longer be accessible to the
shuttle for in-orbit repair.
Upgrading AXAF-I's classification could result in significant
programmatic and cost impacts. For example, only the highest quality
parts would be used in the system's design, a complete set of flight
spare replacement units could be required, and more stringent
qualification testing would have to be performed. The Chief of the
Payloads Assurance Office estimated that these changes could increase
AXAF-I's development cost by one third to one half. The changes
could also delay completion of the development program.
--------------------
\5 As a consequence of the recent congressional action, our analysis
relates only to AXAF-I risk.
REDESIGNED OBSERVATORIES ARE TO
PROVIDE SIGNIFICANT SCIENCE
------------------------------------------------------------ Letter :6
In September 1992, the National Research Council was asked to
evaluate the scientific content and return of the restructured
program. In an April 28, 1993, report, the task force assigned to
the review reported that the restructured program ". . .
maintained essentially all of the outstanding scientific capabilities
of the baseline mission." The task force concluded that the program
was capable of being the ". . . centerpiece of international
efforts in x-ray astronomy for the foreseeable future." AXAF-I is
expected to achieve a greater angular resolution than any prior x-ray
mission. This improvement should make it possible to resolve and
distinguish between energy sources, such as neutron stars and
quasars, at greater distances and in closer proximity to each other
than ever before.
The council has remained steadfast in its support for the program.
In 1982 and 1991, the council's astronomy and astrophysics committees
recommended new ground- and space-based programs for development.
The committees considered large and expensive development efforts, as
well as smaller, more focused experiments. Both committees strongly
recommended AXAF.
NASA plans to place AXAF-I into a high-earth elliptical orbit at a
minimum altitude ranging from 10,000 to 100,000 kilometers. The
original AXAF design provided for a low-earth orbit of about 600
kilometers. In a low-earth orbit, the earth would periodically
obstruct the observatory's view of a target, which, in turn, would
require NASA to repoint the antennas and reacquire the target.
According to NASA, the higher orbit would eliminate this problem for
most targets. (See fig. 1 for depiction of orbital viewing
efficiencies.)
Figure 1: AXAF Orbital Viewing
Efficiency
(See figure in printed
edition.)
In addition, the new design would permit more efficient use of the
science instruments. This approach separated the imaging mission
from the primary spectroscopy mission, and eliminated one
spectrometer. This would have meant that all of the remaining
instruments could be used for longer periods of time, and
simultaneous imaging and spectroscopic observations could be made.
As a result, more observations per unit of viewing time would have
been possible.
NASA officials told us the Fiscal Year 1994 Appropriations Conference
Committee's direction to fly the X-ray Spectrometer on Astro-E (the
Japanese mission) could affect the instrument's performance as well
as the potential for simultaneous viewing of cosmic events.
According to AXAF-S program management officials, instrument weight,
life expectancy, spacecraft orbit, launch date, proprietary use of
science data, and number of instruments to be flown on the satellite
could affect the scientific return from the X-ray Spectrometer.
According to the officials, if Astro-E is used to launch the
spectrometer, each of these matters will be subject to negotiation
with Japan. Therefore, NASA has not determined the impact of this
redirection on the program's science capabilities.
RECOMMENDATION
------------------------------------------------------------ Letter :7
We recommend that the NASA Administrator review the AXAF-I
development program to determine whether (1) funding reserves are
realistic in light of the program's uncertainty and risk and (2) the
AXAF-I payload classification is prudent given the complexity, costs,
and lack of repair capability inherent in the new design.
AGENCY COMMENTS
------------------------------------------------------------ Letter :8
NASA found our report to be accurate and clear and agreed with our
recommendations. NASA's detailed comments are reprinted in appendix
I.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :9
We analyzed budget submission documentation, cost estimates, briefing
reports, legislative language, project planning documents, and
technical studies at Marshall Space Flight Center and NASA
headquarters. We then discussed cost and performance issues with
program management, engineering, quality assurance, and procurement
officials.
To assess the reasonableness of the cost estimates, we compared the
detailed cost breakouts of the original and restructured programs,
identified significant differences, verified the accuracy of NASA's
calculations when possible, and discussed NASA's position with
program and project management officials. We also assessed the
adequacy of funding reserves by (1) comparing the reserve totals as
allocated by year to a generally accepted standard for similar
development programs, (2) analyzing known programmatic requirements,
(3) identifying potential additional costs unaccounted for in the
overall estimate, and (4) obtaining NASA's position relating to
funding adequacy.
To determine the scientific capabilities of the two redesigned
satellites, we compared the major performance characteristics of the
original design with those projected for the new design. We then
identified the types and amount of scientific investigations possible
under the new design. We also analyzed internal studies and policies
regarding technical and programmatic risks giving particular
attention to the lack of serviceability in the new design.
We performed our review from December 1992 through November 1993 in
accordance with generally accepted government auditing standards.
We are also sending copies of this report to the Administrator, NASA;
other appropriate congressional committees; the Director, Office of
Management and Budget; and other interested parties.
Please contact me at (202) 512-8412 if you or your staff have any
questions concerning this report. The major contributors to this
report are listed in appendix II.
Donna M. Heivilin, Director
Defense Management and NASA Issues
(See figure in printed edition.)Appendix I
COMMENTS FROM THE NATIONAL
AERONAUTICS AND SPACE
ADMINISTRATION
============================================================== Letter
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix II
NATIONAL SECURITY AND
INTERNATIONAL AFFAIRS DIVISION,
WASHINGTON, D.C.
David R. Warren, Associate Director
ATLANTA REGIONAL OFFICE
Lee Edwards, Acting Assistant Director
John Gilchrist, Evaluator in Charge
Wendy Smythe, Site Senior