NASA Procurement: Challenges Remain in Implementing Improvement Reforms
(Chapter Report, 08/18/94, GAO/NSIAD-94-179).
In recent years, the National Aeronautics and Space Administration
(NASA) has experienced a variety of development and performance problems
with expensive space hardware. These problems have been a primary
impetus behind the mounting Congressional pressures for NASA to improve
its procurement practices. NASA has partly or fully implemented all but
one of the eight key procurement management improvement initiatives it
was working on in June 1993. Some were developed slowly and implemented
quickly or achieved goals ahead of schedule. GAO questions whether some
of the initiatives will be fully effective as a result of planning and
implementation problems. Specifically, (1) the proposed contractors'
liability policy will be hard to administer, may produce higher contract
prices, and could harm subcontractors who are unwilling or unable to
risk increased liability; (2) the potential effect of the proposed
contractor liability policy on small and disadvantaged businesses has
not been evaluated; and (3) the policy guidance for the training of
contracting officers' technical representatives is incomplete.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: NSIAD-94-179
TITLE: NASA Procurement: Challenges Remain in Implementing
Improvement Reforms
DATE: 08/18/94
SUBJECT: Federal procurement
Contract administration
Aerospace contracts
Contract modifications
Contract costs
Contractor performance
Work measurement standards
Small business contractors
Human resources training
Contractor responsibility
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to the Chairman, Committee on Government Operations, House of
Representatives
August 1994
NASA PROCUREMENT - CHALLENGES
REMAIN IN IMPLEMENTING IMPROVEMENT
REFORMS
GAO/NSIAD-94-179
NASA Procurement
Abbreviations
=============================================================== ABBREV
COTR - contracting officer's technical representative
CPAF - cost-plus-award-fee
FAR - Federal Acquisition Regulation
GAO - General Accounting Office
NASA - National Aeronautics and Space Administration
SBIR - Small Business Innovative Research
SDB - small and disadvantaged business
Letter
=============================================================== LETTER
B-257226
August 18, 1994
The Honorable John Conyers, Jr.
Chairman, Committee on Government
Operations
House of Representatives
Dear Mr. Chairman:
This report responds to your request that we review the status of the
National Aeronautics and Space Administration's (NASA) procurement
management improvement initiatives and assess their planning and
implementation.
Our recommendations to the NASA Administrator focus on improving the
agency's procurement initiatives, including providing additional
guidance and evaluating the potential for unintended consequences.
As requested, we plan no further distribution of this report until 30
days after its issue date. At that time, we will send copies to the
Administrator, NASA; the Director, Office of Management and Budget;
and other interested parties.
If you or your staff have any questions about this report, please
contact me at (202) 512-8412. Major contributors to this report are
listed in appendix V.
Sincerely yours,
Donna M. Heivilin
Director, Defense Management
and NASA Issues
EXECUTIVE SUMMARY
============================================================ Chapter 0
PURPOSE
---------------------------------------------------------- Chapter 0:1
In recent years, the National Aeronautics and Space Administration
(NASA) has experienced a variety of development and performance
problems with expensive space hardware. Such problems have
contributed to an increasing awareness of NASA's difficulties in
managing its contracts and in buying hardware that works properly,
and they have been a primary impetus behind the mounting pressure
from the Congress for NASA to improve its procurement practices. The
Chairman of the House Committee on Government Operations asked GAO to
(1) determine the status of NASA's initiatives to improve and
streamline its procurement activities and (2) assess NASA's planning
and implementation of the initiatives.
BACKGROUND
---------------------------------------------------------- Chapter 0:2
NASA spends about 90 percent of its funds--$13.2 billion in fiscal
year 1993--on contracts. It is the second-largest civilian
contracting agency in the federal government.
Since 1987, NASA has acknowledged that its procurement and contract
management are vulnerable to waste and mismanagement, based on its
own internal management reviews and audits by the NASA Inspector
General. Well-publicized performance problems and numerous
assessments by NASA management, the NASA Inspector General, and GAO
have demonstrated the need for NASA to improve its procurement
practices.
RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3
NASA has partly or fully implemented all but one of the eight key
procurement management improvement initiatives it was working on in
June 1993. Some were developed slowly and raised concerns inside and
outside the agency. Others were developed and implemented quickly or
achieved goals ahead of schedule.
GAO questions whether some of the initiatives will be fully effective
due to planning and implementation problems. Specifically, (1) the
proposed contractor liability policy will be difficult to administer,
may result in higher contract prices, and could adversely affect
subcontractors who are unwilling or unable to risk increased
liability; (2) the potential effect of the proposed contractor
liability policy on small and disadvantaged businesses has not been
evaluated; and (3) the policy guidance for the training of
contracting officers' technical representatives is incomplete.
PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4
STATUS OF INITIATIVES FOR
IMPROVING PROCUREMENT
MANAGEMENT
-------------------------------------------------------- Chapter 0:4.1
Over the years, there have been a variety of serious weaknesses in
NASA's award fee contracting. For more than 2 years, NASA planned
and developed a new award fee policy. The new policy was issued in
October 1993. It increases emphasis on selection of contract type,
cost control, end product, and overall contractor performance. It
also eliminates the use of rollovers (allowing fee that was not
earned in a prior period to be available in a subsequent period);
restricts base fee (fee paid regardless of performance); provides for
a uniform and simplified scoring system throughout the agency; and
requires that some award fee contracts include incentives to reward
or penalize contractors based on the performance of delivered
hardware.
In November 1992, the Congress directed NASA to review options and
develop a policy for sharing risk with research and development
contractors. This direction stemmed from concerns about the agency's
payments to contractors for the costs of repairing or replacing
articles when the defects were caused by contractors' negligence. In
June 1993, NASA reported the results of its assessment, and, in
February 1994, it published a proposed policy for public comment.
The proposed policy, if implemented as published in draft, would
increase contractors' liability for correcting defects in material or
workmanship, or other failures to conform to requirements of
high-dollar value research and development, cost-type contracts. The
new policy would increase a contractor's liability if NASA determined
that the contractor (1) had not applied its best efforts toward the
research and development objectives of the contract or (2) had not
followed proper procedures conforming to generally accepted practices
in performing routine operations, such as moving equipment or
conducting standardized tests. A NASA procurement official estimated
that a policy increasing contractor liability probably will become
effective in summer 1994.
In January 1993, 6 months after the NASA Administrator directed that
it be developed, the agency implemented a system of contractor
metrics, a set of standardized data for reporting specific measures
of contractor performance. The system provides agency and corporate
senior managers with an overview of contractor performance on major
NASA contracts. NASA designed this agencywide system to meet two
recognized needs: (1) establish an ongoing dialogue between
upper-level NASA and corporate management and (2) simplify and
standardize the information given to high-level NASA officials on
active contracts.
A fourth initiative focused on reducing the number and value of the
agency's unpriced contract changes. When contract changes are
unpriced, the government's cost-risk increases. The longer changes
remain unpriced, the greater the risk. Past efforts to manage
unpriced contract changes more effectively have either failed to meet
goals or have not resulted in lasting reductions in the value of such
changes. In June 1993, the agency designated its ongoing efforts as
a formal procurement initiative. In May 1994, NASA implemented an
agencywide policy on pricing such changes that requires top
management approval of high-value changes--a control element absent
from prior attempts to significantly and permanently lower the value
of outstanding unpriced changes.
The four remaining initiatives focused on (1) establishing minimum
training requirements for contracting officers' technical
representatives, (2) achieving small and disadvantaged business
goals, (3) communicating with industry representatives about
procurement issues and concerns, and (4) testing methods to reduce
the time and effort applied to the majority of NASA's contracting
actions without adversely affecting their quality.
PROBLEMS IN PLANNING AND
IMPLEMENTING PROCUREMENT
INITIATIVES
-------------------------------------------------------- Chapter 0:4.2
NASA's proposed contractor liability policy will be difficult to
administer. The policy increases contractors' liability for the cost
of replacement or correction of defects. A NASA official said he
assumes that NASA would apply this policy even in cases where the
agency does not intend to replace or repair a faulty item. This
option is not addressed, however, either in the existing contract
clause or in the proposed policy. Consequently, NASA's decision to
assess liability in cases when the agency does not intend to replace
or repair an item could be disputed by the contractor.
NASA's determinations of contractor liability for the cost of
replacement or correction of defects requires applying general
concepts that are yet to be precisely defined, such as "best
efforts," "routine operations," and "generally accepted industrial or
engineering practices." NASA's decision to assess liability can be
disputed by the contractor and could result in protracted
negotiations and potentially costly litigation. Also, the proposed
policy could (1) lead to higher contract prices and (2) have
unintended adverse consequences on subcontractors with limited
capital.
NASA has not evaluated the impact of its proposed contractor
liability policy on its goals for contracting with small and
disadvantaged businesses. The policy, if implemented, could affect
the ability or willingness of subcontractors that are small and
disadvantaged businesses or universities to work on NASA contracts.
Specifically, the policy could result in a potentially significant
risk of liability, which could, in some cases, undermine the economic
viability of such subcontractors. Also, liabilities for hardware
that fails to perform properly or for other failures to comply with
contractual requirements could affect large contractors' willingness
to subcontract with such businesses, which could inadvertently affect
NASA's ability to achieve future small and disadvantaged business
contracting goals, especially in high-technology areas.
NASA's guidance on new requirements for the training of contracting
officers' technical representatives is incomplete. It does not
discuss available options for fulfilling mandatory training
requirements and for establishing and applying criteria to determine
when no additional training is needed.
RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:5
GAO recommends that the NASA Administrator
evaluate the potential impact of the proposed contractor liability
policy on small and disadvantaged businesses and consider
options to mitigate any adverse impacts identified;
define "high-technology" work and develop and implement a
methodology for consistently measuring progress toward NASA's
goal of increasing small and disadvantaged businesses'
participation in such work, either as prime contractors or
subcontractors;
direct that special reviews be conducted at centers to ensure that
training courses for contracting officers' technical
representatives sufficiently address new minimum training
requirements; and
provide guidance to the centers on establishing and applying
criteria to determine when no additional training is needed to
meet new minimum training requirements for contracting officers'
technical representatives.
AGENCY COMMENTS
---------------------------------------------------------- Chapter 0:6
NASA agreed with GAO's recommendations and generally agreed with the
report's contents. NASA provided comments on GAO's analyses of some
of the initiatives, and these comments were considered in preparing
this report. NASA's comments are reprinted in appendix II.
INTRODUCTION
============================================================ Chapter 1
The National Aeronautics and Space Administration's (NASA)
procurement budget is one of the largest of all civilian agencies in
the federal government. Each year, the agency spends about 90
percent of its funds on contracts. Over the last decade, the
reported value of NASA's annual procurements in 1993 dollars has
increased by almost 30 percent--from $10.4 billion to $13.2 billion.
NASA employs more than 1,000 people in procurement activities
throughout the agency.
CONCERN ABOUT NASA'S
PROCUREMENT MANAGEMENT
---------------------------------------------------------- Chapter 1:1
Based on its own internal management reviews and audits by the NASA
Inspector General, the agency initially acknowledged in 1987 that its
procurement and contract management were vulnerable to waste and
mismanagement. Since then, there have been a number of
well-publicized problems with expensive space hardware, including the
Hubble Space Telescope and an advanced series of weather satellites
(GOES-Next). These problems have contributed to an increasing
awareness outside of NASA of the agency's difficulties in managing
its contracts and in buying hardware that works properly. This
awareness has been a primary factor in mounting pressure from the
Congress that NASA improve its procurement practices. Among other
areas, the effectiveness of NASA's cost-plus-award-fee (CPAF)
contracts in motivating contractors to control costs and to deliver
quality products has been questioned. The Congress has also raised
concerns about NASA contracts that require contractors to be paid for
repairing defects caused by their own negligence.
We have noted that NASA's procurement management problems are linked
to the agency's overall program management process.\1 NASA recognizes
this interdependence and, in recent years, has conducted many
internal reviews to identify problems and develop methods for
improving agency operations and programs. In early 1993, NASA
announced a set of initiatives to improve both program and
procurement management.
Well before that, however, NASA had been working to improve its
procurement activities. Over the past several years, NASA has had a
total of 19 procurement improvement initiatives. Some of these have
been completed; others have been added or dropped as management
priorities changed based on updated information. Earlier initiatives
included establishing a new Contract Management Division within the
Office of Procurement at NASA headquarters, improving the training of
procurement personnel, and streamlining grant processing. As of June
1993, NASA was working on eight procurement management improvement
initiatives. Since then, two more have been added. Although these
varied efforts are intended to improve and streamline NASA's
procurement process, agency officials acknowledged that associated
potential savings are not easily quantifiable or likely to be
realized or significant in the near term.
--------------------
\1 NASA Contract Management (GAO/HR-93-11, Dec. 1992).
OBJECTIVES, SCOPE, AND
METHODOLOGY
---------------------------------------------------------- Chapter 1:2
The Chairman, House Committee on Government Operations, requested
that we review NASA's efforts to improve and streamline its
procurement activities. Specifically, we were asked to (1) determine
the status of NASA's procurement management improvement initiatives
and (2) assess their planning and implementation.
We reviewed the eight procurement initiatives NASA identified in June
1993. In response to another congressional request, we were already
evaluating one of the initiatives.\2 Consequently, we focused most of
our efforts on the seven remaining. To satisfy our objectives, we
interviewed cognizant officials in the Offices of Procurement and
Small and Disadvantaged Business Utilization at NASA headquarters,
and we reviewed documents related to the initiatives, including
current, proposed, and final policies; public comments NASA received
on those initiatives published in the Federal Register; and center
responses to some proposed policy changes. We also reviewed sections
of the Federal Acquisition Regulation (FAR) and the NASA FAR
Supplement, NASA handbooks and other guidance, NASA's annual
procurement reports, reports required by the Federal Managers'
Financial Integrity Act, NASA Inspector General and internal
management review reports, and congressional hearings.
During the course of our review, NASA announced two new initiatives.
We did not include these in the scope of our review, but we did
obtain preliminary information on them. A description of these
initiatives is included in appendix I. In commenting on our draft
report, NASA provided information on a new initiative on past
contractor performance. This information is included in appendix II,
where NASA's comments are reprinted in their entirety.
We performed our work at NASA headquarters in Washington, D.C., from
August 1993 through June 1994 in accordance with generally accepted
government auditing standards.
--------------------
\2 We subsequently reported on this initiative in NASA Procurement:
Planning for Pilot Test of New Procurement Procedures Is Adequate
(GAO/NSIAD-94-67, Nov. 4, 1993).
NASA'S EFFORTS TO DEVELOP AND
IMPLEMENT PROCUREMENT INITIATIVES
============================================================ Chapter 2
As of June 1993, NASA was working on eight key procurement management
improvement initiatives; all but one have been partly or fully
implemented. Some of the improvement efforts developed slowly,
raised the concerns of industry and agency personnel, or failed to
meet goals. On the other hand, others were developed and implemented
quickly or achieved goals ahead of schedule. Table 2.1 provides a
summary of information and comments on the eight initiatives
discussed in chapters 2 and 3.
Table 2.1
Overview of NASA's Eight Procurement
Initiatives
Problems/ Date
Focus of concerns initiative Actions Actions Observations/
initiative addressed announced\a taken remaining comments
----------- ----------- ----------- ----------- ----------- ---------------
Improve Excessive 3/92 Policy Complete Complete
award fee fees implemented development restructuring
contracting awarded, in October of of award fee
little 1993. agencywide contracting.
emphasis on course. Increases
end Compliance emphasis on end
results, to be results,
product monitored product
performance by performance,
, and cost headquarter and cost
control. s' reviews control.
Inconsisten and
t field procurement
center management
practices. surveys.
Increase Contractors 6/93 Published Estimated Increases
contractor reimbursed proposed implementat contractor
liability for repairs policy in ion summer liability.
when February 1994. Difficult to
defects 1994. administer.
were caused Could lead to
by higher contract
contractors prices and
' adversely
negligence. affect small
and
disadvantaged
businesses
(SDB).
Report on Limited 4/93 Designed Continue New system
contractor high-level and regular providing
performance communicati implemented reporting increased
on. Time- agencywide cycles. communication
consuming reporting with industry
review of system in and simplified
contract January review within
status in 1993. NASA. Could
nonstandard report on SDB
format. high-
technology
work.
Reduce Billions of 6/93 Implemented Continued Part of
value of dollars in agencywide monitoring continuing
unpriced unpriced policy in of value of effort. Recent
contract change May 1994. unpriced effort resulted
changes orders. changes at in significant
Contractor centers. reduction in
has limited value of
incentive unpriced
to control contract
costs. changes.
Strengthen Problems 6/93 Issued All COTRs New requirement
training with COTRs' agencywide to be to ensure COTRs
for understandi guidance in trained by adequately
contracting ng of January April 1, trained. Does
officers' roles, 1994. 1995. not ensure
technical responsibil sufficiency of
representat ities, and COTR training
ives (COTR) scope of courses.
authority. Guidance is
Amount and incomplete.
quality of
COTR
training
inconsisten
t.
Achieve SDB Agency 3/92 Developed Increased Designed to
goals directed to plan in emphasis on meet mandated
award 8 1992 to high- contracting
percent of achieve 8- technology goal. Achieved
contract percent contracts. goal ahead of
dollars to goal. schedule.
SDBs. Developed
strategies for
achieving
future goals.
Communicate Issues 3/92 Established Continue Enables NASA to
with hindering Industry dialogue consider and
industry the Process with address
effectivene Action Team industry on industry
ss of in October procurement concerns on
NASA's 1991. issues. procurement
acquisition issues.
process.
Streamline Disproporti 3/92 Began pilot Expand Designed to
midrange onate test of new pilot test reduce
procurement amount of procedures to all NASA procurement
procedures time and at Marshall centers in time and
effort for Space near effort.
procurement Flight future. Preliminary
s in Center in Must obtain test results
$25,000 to July 1993. waivers for show decrease
$500,000 exclusive in average lead
annual use of time for
range. electronic awarding
bulletin midrange
board. contracts.
--------------------------------------------------------------------------------
\a Indicates when NASA formally announced the initiative, not when
the problem was first identified or work actually began.
This chapter discusses the problems and issues addressed by the first
seven initiatives and the status of efforts to develop and implement
them. We discussed the planning for the last initiative in an
earlier report; this chapter provides updated information on that
effort. Chapter 3 discusses factors that may impede the initiatives'
effectiveness, including problems with administration, potential
adverse effects, and policy guidance.
CHANGING AWARD FEE CONTRACTING
---------------------------------------------------------- Chapter 2:1
Over the years, reviews of CPAF contracting practices at NASA have
identified a variety of serious weaknesses, including the award of
excessive fees and limited emphasis on end results, product
performance, and cost control. For example, in 1993, the NASA
Inspector General reported that contract cost overruns were not
adequately considered as part of the cost incentive evaluation
criteria in determining fees, and contractors were paid millions of
dollars in fees on contracts with hundreds of millions of dollars in
cost overruns. Other issues addressed in recent internal reviews of
CPAF contracting practices include (1) inconsistencies among NASA
field centers in how they evaluate contractor performance and (2) the
potential for requiring contractors to return fees previously
awarded. These issues were similar to those raised a decade ago in
earlier reviews of CPAF contracting at NASA. However, those earlier
reviews did not lead to effective corrective actions.
NASA began its award fee initiative in August 1991 but did not issue
a new award fee contracting policy until more than 2 years later.
One factor contributing to the slow development was the need to
address concerns expressed by industry and by the NASA centers'
program, technical, and procurement personnel. Due to industry's
concerns, NASA rescinded an earlier proposed change to the policy
only 5 days after it was published in the Federal Register. Industry
concerns centered mainly on the proposed use of negative performance
incentives and a prohibition on the use of base fees.\1 The use of
negative performance incentives did not change in the final policy;
the base fee prohibition was modified as discussed below. Many of
the NASA centers' concerns were similar to those of contractors, but
they were also concerned about the prohibition on rollovers (the
practice of making a fee that was not earned in a prior period
available to be earned in a subsequent period). That prohibition
remained in the final version of the policy.
The new award fee policy applies to all solicitations issued for CPAF
contracts after October 8, 1993. The agency developed a handbook on
the policy and is working on an agencywide training course. To
assess compliance with the new policy, NASA headquarters officials
and procurement management survey teams will review pre-award
activities for selected cost-plus-award-fee contracts. Procurement
survey teams will also assess center compliance with post-award
provisions. However, such surveys are conducted infrequently and
address a broad range of subjects with limited time and resources.
Consequently, procurement management surveys may be limited in their
ability to identify some compliance problems. The potential for
noncompliance is an area of special concern. For example, the NASA
Inspector General recently reported that a major procurement center,
which expended $1.2 billion on award fee contracts in fiscal year
1993, was not consistently adhering to award fee policies,
procedures, and guidelines. Our past reviews have also shown that a
primary cause of compliance problems at centers was ineffective
oversight by NASA headquarters.
The new award fee policy emphasizes the importance of selecting the
appropriate type of contract to reasonably allocate performance risk
between the contractor and the government. In particular, the policy
notes that CPAF contracts should be used only where there is
substantial technical, cost, or schedule uncertainty and the
evaluation of contractor performance must be subjective. Where
uncertainties are fewer and risk is lower, the new policy suggests
the use of other contract types, including cost-plus-incentive-fee,
fixed-price-incentive, or firm-fixed-price (where the contractor
assumes maximum risk).
--------------------
\1 Base fees on cost-type contracts were paid regardless of
contractor performance.
AWARD FEE POLICY
-------------------------------------------------------- Chapter 2:1.1
The award fee policy states that cost control will be emphasized in
all CPAF contracts. When weighted evaluation factors are used, the
assigned weight for cost control will be no less than 25 percent of
the total award fee, excluding any base fee. The contractor must
receive a rating of "very good" or "excellent" on technical factors
before being rewarded for cost savings. The policy also prescribes a
uniform and simplified award fee scoring system for the entire
agency; establishes 6 months as a standard evaluation period;
eliminates the use of rollovers; and emphasizes the timely payment of
provisional, interim award fees.
Under the previous award fee policy, base fee was allowed on all CPAF
contracts. NASA's new policy eliminates base fee on service
contracts, but allows its use on other contracts, such as those for
study, design, or hardware, provided contractor performance is at
least "satisfactory" for the entire contract.
Another key element of the new award fee policy is increased emphasis
on overall contractor performance and the end product, rather than on
interim progress. For example, on contracts for study, design, or
hardware, where the quality of contractor performance cannot be
measured until the end of the contract, the award fee earned is
linked to a final, comprehensive rating.
The policy also requires the use of both positive and negative
performance incentives on all CPAF contracts when the total estimated
cost and fee is greater than $25 million and the primary deliverable
is hardware. Performance incentives are based on measurements of
hardware performance against objective criteria. They are separate
and distinct from award fee, which is subjective and measures
contractor performance up to delivery and acceptance.\2
The new policy stipulates that at least one-third of the total
potential contract fees must be placed in the performance incentive
pool. The contractor earns no performance incentive when the
hardware performs at the minimum acceptable level established in the
contract, since this performance level is covered by award fee
payments. Performance incentives reward contractors when delivered
hardware performs above minimum contract requirements. Conversely,
the contractor is penalized if the product does not meet such
requirements. In the event of a total, immediate, and permanent
hardware failure, the contractor is to return an amount equal to the
entire earned award fee, including any base fee. In certain
circumstances, the payback amount would be based on the maximum
potential award fee the contractor could earn, not the amount
actually earned. Appendix III describes how a performance incentive
might be structured.
--------------------
\2 Unlike award fee decisions, both negative and positive performance
incentive decisions can be legally disputed.
PROJECTED EFFECT OF AWARD
FEE CHANGES
-------------------------------------------------------- Chapter 2:1.2
NASA procurement officials believe that the new award fee policy will
encourage contractors to deliver quality products and services at a
reasonable cost and in a timely manner. However, they noted, it will
be several years before they will be able to fully assess the
effectiveness of award fee contracting changes. The new policy is
based on overall contract performance, rather than on interim
progress. Consequently, officials said, they will be able to gauge
definitive, long-term results only after contract completion.
Analysis of contract costs will be critical at this juncture because,
according to a NASA procurement official, the award fee policy may
not be entirely effective in ensuring that contractors bid
realistically on contracts.
The impact of the new performance incentive requirement will be
limited initially because, in fiscal year 1993, NASA awarded only two
contracts that would have been required to incorporate performance
incentives had the policy been in effect. A NASA procurement
official believes that, although there is no requirement to do so,
performance incentives will also be used on some hardware contracts
with estimated cost and fee under $25 million. Procurement officers
must approve the use of performance incentives on such contracts
after determining that expected benefits will outweigh the associated
administrative burden. NASA was not able to estimate the likely
frequency of the use of incentives on CPAF contracts under the
$25-million threshold.
While the effect of the performance incentive feature may be limited,
the award fee contracting policy should have a substantial impact on
agency operations because the percentage of contract dollars that
NASA awards to business firms through CPAF contracts has been
increasing for some years, as illustrated in figure 2.1. In fiscal
year 1986, CPAF awards represented only about 43 percent of the total
awards to business firms. By fiscal year 1993, contracts and
modifications to contracts having CPAF provisions accounted for 76
percent, or $7.8 billion, of the total value of awards to business
firms, as shown in figure 2.2. This increase was primarily because
of the award of a few very large CPAF contracts, including those for
the space station and solid rocket motors for the space shuttle.
Figure 2.1: Eight-Year Trend
in the Percentage of Dollars
Awarded Annually to Business
Firms for Cost-Plus-Award-Fee
Contracts
(See figure in printed
edition.)
Note: Excludes smaller procurements, generally those of $25,000 or
less.
Figure 2.2: Percentage of
Dollars Directly Awarded to
Business Firms by Contract Type
for Fiscal Year 1993
(See figure in printed
edition.)
Note: Excludes smaller procurements, generally those of $25,000 or
less and orders under GSA Federal Supply Schedule contracts.
NASA uses CPAF contracts more than any other government organization
does. For example, in fiscal year 1993, the Department of Defense
used contracts and modifications to contracts having CPAF provisions
for less than 8 percent of the total value of its procurements, while
NASA used CPAF provisions for more than three-quarters of the total
value of such awards to businesses.
NASA uses CPAF contracts for operations and maintenance activities at
its field centers, as well as for procuring design, development, and
initial fabrication of state-of-the-art hardware, such as a new type
of planetary probe. According to NASA officials, NASA managers
prefer CPAF contracts because they give them more control and
flexibility. Under CPAF contracts, contractors' profits are linked
to NASA's evaluations of contractor performance, and NASA can adjust
the evaluation criteria periodically to reflect changes in management
emphasis or concern.
INCREASING CONTRACTOR LIABILITY
---------------------------------------------------------- Chapter 2:2
Like the award fee contracting initiative, the effort to increase
contractor liability developed slowly, partly because a number of
individuals and organizations raised concerns about the use and
impact of the proposed policy. NASA developed this policy primarily
to address congressional concerns about NASA's payments to
contractors for the costs of repairing or replacing articles when the
defects were caused by contractors' negligence. Currently,
contractors' liability for the cost to correct failures to comply
with requirements under cost-type, research and development contracts
is limited. Federal agencies, including NASA, currently assume total
responsibility for the cost of correcting defects under such
contracts, except in cases of fraud, willful misconduct, and lack of
good faith by contractors' senior management.
In November 1992, the Congress directed NASA to review options and
develop a policy for sharing risk with research and development
contractors.\3 In June 1993, NASA reported the results of its
assessment and, in February 1994, published for public comment a
proposed policy that may be modified before it is implemented. The
proposed policy would increase contractor liability for correcting
defects in material or workmanship or other failures to conform to
requirements of high-dollar value, research and development,
cost-type contracts. If implemented after a public comment period
that ended on April 18, 1994, the policy probably will become
effective in summer 1994, according to a NASA procurement official.
The proposed policy adds two situations under which NASA could impose
on contractors additional responsibility to remedy failures to comply
with contract requirements. Specifically, a contractor would be
financially liable for correction or replacement if NASA determined
that the contractor (1) had not applied its best efforts toward the
research and development objectives of the contract or (2) had not
followed proper procedures conforming to generally accepted practices
in performing routine operations, such as moving equipment or
conducting standardized tests. In these instances, NASA could hold
the contractor liable for 50 percent of the cost for remedying each
failure or 10 percent of the contract value at the time the failure
occurred, whichever is less.\4
NASA developed its contractor liability policy slowly for two main
reasons. First, NASA had focused its efforts primarily on developing
a new approach to award fee contracting because the agency
anticipated that award fee changes would be sufficient to deal with
hardware performance concerns and a separate contractor liability
policy would not be needed. Second, industry, the Department of
Defense, the General Service Administration, and others expressed
concerns about the initiative. In March 1993, NASA published
preliminary information about proposed changes in the Federal
Register. Comments on the proposed contractor liability policy, as
well as those from NASA personnel during the policy development
process, focused on a variety of issues, including concerns that
contractors would focus on reducing risk by avoiding technically
risky, cutting-edge projects and solutions;
NASA's contracts applying this policy would be less attractive to
potential offerors, thus reducing competition;
increased allocation of risk to the contractor is inconsistent with
the use of cost-type contracts and would, in effect, convert
cost-type contracts into fixed-price contracts; and
use of the negative incentive fee provision in the new award fee
policy concurrently with the proposed contractor liability
policy would be tantamount to penalizing the contractor twice
should a product fail to perform.
A NASA procurement official said he does not believe that contractors
will avoid high-risk projects or decline to bid for contracts if the
proposed policy is implemented. He noted that the proposed policy
explicitly exempts failures in high-risk projects that occur despite
the contractor's best efforts. The official also noted that
contractors will probably be willing to compete and assume more risk
because there are relatively few large dollar contracting
opportunities at NASA. In any event, NASA's judgment to cite a
contractor for not applying its best effort or for not following
proper procedures can be challenged by the contractor.
The new contractor liability rule would apply only to hardware
contracts with an estimated award value of $50 million or more. In
fiscal year 1993, NASA awarded only one such contract. Only one
additional contract awarded in fiscal year 1993 would have been
covered by the proposed policy if the threshold were lowered to $25
million or more.
--------------------
\3 Section 401 of the National Aeronautics and Space Administration
Authorization Act for Fiscal Year 1993 (P.L. 102-588) directed the
agency to review its contracting procedures.
\4 Ten percent is the maximum liability for the entire contract, not
for each incident. These limits were not intended to bear a true
relationship to the damages suffered by NASA, according to NASA
officials.
REPORTING ON CONTRACTOR
PERFORMANCE
---------------------------------------------------------- Chapter 2:3
In January 1993, 6 months after the NASA Administrator directed that
it be developed, the agency implemented a system of contractor
metrics, a set of standardized data for reporting specific measures
of contractor performance. NASA designed this agencywide system to
meet two recognized needs. Specifically, its purpose is to (1)
establish an ongoing dialogue between upper-level NASA and corporate
management and (2) simplify and standardize the information given to
high-level NASA officials on active contracts. Previously, NASA had
communicated with its contractors mainly at lower levels, and
high-level NASA officials, in a time-consuming process, reviewed
large amounts of data presented in a variety of formats. NASA
officials also believe that the metrics system may help management
identify troubled programs that need additional review or should be
considered for cancellation because of cost, schedule, or technical
problems.
Implementation began in January 1993, and the present system may be
expanded or modified. Currently, the metrics system reports on
approximately 30 NASA contracts and uses data already routinely
collected on all NASA contracts with an estimated value greater than
$25 million. Contracts for metrics reporting are selected annually
and are usually those with substantial near-term funding
requirements. In fiscal year 1993, the metrics system was used to
report on the performance of 14 contractors with 30 contracts valued
at $5.9 billion, or about 60 percent of NASA's fiscal year 1993
funding to business firms.
The contractor metrics system addresses seven key categories. Trend
information is reported in the following four areas: cost, schedule,
subcontracting plan, and award fee. Information includes the extent
to which the contractor has met cost, schedule, and subcontracting
plans, and the percentage of available award fee paid. A sample cost
metric for a service contract is provided in appendix IV.
There is no single measure, or set of measures, used on contracts for
two other performance standards--technical and safety and
mission-assured performance. These metrics are uniquely tailored to
each contract and may change as the program evolves. The technical
metric could focus on a number of different areas, including
compliance with weight specifications or payload status. The safety
and mission-assured performance metric could report on such items as
defects per unit. These six metrics are reported on a
contract-by-contract basis. The final metric--continual
improvement--is a voluntary metric reported by contractors that
addresses improvements in their overall business practices. Nine of
14 contractors described their continual improvement activities in
the most recent reporting cycle.
The seven metrics, together with contract status summary information
and an overall assessment by the NASA project manager, is reported
semiannually to each contractor's chief executive officer. For
comparison purposes, these reports to contractors also include
unlabeled information on other contracts' metrics, including data on
estimated cost growth at contract completion. In March 1994, these
estimates ranged from about minus 20 percent to about 270 percent and
averaged 50 percent. NASA has issued reports to contractors
twice--first in July 1993 and again in March 1994. The status
summaries and project manager assessments and two metric
categories--cost and schedule--are updated and reviewed quarterly
within NASA.
REDUCING THE VALUE OF UNPRICED
CONTRACT CHANGES
---------------------------------------------------------- Chapter 2:4
In an effort to manage contract changes more effectively and to help
control cost growth, NASA has taken several steps over a 3-year
period to reduce the number and value of the agency's unpriced
contract changes and to price them in a more timely manner. Although
earlier efforts have not generally brought about lasting
improvements, recent actions have resulted in a significant reduction
in the value of unpriced contract changes.
Government contracts contain a clause that allows (1) contracting
officers to make certain changes within the general scope of the
contract and (2) contractors to start work and incur costs before
agreeing with the contracting agency on terms and conditions of the
change, including price. However, issuing unpriced contract changes
should not be standard operating procedure, and, when they are used,
they should be priced as soon as practicable. Until firm prices are
negotiated for contract changes, the contractor has limited incentive
to control costs. Also, if work is completed before pricing the
change, the government will not have had the opportunity to review
proposed costs and consider more efficient production methods or
management controls.
NASA designated the effort to reduce the value of unpriced contract
changes as a formal procurement initiative in June 1993, and in May
1994, NASA implemented an agencywide policy on pricing such changes.
To monitor the effectiveness of this policy, the agency will continue
to gather data on the number, value, and age of unpriced change
orders at field centers. NASA's new policy requires that unpriced
change orders be issued on a strict exception basis; center directors
approve unpriced change orders with an estimated value of more than
$1 million; and the cognizant Deputy Associate Administrator approve
unpriced change orders for the space station with an estimated value
of more than $10 million.\5 However, approval requirements may be
waived in certain instances, including changes that involve safety
issues or decreases in contract value. NASA's current initiative
also focuses on improving the change order management process,
including the need to develop a clear and complete technical
definition of changes, as well as realistic cost estimates.
Before agencywide implementation, the new policy had been in effect
since October 1993 at the four NASA centers that accounted for about
90 percent of the value of unpriced change orders. During a 9-month
period, from October 1, 1993, when the policy became effective,
through June 30, 1994, the value of unpriced change orders at these
four centers dropped from almost $5 billion to about $1.8 billion,\6
a 63-percent reduction. Also during this 9-month period, the value
of unpriced change orders over 180 days old dropped by 87 percent at
the four centers.
In 1992 and 1993, the two centers with the largest backlogs of older,
unpriced change orders reviewed and revised center policies to
improve pricing timeliness. These actions, however, did not lead to
a lasting reduction in the value of change orders unpriced for more
than 180 days.\7 As of September 30, 1993, there were about $3.4
billion in change orders unpriced for more than 180 days at these two
centers, an increase of about $500,000 from the previous year. The
value of such unpriced change orders has decreased since October 1,
1993, when NASA's new policy took effect at four centers, as
discussed above.
Earlier agencywide efforts to reduce unpriced change orders began in
late 1990, when NASA headquarters increased oversight and reporting
requirements for unpriced contract changes. This effort
significantly reduced the number of unpriced contract changes
outstanding for extremely long periods of time. For example, in
1991, the value of unpriced contract changes over 2 years old dropped
from more than $3 billion to just over $131 million in a 6-month
period.
NASA, however, did not meet other goals established in October 1991,
more than a year and a half before the agency formally announced its
initiative to reduce the value of unpriced contract changes. For
example, NASA had wanted to eliminate all unpriced changes over 540
days old by the end of March 1992. But, by that date, 17 changes,
with an estimated total value of more than $114 million, were over
540 days old. The average age of these unpriced changes was 750
days, and one change was more than 1,300 days old. NASA also wanted
to eliminate all unpriced changes over 360 days old by the end of
September 1992. As of September 30, 1993, 1 year past the target
date, 69 changes, with an estimated total value of $1.9 billion, were
over 360 days old.
A NASA official noted that increasing the awareness of risk is one
major requirement for success in efforts to achieve a long-term,
permanent reduction in the value of unpriced change orders.
Specifically, he said, NASA must "sensitize people" to the pitfalls
of issuing such changes under cost-type contracts. This official
believes that NASA has begun to sensitize both procurement and
program office personnel to the inherent risk of unpriced contract
changes through various means, including offering formal training
classes and using employee teams to develop methods to streamline the
pricing process.
--------------------
\5 A NASA official said the space station program approval level and
authority is different because managers monitor these changes very
closely, and the value of such changes is likely to be large because
of the relative size of the program.
\6 About 75 percent of the total $1.8 billion in unpriced change
orders was due to changes that reduced the scope and value of two
contracts.
\7 NASA officials generally use the 180-day period as a guideline for
completing negotiations on unpriced contract changes.
STRENGTHENING TRAINING FOR
CONTRACTING OFFICERS' TECHNICAL
REPRESENTATIVES
---------------------------------------------------------- Chapter 2:5
Contracting officers appoint contracting officers' technical
representatives (COTRs) to assist them in managing the technical
aspects of contracts. NASA's efforts to strengthen training for
COTRs stemmed directly from evaluations demonstrating the need for,
or recommending changes in, such training. These evaluations
identified problems with COTRs' understanding of their roles,
responsibilities, and scope of authority; noted instances where
COTRs' actions eroded the authority of contracting officers; and
cited cases where unauthorized technical representatives directed
contractors to work beyond the contracts' original requirements and
inappropriately issued task orders. The amount and quality of
training for COTRs have been inconsistent throughout NASA. NASA
officials acknowledge that this may have been a contributing factor
to significant deficiencies in the management of prime and
subcontracts. In 1992, we recommended that the NASA Administrator
establish and enforce minimum training requirements for technical
representatives--requirements that emphasize COTRs' roles and
responsibilities, scope of authority, and relationship to other
members of the procurement management team.\8
NASA identified COTR training as an agencywide initiative in June
1993, and, in January 1994, it issued agencywide guidance that
specifies mandatory COTR training areas, including contracting
authority, contract modifications, surveillance plans, and government
property. The guidance makes procurement officers at each center
responsible for ensuring that the COTR training provided addresses
these areas in sufficient detail. All COTRs must be fully trained in
these required areas by April 1, 1995. After that date, contracting
officers will be required to verify that technical personnel have
been trained in the mandatory topics before appointing them as COTRs.
The new policy does not require formal training for those COTRs who
contracting officers believe have met the requirement through
previous training and experience.
--------------------
\8 NASA Procurement: Agencywide Actions Needed to Improve Management
of Contract Modifications (GAO/NSIAD-92-87, Mar. 2, 1992).
ACHIEVING SMALL AND
DISADVANTAGED BUSINESS GOALS
---------------------------------------------------------- Chapter 2:6
NASA's efforts to increase small and disadvantaged business (SDB)
contracting have helped the agency exceed established goals 1 year
ahead of schedule. In 1989, the Congress directed NASA to establish
and implement a plan to award at least 8 percent of its contract
dollars to SDBs. At that time, NASA was doing about 5.5 percent of
its business with SDBs. As currently defined, SDBs include small
business concerns or other organizations owned or controlled by
minorities, women, historically black colleges and universities, and
educational institutions serving other minorities.\9
In 1990, NASA set the end of fiscal year 1994 as the target date for
meeting its congressionally mandated 8-percent goal. However, by the
end of fiscal year 1993, 1 year earlier than planned, NASA exceeded
its goal by awarding almost $1 billion, or 8.5 percent of its total
awards to business, to SDBs. This amount included about $443 million
awarded to prime contractors and about $557 million awarded to
subcontractors. The NASA Associate Administrator, Office of Small
and Disadvantaged Business Utilization, projects that the agency will
meet its SDB contracting goal again in fiscal year 1994.
Pivotal to NASA's efforts to achieve its SDB contracting goal was a
1992 plan that included provisions for
rating upper-level agency managers on their performance in meeting
socioeconomic goals;
directing high-level managers to report on actions to increase SDB
subcontracting on NASA's top 100 contracts;
requiring headquarters to review and approve of contract
consolidations--combining several small contracts into one large
contract for administrative efficiency--that might reduce prime
awards to SDBs; and
setting aside 26 contracts with an estimated value of more than
$300 million for SDBs. (As of June 24, 1994, NASA had awarded
16 of these contracts--an estimated total value of $97.5
million.)
NASA's future plans call for increasing the value of SDB contracts in
high-technology areas, an area of interest to both the Congress and
the NASA Administrator. NASA has taken several steps to achieve this
goal, including establishing a committee to advise the agency on
increasing awards to SDBs, with emphasis on high-technology areas.
The agency has developed a SDB/aeronautics forum to provide SDBs with
high-technology capabilities an opportunity to brief center technical
personnel about their qualifications and experience. In addition, by
October 1994, NASA plans to implement a mentor/protege program to
encourage NASA contractors to assist SDBs in enhancing their business
and technical capabilities and to increase their participation on
NASA contracts.
--------------------
\9 These minorities include Hispanic and Asian Americans, and native
Americans, Alaskans, and Hawaiians.
COMMUNICATING WITH INDUSTRY
---------------------------------------------------------- Chapter 2:7
In October 1991, NASA established the NASA/Industry Process Action
Team to identify and discuss procurement-related issues that hinder
the effectiveness and efficiency of NASA's acquisition process. The
team currently has 31 members, including NASA contractors, a law
firm, and NASA representatives. Membership is rotated each year
among interested industry organizations through a random selection
process. The team has representatives from all segments of NASA's
contractor community, including both large and small NASA contractors
and SDBs.
Meetings are held about 8 to 10 times per year, or as often as
necessary to discuss major procurement issues and initiatives. Team
members present their individual or organizational ideas and
comments. The team is not permitted to vote on the issues presented
or to form a consensus on proposed changes. NASA briefs the team,
which, in turn, provides input on many of its procurement improvement
initiatives, including contractor metrics, contractor liability,
award fee policy, and cooperative agreements. The team provides NASA
policy representatives with an opportunity to learn about various
industry perspectives before developing draft policy that will be
available for public comment through the Federal Register. NASA is
not obligated to accept any of the viewpoints offered.
The initial team assisted in the development and presentation of a
NASA/industry conference in Houston, Texas, in March 1992. During
this conference, the team cataloged 61 areas of possible improvement,
and NASA has completed action on all but 10 of these. In some cases,
after evaluation, NASA decided not to implement the team's
suggestions. However, NASA acted on many, including those related to
small business and SDB goals, early industry involvement in NASA
procurements, more timely proposal evaluation and source selection,
subcontract management, and change orders.
One issue that was originally closed--termination liability--was
subsequently reopened when industry members wanted to further discuss
the financial risk to industry. Following the team discussions, NASA
drafted an amendment to the NASA FAR Supplement, establishing a
mechanism for funding termination liability on cost-type contracts.
This change addressed the major concerns of team members, while
providing NASA headquarters personnel with a better estimate of
potential liability.
PROGRESS OF PILOT TEST ON
STREAMLINING MIDRANGE
PROCUREMENT PROCEDURES
---------------------------------------------------------- Chapter 2:8
NASA's new midrange procurement procedures were designed to reduce
procurement time and effort without adversely affecting quality. As
currently designed, each midrange acquisition of supplies and
services is reserved exclusively for small business concerns. In
fiscal year 1993, NASA awarded 1,863 new contracts, of which 1,579,
or approximately 85 percent, were midrange procurements with annual
values between $25,000 and $500,000 and total 5-year values up to
$2.5 million.
In July 1993, NASA began a pilot test of midrange procurement
procedures at Marshall Space Flight Center. By the end of March
1994, Marshall had awarded 72 contracts, all for Small Business
Innovative Research (SBIR), under the pilot test. Preliminary
results show a reduction in average lead time for awarding such
contracts. A NASA official noted that this reduction is especially
significant. Since Marshall already had the lowest SBIR lead time in
the agency, lead time reductions are likely to be achievable at other
centers.
A NASA official noted that the introduction of an electronic commerce
system (a computer bulletin board) as the sole means of providing
information about procurements is also likely to reduce average lead
time. The exclusive use of an electronic commerce system for
expediting communication with prospective offerors in announcing and
awarding midrange procurements has not yet been approved. Currently,
all proposed procurement actions over $25,000 must be published in
the Commerce Business Daily.\10 This process, including required
response waiting times, is slow.
The agency cannot yet gauge the ultimate impact of midrange
procurement procedures on the business community because only limited
data are available. However, NASA has not received any unfavorable
comments from offerors, and no formal bid protests have been filed.
Also, a NASA official noted Marshall personnel are enthusiastic about
the midrange procurement procedures, and other centers have asked
headquarters to expand the pilot. NASA expects to begin
implementation at Johnson Space Center and at its headquarters'
Acquisition Division in the summer of 1994.
--------------------
\10 NASA must obtain waivers before it can use an electronic bulletin
board as the sole means of communicating requirements and providing
copies of solicitations.
PROBLEMS IN PLANNING AND
IMPLEMENTING INITIATIVES
============================================================ Chapter 3
The effectiveness of some of NASA's procurement improvement
initiatives may be limited. Specifically, (1) the proposed
contractor liability policy will be difficult to administer, may
result in higher contract prices, and could adversely affect
subcontractors who are unwilling or unable to risk increased
liability; (2) the potential effect of the proposed contractor
liability policy on SDBs has not been evaluated; and (3) the policy
guidance for the training of COTRs is incomplete.
POTENTIAL PROBLEMS WITH
PROPOSED CONTRACTOR LIABILITY
POLICY
---------------------------------------------------------- Chapter 3:1
The concept behind NASA's proposed contractor liability policy
appears to be a simple one--contractors working under cost-type,
research and development contracts should bear additional financial
responsibility for failures to comply with contract requirements.
The proposed policy, if properly implemented, may improve NASA
procurements under some high-dollar value contracts. However, this
policy will be difficult to administer, may lead to higher contract
prices, and could affect subcontractors with limited capability to
risk increased liability.
The proposed policy increases contractors' liability for the cost of
replacement or correction of defects. A NASA official said he
assumes that NASA would apply this policy even in cases where the
agency does not intend to replace or repair a faulty item. This
option is not addressed, however, either in the existing contract
clause or in the proposed policy. Consequently, NASA's decision to
assess liability in cases when the agency does not intend to replace
or repair an item could be disputed by the contractor.
The proposed clause also requires that NASA determine liability for
the cost of replacement or correction of defects. NASA's
determinations of contractor liability for the cost of replacement or
correction of defects requires applying general concepts that are yet
to be precisely defined, such as "best efforts," "routine
operations," and "generally accepted industrial or engineering
practices." Even determining "failures to comply with the
requirements of the contract" could be difficult, especially in a
cost-type, research and development environment, where initial
requirements are frequently broadly stated. NASA's decision to
assess liability can be disputed by the contractor and could result
in protracted negotiations and potentially costly litigation.
A NASA procurement official acknowledged that the proposed clause
will be difficult to administer and will probably lead to an
increasingly adversarial relationship between NASA and its
contractors. He noted, however, that NASA has tried to address some
concerns. For example, the policy reduces the potential for disputes
and litigation by requiring the use of advance agreements to identify
routine operations and research and development activities and by
placing the burden of proof on the agency for determining
liability.\1
Although contractors could not claim additional costs under the
contract to cover their increased risk of liability, contract prices
could increase if contractors request higher award fees to compensate
for this increased risk. Moreover, their costs could be higher if
they choose to increase their supervision and internal review
activities and/or develop and implement stricter standards and
procedures to reduce the chances of a partial or complete failure. A
NASA procurement official acknowledged that individual contracts may
cost more as a result of the contractor liability policy but noted
that NASA hopes ultimately to save money by avoiding catastrophic
losses.
As previously noted, in fiscal year 1993, NASA awarded only one
contract that would have been subject to the proposed contractor
liability provision had it been in effect. Furthermore, in view of
NASA's constrained budget situation, it is unlikely that many such
contracts--cost-type, research and development contracts for hardware
with an estimated value of $50 million or more--will be awarded in
the foreseeable future. Therefore, major aerospace companies and
smaller subcontractors probably will not be immediately affected by
the policy. In the future, however, even with a limited number of
prime contractors, larger numbers of subcontractors, including SDBs
and universities, could be affected if prime contractors pass on a
share of the potential liability through "flow-down" provisions in
their contracts. Subcontractor liability under such provisions would
be based on the value of the subcontract and apply to the activities
performed by the subcontractor. Comments in response to the proposed
policy published in the Federal Register noted that it could affect
the ability or willingness of subcontractors who are SDBs or
universities to work on NASA contracts because of the potentially
significant risk of liability that could, in some cases, undermine
their economic viability.
--------------------
\1 An earlier draft of the proposed policy required that the
contractor prove that it was not liable.
EFFECT ON SDBS NOT THOROUGHLY
EVALUATED
---------------------------------------------------------- Chapter 3:2
NASA developed and implemented its contractor liability and SDB
initiatives independently. Consequently, the agency has not yet
evaluated the potential effects the proposed contractor liability
policy may have on SDBs or on NASA's ability to achieve future SDB
goals.
As previously noted, increasing financial liability for contractors
could affect the ability of SDBs with limited resources to work on
NASA subcontracts. In addition, liability for hardware that fails to
perform properly or for other failures to comply with contractual
requirements could affect large contractors' willingness to
subcontract work to SDBs, particularly in high-technology areas. An
official in the Office of Small and Disadvantaged Business
Utilization at NASA headquarters acknowledged that the agency has not
fully considered the potential impact of the proposed contractor
liability policy on subcontractors, including SDBs. He said the
proposed contractor liability policy may make it more difficult for
some SDBs to subcontract with NASA and noted that some prime
contractors are already hesitant to do business with SDBs, especially
those working in high-technology areas. In June 1993, a report by
NASA's Minority Business Resources Advisory Committee\2 also raised
concerns about the willingness of prime contractors to subcontract
high-technology work to SDBs and noted that some contractors classify
such subcontracting as "too risky" to consider.
Even though NASA has coordinated initiatives in some instances, there
are further opportunities for enhancing coordination. For example,
NASA revised its award fee metrics to reflect changes in the new
award fee process. The contractor metrics system also supports the
SDB initiative by presenting information on contractors' progress in
meeting SDB goals. However, contractor metrics could provide
additional support for the SDB goal of increasing subcontracting
opportunities for SDBs in high-technology areas. The contractor
metrics system does not currently report information on the
percentage of SDB subcontracting work that could be classified as
"high-technology."
--------------------
\2 The committee was established by the NASA Administrator. It
includes representatives from SDBs and academia.
GUIDANCE FOR COTR TRAINING
INCOMPLETE
---------------------------------------------------------- Chapter 3:3
NASA has issued incomplete guidance on a new policy that is intended
to strengthen COTR training requirements. This guidance identifies
mandatory training areas and specifies that COTRs must be fully
trained in these areas by April 1, 1995. A NASA procurement official
told us that new training would not be required for those technical
personnel who are judged to already have sufficient knowledge of the
mandatory areas as a result of previous training and experience.
Contracting officers are responsible for determining if technical
personnel need additional training before appointing them as COTRs,
he noted. The training policy guidance, however, does not provide
any information on available options for fulfilling mandatory
training requirements and on establishing and applying criteria to
determine when no additional training is needed.
The lack of guidance in these areas is likely to lead to inconsistent
agencywide application of the new policy. In commenting on the
proposed new policy, several centers expressed concerns that limited
funding could make it difficult to meet the new COTR training
requirements. Constrained training budgets and heavy competing
demands do not bode well for the strict, uniform application of the
new policy from center to center. Additional guidance would help
ensure more consistency in interpreting and applying the new COTR
training requirements.
CONCLUSIONS, RECOMMENDATIONS, AND
AGENCY COMMENTS
============================================================ Chapter 4
CONCLUSIONS
---------------------------------------------------------- Chapter 4:1
For 7 successive years, NASA has acknowledged that its procurement
system and contract management had serious internal control
deficiencies; over the years, the agency has designed, developed,
and, in many cases, implemented a variety of improvement initiatives
to address these and other problems. The initiatives we reviewed
address a variety of problems, some of which are long standing,
persistent, and complex. Correcting them will be a demanding
process, and much hard work remains in order to achieve effective,
long-term results.
NASA has made uneven progress in planning and implementing its
procurement improvement efforts. The agency's cycle of successes and
failures in its efforts to reduce the value of its unpriced change
orders provides a cogent example of why prolonged, intensive effort
is frequently needed to correct persistent problems. Progress on
some initiatives has been slow for several reasons, including
centers' and contractors' objections to changing the traditional
approaches to motivating and rewarding contractors and allocating
risk. NASA has dealt with such resistance by soliciting center and
industry comments and making changes to address some of their
concerns as it developed new policies.
Some of NASA's procurement improvement initiatives have the inherent
potential to create a more adversarial relationship between NASA and
its contractors or have other unintended, potentially adverse side
effects. Key among these are the potential effects of increasing
contractors' liability on SDBs, especially those qualified to do
high-technology work--a focus of NASA's SDB initiative. Where such
potential effects are identified, NASA may need to consider options
for mitigating them. Such options should not involve lowering
applicable standards or exempting SDBs from potential liability.
Instead, options could include offering further incentives through
the award fees to prime contractors that subcontract with SDBs,
especially in high-technology areas; providing advice and assistance
to SDBs, if needed, through training and mentoring activities; and
redoubling ongoing efforts to acquaint center personnel with SDBs
with high-technology capabilities.
To further support its efforts, NASA could adjust its SDB metric
category to present information on the percentage of high-technology
work subcontracted to SDBs. To do this, NASA needs to establish a
standard definition of high-technology work and to develop official
statistics on the value of such work awarded to SDBs, either as prime
contractors or subcontractors. Using its metrics reporting system
would enable NASA to effectively identify and track its progress
toward involving SDBs in high-technology work for critical contracts
that account for more than half of the agency's funding to business
firms.
NASA's procurement reform initiatives are a vital part of its overall
efforts to improve operations. The problems or difficulties the
agency has had in planning and implementing its initiatives must not
be confused with failure. They do, however, demonstrate the need for
sustained management attention, routine monitoring, and effective
followup. Particularly critical are NASA's plans for monitoring
centers' compliance with the new award fee policy because of the
nature and scope of revisions, their impact on agency operations, the
difficulty of implementing changes affecting the agency's traditional
ways of conducting business, and a history of compliance problems at
field centers. Close monitoring is needed to ensure that center
personnel have properly interpreted and are adequately complying with
new rules.
Continuing commitment to improving procurement activities is
essential because correcting problems is critical to safeguarding
increasingly scarce resources and ensuring their efficient and
effective use on behalf of the government. Without effective
procurement and contract management, NASA cannot reasonably ensure
that billions of dollars in contract funding will be spent
effectively and that the products it purchases will function
properly.
RECOMMENDATIONS
---------------------------------------------------------- Chapter 4:2
We recommend that the NASA Administrator
evaluate the potential impact of the proposed contractor liability
policy on SDBs and consider options to mitigate any adverse
impacts identified;
define "high-technology" work and develop and implement a
methodology for consistently measuring progress toward NASA's
goal of increasing SDB participation in such work, either as
prime contractors or subcontractors;
direct that special reviews be conducted at centers to ensure that
training courses for COTRs sufficiently address new minimum
training requirements; and
provide guidance to the centers on establishing and applying
criteria to determine when no additional training is needed to
meet new minimum training requirements for COTRs.
AGENCY COMMENTS
---------------------------------------------------------- Chapter 4:3
NASA agreed with our recommendations and generally agreed with the
report's contents. NASA provided comments on our analyses of some of
the initiatives. We considered these comments in preparing this
report. NASA's comments are reprinted in their entirety in appendix
II.
NEW INITIATIVES: PROCUREMENT
REINVENTION LABORATORY AND
COOPERATIVE AGREEMENTS
=========================================================== Appendix I
Since our review began, NASA has started two new procurement
improvement initiatives. In October 1993, NASA established a
procurement reinvention laboratory to identify and implement
innovative methods for acquiring goods and services. The initiative
resulted from, and operates under the auspices of, the National
Performance Review. The headquarters' Acquisition Division, which
accounts for about 6 percent of the total value of the agency's
acquisitions, was designated as the reinvention laboratory. The
Associate Administrator for Procurement has waived the requirement
for the laboratory to comply with the NASA Supplement to the Federal
Acquisition Regulation. The laboratory also has requested or has
received waivers from other agencies' internal procedures that affect
NASA's procurement process.
NASA's laboratory is currently involved in acquiring technology for
small spacecraft and is using this procurement to develop and test
strategies for focusing on reducing procurement time, streamlining
internal reviews, and motivating contractor performance. Although
the laboratory was originally expected to operate for 1 year, NASA
officials are considering extending the duration to 2 years. At that
time, NASA will evaluate the results and decide if the initiative
will be continued or expanded.
In February 1994, NASA identified cooperative agreements with
profit-making organizations as a procurement improvement initiative.
NASA expects to use these agreements to increase technology transfer
to industry. Although NASA has used cooperative agreements with
universities and nonprofit organizations in the past, cooperative
agreements between NASA and members of the private sector is a new
concept.
Cooperative agreement notices will be published in the Commerce
Business Daily to solicit industry interest in a particular effort.
NASA will also consider unsolicited proposals from industry.
Although NASA prefers working with teams of representatives from
different companies, an agreement with a single firm will be
considered. NASA has developed procedures for structuring
cooperative agreements and for monitoring progress--both technical
and financial. While cooperative agreements will not replace
contracts, NASA believes that such agreements will foster a closer
working relationship between government and private industry and will
assist for-profit firms in advancing and commercializing technologies
in which the government has unique capabilities.
(See figure in printed edition.)Appendix II
COMMENTS FROM THE NATIONAL
AERONAUTICS AND SPACE
ADMINISTRATION
=========================================================== Appendix I
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
The following are GAO's comments on the NASA's letter dated July 21,
1994.
GAO COMMENTS
--------------------------------------------------------- Appendix I:1
1. We discuss a variety of factors that demonstrated the need for
change and led NASA to develop and implement procurement improvement
efforts and initiatives. Other than NASA Inspector General and GAO
reports, factors mentioned include performance problems, assessments
by NASA management, and direction from the Congress and the NASA
Administrator.
2. The results of the change order initiative are not diminished by
the presentation. Rather, the presentation provides a historical
perspective on the difficulty of correcting this long-standing
problem. Moreover, we note in the first paragraph of the text on the
change order initiative that "recent actions have resulted in a
significant reduction in the value of unpriced contract changes." We
also provide details of NASA's current initiative and note the
63-percent reduction over a 9-month period at four centers.
PERFORMANCE INCENTIVES IN NASA'S
NEW AWARD FEE POLICY
========================================================= Appendix III
NASA provided the following example to illustrate how a performance
incentive might be structured under the agency's new award fee policy
guidance: In a contract requiring the delivery of a spacecraft, the
performance incentive unit of measurement is useful months in orbit.
If 12 months is the expected performance level, the 12th month could
be identified as the standard performance for which no incentive is
earned. If 24 months is the maximum useful life of the spacecraft,
the 24th month could be identified as the maximum performance level,
at which the contractor would earn the maximum performance incentive.
Interim measures of spacecraft life from 12 to 24 months would then
be identified with fees ranging from $0 to the maximum positive
incentive. The amounts associated with these interim measures should
correspond to the relative value to the government of each additional
month in orbit. NASA will not reward the contractor for above
standard performance that does not benefit the government.
A similar scale would be established for the negative incentive,
ranging from the 12th month for standard performance, for which no
penalty would be due, to a total, immediate, and permanent system
failure, for which the maximum negative performance incentive would
be due from the contractor. The amount of the maximum negative
incentive would depend on the type of hardware provided under the
contract. For research and development efforts (that is, the first
and second units produced), the maximum negative performance
incentive would be equal to the total amount of award fee actually
paid. For production efforts (that is, the third and all subsequent
units of any hardware item), the maximum negative performance
incentive would be equal to the total amount of award fee the
contractor could have earned. In other words, if there were a total,
complete, and permanent failure of a production item, the contractor
could experience a net loss on the contract.
SAMPLE METRIC: CONTRACT BASELINE
COST TREND ANALYSIS FOR A SERVICE
CONTRACT
========================================================== Appendix IV
(See figure in printed
edition.)
Note: Trend analysis chart presentation may include narrative and
supplementary data, including information on whether cost variations
are due to contractor or NASA changes.
Source: NASA.
MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix V
NATIONAL SECURITY AND
INTERNATIONAL AFFAIRS DIVISION,
WASHINGTON, D.C.
David R. Warren
Frank Degnan
Sandra D. Gove
Roberta H. Gaston