Business Systems Modernization: Air Force Needs to Fully Define Policies and Procedures for Institutionally Managing Investments
GAO-08-52
Published: Oct 31, 2007. Publicly Released: Oct 31, 2007.
Skip to Highlights
Highlights
In 1995, GAO first designated the Department of Defense's (DOD) business systems modernization program as "high-risk" and continues to do so today. In 2004, Congress passed legislation reflecting prior GAO recommendations that DOD adopt a corporate approach to information technology (IT) business systems investment management including tiered accountability for business systems at the department and component levels. To support GAO's legislative mandate to review DOD's efforts, GAO assessed whether the investment management approach of one of DOD's components--the Department of the Air Force (Air Force)--is consistent with leading investment management best practices. In doing so, GAO applied its IT Investment Management (ITIM) framework and associated methodology, focusing on the stages related to the investment management provisions of the Clinger-Cohen Act of 1996.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address specifying how systems that are in operations and maintenance will be reviewed and specifying how Air Force's business investment management system is coordinated with Joint Capabilities Integration and Development Syste, planning, programming, budgeting, and execution, and Defense Acquisition System. |
Closed – Implemented
Although Air Force still needs to specify how its business investment management system is to be coordinated with the requirements, budgeting, and acquisition systems, the department has taken actions to develop and issue the policies and procedures that address how these business systems will be reviewed by its investment review board. For example, in November 2008, Air Force updated its IT investment review guidance to require an annual review of all business systems, including those in operations and maintenance, and it defined the process by which its investment review board would review these systems. Further, the guidance specified how business investments, including those in operations and maintenance, were to be prioritized, using factors such as mission, strategic value, and risk. Collectively, these actions should help strengthen Air Force's investment management process by assuring that systems throughout the investment life cycle, including those in operations and maintenance, have adequate oversight and align to business needs.
|
| Department of Defense | To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address ensuring that systems in operations and maintenance are aligned with ongoing and future business needs. |
Closed – Implemented
The department has taken steps to address this recommendation. For example, Air Force's 2008 IT investment review guidance calls for the review of all business systems, to include those in operations and maintenance, and specifies how these business investments are to be prioritized using factors such as mission and strategic value and risk.
|
| Department of Defense | To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address selecting investments, including specifying how factors, such as cost, schedule, and performance data are to be used in making certification decisions. |
Closed – Implemented
The department has taken sufficient steps to meet the intent of the recommendation. For example, Air Force has developed IT investment management guidance that includes predefined criteria for adherence to cost, schedule, and performance milestones, and requires that all investments be scored against this criteria as part of the selection process. Air Force officials stated that the department plans update its IT investment guidance in November 2012.
|
| Department of Defense | To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address reselecting ongoing investments, including specifying how factors, such as cost, schedule, and performance data are to be used in making reselection decisions during the annual review process and providing for the reselection of investments that are in operations and maintenance. |
Closed – Implemented
The department has taken steps to meet the intent of the recommendation. For example, Air Force has developed IT investment management guidance that includes predefined criteria for adherence to cost, schedule, and performance milestones, and requires that all investments be scored against this criteria as part of the reselection process. In addition, the Air Force IT investment review guidance provides for the annual review of all business systems investments, to include those in operations and maintenance. Further, Air Force has stated its intent to update its IT investment guidance in November 2012.
|
| Department of Defense | To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address integrating funding with the process of selecting an investment, including specifying how the precertification authority is using funding information in carrying out decisions on system certification and approvals. |
Closed – Not Implemented
In 2011, Air Force officials identified activities currently underway to inform the Investment Review Boards and precertification authorities about funding during certification reviews. However, the department has yet to update its policies and procedures to specify how investment funding and selection are to be integrated and how the precertification authority is to use funding information to make certification or approval decisions. Air Force had stated its intent to replace its IT portfolio management and IT investment guidance in November 2012.
|
| Department of Defense | To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address overseeing IT projects and systems, including specifying policies and procedures that guide the implementation of corrective actions when program expectations are not met. |
Closed – Implemented
The department has taken steps to meet the intent of the recommendation. For example, Air Force has developed IT investment management guidance that includes predefined criteria and checkpoints for meeting cost, schedule, and performance milestones, and requires the development of corrective actions when a project deviates from milestones. In addition, while the department's IT investment management policies and procedures do not address the implementation of corrective actions for all systems, they do so for the larger business systems. Collectively, these actions should help strengthen Air Force's investment management process by assuring that systems have adequate oversight.
|
| Department of Defense | The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address creating and modifying IT portfolio selection criteria for business system investments. |
Closed – Not Implemented
While Air Force has assigned the Major Command and Functional Investment Review Boards responsibility for creating and modifying portfolio criteria (e.g., prioritization and investment trade-offs) for business system investments, it has not yet finalized draft policies, procedures, or criteria for conducting IT portfolio selection. The Air Force has described its current efforts to change its IT investment management organization and form a new branch to lay the foundation for integrated, efficient IT investment management processes, but this office is not yet established. Further, Air Force had stated its intent to replace its IT portfolio management and IT investment guidance, but this update is not expected until November 2012.
|
| Department of Defense | The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address defining the roles and responsibilities for the development and modification of the IT portfolio selection criteria. |
Closed – Implemented
The department has taken steps to address this recommendation. For example, Air Force's investment management guidance assigns responsibility for the development and modification of the IT portfolio selection criteria to an integrated working team consisting of Major Command representatives, Functional Portfolio Managers, and the Certification Process Manager; and assigns approval of the criteria to a Senior Working Group.
|
| Department of Defense | The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address analyzing, selecting, and maintaining business system investment portfolios. |
Closed – Implemented
The department has taken steps to meet the intent of the recommendation. For example, Air Force's IT Investment Review Guide describes an overview of four continuous integrated investment activities, including analysis, selection, control, and evaluation, to manage its investment portfolios. Further, Air Force has stated its intent to replace its IT portfolio management and IT investment guidance in November 2012.
|
| Department of Defense | The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address reviewing, evaluating, and improving the performance of the portfolio(s) by using project indicators, such as cost, schedule, and risk. |
Closed – Not Implemented
While the Major Command and Functional Investment Review Board draft charters state that the boards are responsible for reviewing indicators associated with portfolio management, such as architecture alignment, capability delivery, and risk, there are no policies and procedures indicating how they are to use these and other project indicators--such as cost, schedule, and risk--to review, evaluate, and improve their portfolios. The Air Force has described its current efforts to change its IT investment management organization and form a new branch to lay the foundation for integrated, efficient IT investment management processes, but this office is not yet established. Further, Air Force had stated its intent to replace its IT portfolio management and IT investment guidance, but this update is not expected until November 2012.
|
| Department of Defense | The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address conducting post-implementation reviews for all investment tiers and directing the investment boards to consider the information gathered to develop lessons learned from these reviews. |
Closed – Implemented
The department has taken steps to address the recommendation. For example, Air Force has developed policies and procedures for conducting post-implementation reviews for all defense business system investments that cost more than $1 million. Further, Air Force's acquisition guidance directs program managers to document in post-implementation reviews any identified shortfalls between actual program results and established performance goals and incorporate the results in lifecycle management planning documentation.
|
Full Report
GAO Contacts
Office of Public Affairs
Topics
DocumentationInformation technologyInternal controlsPolicy evaluationProgram managementReporting requirementsRisk assessmentRisk managementStrategic information systems planningStrategic planningSystems conversionsBusiness operationsBusiness planningBusiness transformationPolicies and proceduresProgram implementation