Business Systems Modernization: IRS Has Made Significant Progress in Improving Its Management Controls, but Risks Remain

During fiscal year 2004, IRS reported that it had made further progress toward establishing mature configuration management (CM) processes, including (1) developing and delivering training modules on CM processes and configuration control boards (CCB) and (2) completing internal compliance assessments on four Business Systems Modernization (BSM) projects to evaluate each project's practices to determine whether they are being followed and support the project's goals. However, the results of these assessments demonstrated the need for additional improvements. Several discrepancies were identified against three of the projects evaluated, including processes or documentation that were found to be noncompliant with Modernization and Information Technology Services (MITS) CM processes and procedures and/or were inadequate to perform the intended function. The assessment reports cited significant deficiencies and areas of concern related to requirements management, requirements traceability, and configuration audits, and they noted that many of the discrepancies identified during prior assessments of two of the projects still remained unresolved or only partially addressed. In September 2004, the CIO issued a revised MITS CM Directive to establish organizational responsibilities for implementing these processes enterprise-wide and addressing deficiencies. The charters for the associated CCBs and the enterprise-wide Engineering Review Board were approved by the CIO in December 2004. In January 2005, a new Associate CIO was hired to head the new Enterprise Services organization within the enterprise, which includes a new enterprise-wide CM policy office. This office is responsible for implementing the CM Directive and institutionalizing these processes. Since January 2005, IRS has (1) developed an implementation guide and checklist for establishing CM across the organization, (2) developed and conducted various CM training modules, (3) established several CCBs, (4) conducted additional CM process compliance assessments on BSM projects, and (5) revised the compliance assessment procedure to assign the issues and risks resulting from these reviews to the applicable projects for mitigation. Future plans include continuing CM process compliance assessments (four per year, depending on resources) on modernization projects both under development as well as transitioned to production.

In September 2003, the Treasury Inspector General for Tax Administration reported that IRS's MITS organization had made significant progress in developing its human capital strategy, including the development of a 5-year human capital plan; however, the strategy was not complete. Specifically, IRS had not yet (1) identified and incorporated human capital asset demands for the modernized organization, (2) developed detailed hiring and retention plans, or (3) established a process for reviewing the development of a human capital strategy and monitoring its implementation. In August 2004, the current CIO identified the completion of a human capital strategy as a high priority. Since then IRS has developed and utilized a human resources capacity management model to identify asset demands and determine where skills gaps exist in the current BSM organization, prioritized its BSM staffing needs within functional divisions, and designed and established a revised organizational structure and staffing profile/level that meets the needs of the current BSM program. In addition, in October 2005, IRS completed its BSM Human Capital Strategy, which includes a detailed recruitment plan to bring necessary personnel on board as well as strategic plans for employee training, leadership development, and workforce retention. IRS is implementing its strategy to develop and maintain a knowledgeable and highly-skilled workforce to more effectively manage the BSM program and support future operations of modernized IT systems. For example, within the new Applications Development (AD) organization (formed by merging BSM and the Business Systems Development production information applications systems support organization), IRS has established a Centers of Excellence office to, among other things, (1) determine required skill sets and level of competency for each area of expertise and grade level in the AD organization and (2) develop a recruitment strategy and define training requirements. The AD organization recently hired and trained 25 entry-level programmers to support project development efforts.

Despite the progress that IRS and the prime contractor made during fiscal years 2003 and 2004 toward establishing a BSM cost and schedule estimation system that is consistent with IT industry best practices, a September 2004 internal review of the prime contractor's cost and schedule estimation capability concluded that estimation processes and compliance were still inadequate in some respects. The report stated that (1) the primary area of weakness was in the prime contractor's execution, since there was little evidence that the cost and schedule estimating guidance was being followed, and (2) future efforts will focus on the institutionalization of established guidance. As a result of IRS's assessment of the prime contractor's performance and fiscal year 2005 budget reductions, the agency decided to shift primary responsibility for cost and schedule estimation from the prime contractor to IRS staff. IRS indicated that, as a part of this transition, it must redefine BSM cost and schedule estimation processes consistent with the current scope of the program and strengthen its capabilities in this area. Although the agency hired additional technical expertise for cost and schedule estimating during fiscal year 2004, it acknowledged in May 2005 that it did not yet have all the expertise necessary to estimate cost and schedule effectively. During 2005 and 2006, IRS took a number of actions to further improve the BSM cost and schedule estimation process and procedures. For example, IRS mapped its existing estimation capabilities to the Software Engineering Institute's requisites for reliable estimating processes, identified and prioritized gaps between the agency's existing capabilities and industry best practices, and developed a phased improvement approach. Consistent with its prioritized phased approach, IRS has contracted with an estimation support contractor to augment the agency's project estimation capability and has established a Cost and Schedule Estimation Support Services Concept of Operations identifying support paths for requesting, managing, and overseeing in-house and contractor estimation support. In addition, the agency is creating an estimation data repository to capture historical cost and schedule data to enable identification and extraction of relevant baseline data to support estimation development. In a related effort, IRS has also developed and implemented, in May 2006, a scalable Basis of Estimate guide to promote uniform standards for development and submission of supporting documentation to enable comprehensive evaluation, replication, and assessment of estimates. IRS plans to continue to leverage and expand upon its current capabilities, within resource constraints, to achieve a cost-effective BSM cost and schedule estimating system.

In June 2003, IRS concurrently approved versions 2.1 of the Enterprise Architecture (EA) and Enterprise Transition Strategy (ETS). By ensuring that the ETS conforms with other revisions to the EA, IRS can better coordinate agencywide systems modernization improvements.

In August and September 2003, IRS issued BSMO guidance, a CIO memorandum, and a BSMO directive outlining (1) IRS's intent to utilize more fixed-price task orders where appropriate and guidance regarding the selection of appropriate contract types for BSM task orders; (2) the new IRS policy stipulating that contracts and task orders for BSM development projects in Milestones 4 and 5 will be fixed-price; and (3) implementation guidance and a strategy to facilitate the use of performance-based contracting in the development of BSM task orders. During fiscal year 2004, IRS' efforts focused on monitoring compliance with BSMO directives and templates for implementing performance-based contracting. IRS developed performance measures for assessing the extent to which certain performance-based contracting practices were introduced into the modernization environment, and used these measures to perform an initial assessment of task orders for modernization projects. Results indicated that IRS staff are applying the new guidelines for performance-based contracting, but a larger number of task orders must be assessed to provide a more reliable basis for evaluating compliance. In August 2004, IRS issued a revised Enterprise Life Cycle (ELC) directive that separated ELC milestone 4 into Milestones 4a (system design) and 4b (system development). This established Milestone 4a as a checkpoint at the end of the design phase where fixed-price task orders can be negotiated for business system development and deployment. During fiscal years 2005 and 2006, IRS (1) issued a Milestone 4A Statement of Work template for use in developing performance-based task orders, (2) developed revised guidance on the selection of appropriate contract types for BSM task orders, (3) issued a BSM performance monitoring directive establishing the requirement for preparing task order monitoring plans, (4) conducted performance-based contracting and performance monitoring training sessions, and (5) performed semiannual compliance assessments. As of September 2005, four assessments had been completed on a total of 19 statements of work and 18 monitoring plans, resulting in a cumulative average of about 80 percent compliance with the applicable performance-based contracting templates and guidance.