Historically, many of the Department of Homeland Security’s (DHS) major IT acquisition programs have taken longer than expected to develop or failed to deliver the desired value. In April 2016, to help improve the department’s IT acquisition and management, DHS identified Agile software development as the preferred approach for all of its IT programs and projects.
The Government Accountability Office (GAO) has recently examined DHS’s adoption of Agile software development to assess the extent to which the department has addressed selected leading practices for its transition.
To conduct its review, GAO identified leading practices for planning, implementing, and measuring organizational change that apply to DHS’s transition to Agile through its review of guidance published by the Project Management Institute and GAO.
GAO also reviewed work it performed to develop leading practices for Agile software development adoption. GAO analyzed DHS documentation, such as policies, guidance, plans, and working group artifacts and assessed them against the selected leading practices. The watchdog also reviewed the implementation of selected practices within individual IT projects and interviewed DHS officials to discuss any practices that were not fully implemented.
As it notes in its June 1 report, GAO found that DHS “has taken steps to implement selected leading practices in its transition from waterfall, an approach that historically delivered useable software years after program initiation, to Agile software development, which is focused on incremental and rapid delivery of working software in small segments”.
GAO continues that DHS has fully addressed one of the three leading practice areas (plan, implement, and measure) for organization change management and partially addressed the other two. Collectively, these practices advise an organization to plan for, implement, and measure the impact when undertaking a significant change.
The department earns praise for its fully defined plans for transitioning to Agile development. However, GAO found DHS has only partially addressed implementation—the department completed 134 activities but deferred roughly 34 percent of planned activities to a later date. These deferred activities are in progress or have not been started. With respect to the third practice – measure, DHS clarified expected outcomes for the transition, such as reduced risk of large, expensive IT failures. However, these outcomes are not tied to target measures. Without these, GAO maintains that DHS will not know if the transition is achieving its desired results.
GAO has previously reported on various programmatic and technical challenges that were limiting DHS’ efforts on Agile programs:
In 2016, it reported that the U.S. Citizenship and Immigration Services Transformation program, which was using Agile software development to modernize citizenship and immigration benefits processing, needed to improve testing of its software code and ensure its approaches to interoperability and end user testing met leading practices. GAO made 12 recommendations to improve Transformation program management, including ensuring alignment among policy, guidance, and leading practices in areas such as Agile software development and systems integration and testing. DHS concurred with the recommendations and has thus far implemented eight of them.
In October 2017, GAO reported that the Transportation Security Administration Technology Infrastructure Modernization program had not defined key roles and responsibilities, prioritized system requirements, or implemented automated capabilities that were essential to ensuring effective adoption of Agile. 14 recommendations were issued including that DHS should prioritize requirements and obtain leadership consensus on oversight and governance changes. DHS concurred with the recommendations and to date has implemented 13 of them.
In November 2018, GAO found that the U.S. Secret Service Office of the Chief Information Officer (OCIO) did not fully measure post-deployment user satisfaction with one project supporting the Information Integration and Technology Transformation investment. 13 recommendations were made to the U.S. Secret Service including that it establish a process that ensures the CIO reviews all IT contracts, as appropriate; and identify the skills needed for its IT workforce. DHS concurred with the recommendations but has not yet implemented them.
GAO reported in April 2019 that the Federal Emergency Management Agency Grants Management Modernization program had not yet fully established plans for implementing new business processes or established completed traceability of IT requirements. The watchdog made eight recommendations to implement leading practices related to reengineering processes, managing requirements, scheduling, and implementing cybersecurity. DHS concurred with the recommendations and has thus far implemented two.
According to GAO’s June 1 report, DHS has also addressed four of the nine leading practices for adopting Agile software development. For example, the department has modified its acquisition policies to support Agile development methods. However, GAO wants DHS to take additional steps to, among other things, ensure all staff are appropriately trained and establish expectations for tracking software code quality.
GAO has therefore made 10 recommendations to the Secretary of DHS, all of which the department agreed with:
- Ensure that the Director of Strategic Technology Management (STM), in collaboration with other members of the Information Technology Program Management Center of Excellence (ITPM COE), identifies the skills and resources needed to complete the work intended for the upcoming fiscal year, including the availability of supplementary staff, such as subject matter experts.
- Ensure that the Executive Steering Committee overseeing the activities of the ITPM COE establishes target measures for the department’s desired outcomes of its transition to Agile development.
- Ensure that the DHS Chief Information Officer (CIO) defines a process and associated set of controls to ensure that Agile programs and projects are reporting a set of core required performance metrics for monitoring and measuring Agile adoption.
- Ensure that the ITPM COE, in coordination with the CIO, begins measuring results associated with the transition to Agile and the success of the transition based on its impact on the department.
- Ensure that the CIO, in collaboration with the Chief Procurement Officer, through the Homeland Security Acquisition Institute, establish Agile training requirements for senior stakeholders.
- Ensure that the Chief Human Capital Officer, in collaboration with the CIO, consider modifications to the current employee recognition and performance management governance to ensure that teamwork and team performance of Agile programs and projects are incentivized.
- Ensure that the CIO, in collaboration with the Chief Procurement Officer, through the Homeland Security Acquisition Institute, establish Agile training requirements for staff outside of the acquisition workforce but assigned to Agile programs.
- Ensure that the CIO, upon establishing a set of core performance metrics, tracks and monitors the pace of Agile team development.
- Ensure that the CIO, in collaboration with the Executive Director of the Office of Program Accountability and Risk Management (PARM), update or develop new guidance on Agile methodologies to describe how Agile teams can estimate the relative complexity of user stories.
- Ensure that the CIO, upon establishing a set of core performance metrics, sets expectations for automated testing and code quality, and tracks and monitors against those expectations.
DHS responded that the first three recommendations have already been met, and GAO is currently assessing these actions. For the remaining seven, the department described actions that it plans to take. For example, DHS stated that it will use the results of its Agile core metrics and Agile Software Delivery Maturity Model to measure the success of the transition to Agile and its impact on the department. According to the department, it expects this action to be completed by June 30, 2021. Further, DHS stated that it will identify Agile training requirements for staff in Agile programs, and will use that to establish Agile training requirements for staff outside of the acquisition workforce but assigned to Agile programs. DHS stated that the DHS OCIO will gather requirements from components via its IT workforce planning integrated project team to identify training resources available across the department that also address the skill sets needed for Agile programs. The department added that the DHS OCIO will utilize information from the April 2019 white paper, titled “OCIO Agile White Paper” to inform proposed Agile program training requirements. The department estimated that these actions are to be completed by September 30, 2020.
