To test security screening at U.S. airports, the Transportation Security Administration (TSA) regularly tries to sneak guns and simulated bombs through checkpoints or in checked baggage.
The Government Accountability Office (GAO) observed 26 covert tests and reviewed the test program and how results are used, publicly releasing its findings on April 4. Overall, GAO found that TSA’s ability to run covert tests has improved, but a new process intended to address vulnerabilities found in testing hasn’t fully worked.
In 2015, TSA identified deficiencies in its covert testing process, and in 2017, the Department of Homeland Security Office of Inspector General’s covert testing identified deficiencies in screener performance. Since these findings, TSA has taken steps intended to improve its covert test processes and to use test results to better address vulnerabilities.
Of the two TSA offices that conduct covert tests, GAO found Inspection officials used TSA’s risk assessment to guide their efforts. However, Security Operations officials relied largely on their professional judgment in making decisions about what scenarios to consider for covert testing. By not using a risk-informed approach, GAO said TSA has limited assurance that Security Operations is targeting the most likely threats.
Both Inspection and Security Operations have implemented processes to ensure that their covert tests produce quality results. However, GAO found that only Inspection has established a new process that has resulted in quality test results. Specifically, for the two reports Inspection completed for testing conducted in fiscal years 2016 and 2017 using its new process, GAO found that the results were generally consistent with quality analysis and reporting practices. On the other hand, Security Operations has not been able to ensure the quality of its covert test results, and GAO identified a number of factors that could be compromising the quality of these results. GAO said that unless TSA assesses the current practices used at airports to conduct tests, and identifies the factors that may be impacting the quality of covert testing conducted by TSA officials at airports, it will have limited assurance about the reliability of the test results it is using to address vulnerabilities.
In 2015, TSA established the Security Vulnerability Management Process to leverage agency-wide resources to address systemic vulnerabilities; however, this process has not yet resolved any identified security vulnerabilities. Since 2015, Inspection officials submitted nine security vulnerabilities identified through covert tests for mitigation, and as of September 2018, none had been formally resolved through this process. GAO found that in some cases, it took TSA officials overseeing the process up to 7 months to assign an office responsible to begin mitigation efforts. In part, this is because TSA has not established time frames and milestones for this process or established procedures to ensure milestones are met, in accordance with best practices for program management.
GAO has made nine recommendations to TSA:
- Document its rationale for key decisions related to its risk-informed approach for selecting covert test scenarios, for both the Security Operations’ and the Inspection’s testing process.
- Incorporate a more risk-informed approach into Security Operations’ process for selecting the covert test scenarios that are used for tests conducted by TSA officials at airports.
- Assess the current covert testing process used by TSA officials at airports—including factors that may affect the covertness and consistency of the tests—to identify opportunities to improve the quality of test data, and make changes as appropriate.
- Assess Security Operations guidance for applying root causes for test failures, and identify opportunities to clarify how they should be applied.
- Document the methodology for using the results of covert testing conducted by headquarters staff as a quality assurance process for covert testing conducted by TSA officials at airports.
- Establish timeframes and milestones for key steps in its Security Vulnerability Management Process that are appropriate for the level of effort required to mitigate identified vulnerabilities.
- Revise existing guidance for the Security Vulnerability Management Process to establish procedures for monitoring vulnerability owners’ progress against timeframes and milestones for vulnerability mitigation, including a defined process for escalating cases when milestones are not met.
- Develop processes for conducting and reporting to relevant stakeholders a comprehensive analysis of covert test results collected by TSA headquarters officials and TSA officials at airports to identify vulnerabilities in screener performance and common root causes contributing to screener test passes and failures.
- Develop a standard process for systematically documenting and disseminating to airport Federal Security Directors beneficial practices for conducting covert tests and using test results.
TSA concurred with the recommendations, all of which it estimates for completion by the end of the calendar year, with some, such as the first and second recommendations, estimated to be completed already by the end of May 2019.
