Inevitably, airport screening technologies can become less effective over time, making regular performance testing crucial. However, a new report from the Government Accountability Office (GAO) says the Transportation Security Administration (TSA) does not continue to fully test screening equipment after it has been installed.
The TSA Modernization Act includes a provision for GAO to review TSA’s deployment of detection technologies and their standards. Detection standards specify the prohibited items (e.g., guns, explosives) that technologies are to detect, the minimum rate of detection, and the maximum rate at which technologies incorrectly flag an item. TSA operationalizes these standards by adapting them as detection requirements, working with manufacturers to develop and test new technologies (software or hardware), and acquiring and deploying technologies to airports. For the standards GAO reviewed, this process took between two and seven years, based on manufacturers’ technical abilities and other factors.
GAO found that TSA’s deployment decisions are generally based on logistical factors and it is “unclear how risk is considered when determining where and in what order technologies are deployed because TSA did not document its decisions”.
While TSA considers risks across the civil aviation system when making acquisition decisions, the review found it did not document the extent risk played in deployment, and could not fully explain how risk analyses contributed to those decisions.
Is TSA Monitoring Technology Degradation?
TSA certifies technologies to ensure they meet requirements before deployment, of course, and screeners are required to regularly calibrate deployed technologies to demonstrate they are minimally operational. However, neither process ensures that technologies continue to meet requirements after deployment.
In 2015 and 2016, the Department of Homeland Security (DHS) tested a sample of deployed explosives trace detection and bottled liquid scanner units and found that some no longer met detection requirements. Despite these findings, TSA still lacks an effective process to ensure technologies continue to meet detection requirements after deployment.
Daily calibration helps to ensure that screening technologies continue to meet detection requirements after deployment. However, calibration demonstrates that the screening technology is minimally operational and is not designed to test whether the screening technology meets detection requirements. For example, each explosives detection system is calibrated with an operational test kit that contains items of various densities. To calibrate explosives detection systems, an officer must run the operational test kit through the unit and verify that the item is correctly displayed on the monitor. This process demonstrates whether the system can identify the known items’ densities, but it does not ensure that the system meets detection requirements.
During GAO’s review, TSA officials stated that they plan to develop a process to review screening technologies on an annual basis to analyze their performance, including detection over time. TSA officials stated that they were actively working on developing a review process for the explosives detection system but did not have a date for when they planned to complete it. TSA officials for the passenger and carry-on screening technologies stated that they had not yet started developing a review process for those technologies and the timeline for developing a review process will depend on funding.
There are of course challenges in designing a process to ensure that screening technologies continue to meet detection requirements after deployment. Conducting live explosives testing in airports would be costly and extremely difficult, and moving large equipment to a test facility on a regular basis is not feasible.
Procuring, deploying and maintaining screening technology
GAO estimates that TSA spent $3.1 billion to purchase, deploy, install, and maintain its inventory of screening technologies as of the end of fiscal year 2018, based on agency estimates of costs. Of this $3.1 billion, GAO estimates that TSA spent 71 percent to purchase screening technologies, 9 percent to deploy, about 12 percent to install, and, for fiscal year 2018, about 9 percent to maintain them for one year.
The highest estimated total expenditures on a per-technology basis were for explosives detection systems ($2.1 billion, or 68 percent), advanced technology x-ray ($443 million, or 14 percent), explosives trace detection ($227 million, or 7 percent), and advanced imaging technology ($197 million, or 6 percent).
In fiscal year 2018, TSA determined that computed tomography is the best technology available to address rapidly evolving threats in the transportation sector, and plans to eventually deploy it to all checkpoints and replace advanced technology x-ray technology. TSA estimates additional life-cycle costs of $804 million to acquire, deploy, and maintain more than 800 computed tomography systems through fiscal year 2026. This cost estimate could decrease however, as the per unit cost for CT machines falls.
As part of the wider review, TSA officials told GAO that they operate under the assumption that every airport is a possible entry point into the aviation system for a terrorist, and they do not consider there to be a significant difference in vulnerability among airports when deploying screening technologies. However, GAO said TSA officials did not provide analysis or documentation that supported this conclusion. Officials noted the exception to this assumption is a handful of airports that are consistently considered to be the highest risk because of known threats and a high volume of international travelers.
Further, officials said that if they had information about a threat to a specific airport that would be mitigated by deploying a screening technology, they would modify their plans for deployment accordingly. But GAO said TSA’s process for how it would change its deployment plans to specific airports based on risk lacks transparency. For example, officials said that as part of the acquisition process they have ongoing discussions with stakeholders about their deployment strategies, including security and intelligence officials who would inform them of any relevant risk information. Officials said these discussions are generally informal and not documented. It is therefore unclear how these discussions have incorporated information about risk in the past, and officials could not provide GAO with an example of when risk information at specific airports had directly influenced deployment of technologies to airports in the recent past.
Logistical factors are a primary factor in screening technology deployment. TSA uses modeling software to determine the most efficient number of units to allocate to an airport for each type of screening system. This analysis takes into account variables such as the number of flights at an airport, airport passenger volumes, items per passenger, and secondary search rates. Additionally, agency officials told GAO that the layout of an airport is a significant determining factor for the number of units it receives. For example, an airport that has centralized checked baggage screening areas will need fewer explosives detection systems than an airport that has checked baggage screening areas dispersed in different locations.
Future-proofing TSA’s screening systems
GAO made five recommendations, with which DHS has concurred. With regard to the first recommendation that TSA update guidance for developing and approving screening technology explosives detection standards, DHS stated that TSA has recently included updated guidance in its Requirements Engineering Integrated Process Manual.
DHS concurred with GAO’s second recommendation that TSA ensure that officials document key decisions supporting the development of screening technology explosives detection standards. DHS stated that the same updated Requirements Engineering Integrated Process Manual describes the process for documenting key decisions, including testing and analysis decisions.
DHS also concurred with the third recommendation that TSA document its assessments of risk and the rationale behind its decisions to deploy screening technologies. According to DHS, TSA has instituted an improved process for documenting elements that contribute to deployment decisions. DHS stated that TSA will continue to include a comparable level of documentation in future deployment plans for screening technologies.
GAO’s fourth and fifth recommendations – that TSA develops and implements a process to ensure that screening technologies continue to meet all detection requirements after deployment to commercial airports, were also agreed. DHS stated that TSA will develop recurring individual post implementation reviews (PIR) for all screening technologies, to assess multiple aspects of system performance, including detection over time. DHS also stated that TSA intends to examine the component performance of the detection chain rather than a direct measure of detection requirements, due to the limitations of using live explosives and simulants. DHS stated that because the detection chain for each technology is unique and will require individual reviews, TSA is developing a policy on the PIR development process, which it estimates will be completed by March 31, 2020. TSA plans to implement the review process on the first screening technology by December 31, 2020.
Shortly after 9/11, many airports were quick to acquire detection systems, but there was criticism from within the industry that in some cases these systems sat idle for too long before being installed and fully functional following operator training. Today, that is no longer an issue, but TSA must take care to ensure that these vital technologies continue to function fully and meet operational standards. It is not enough to install equipment and train users on technology that can degrade over time. There must be regular monitoring and preventative maintenance to ensure these life-saving and not inexpensive systems continue to do the job they were purchased for.
