The Transportation Security Administration (TSA) relies on technologies like imaging systems and explosives detection systems to screen passengers and baggage to prevent prohibited items from getting on board commercial aircraft.
In its fiscal year 2021 budget request, TSA reported that there are approximately 15,000 units of screening equipment operating in or in the process of being deployed to airports across the nation.
As part of its process of acquiring these systems and deploying them to airports, TSA tests the systems to ensure they meet requirements. TSA does most of this testing itself, but to make the process more efficient and get new technology into airports faster, TSA allows companies to have testing undertaken by other testers in certain situations.
But a new report from the Government Accountability Office (GAO) says companies have used this option just five times since 2013.
The third party testing concept means that screening system vendors take this additional step either prior to submitting their technologies to TSA or if their system failed TSA’s test and evaluation process. The goal is for third party testing to reduce the time and resources that TSA spends on its own testing.
In January 2020, TSA reported that approximately 51 percent of all systems failed its testing, which delayed its timeline for acquiring and fielding airport screening technologies. TSA noted that vendors’ systems would get stuck in a test, fix, and retest cycle that was time consuming and expensive for TSA, as its program offices pay for retesting. Also, according to TSA officials, the delays resulted in TSA expending its limited resources and tying up test facilities to remedy the systems that had failed.
In three of the five cases of third party testing uncovered by GAO, the vendors had failed TSA tests. One system took almost four years to complete operational testing, while a second system took over seven years. The third system never completed testing. In the two other cases, TSA supplemented its test capabilities by using third party testers to determine that systems installed at airports were working properly.
TSA officials and industry representatives pointed to several reasons for third party testing’s limited use since 2013, such as the cost to industry to use third party testers and TSA’s reluctance to date to accept third party test data as an alternative to its own. Despite this, TSA officials told GAO they hope to use third party testing more in the future. For example, in recent announcements to evaluate and qualify new screening systems, TSA stated that it will require a system that fails TSA testing to go to a third party tester to address the identified issues.
TSA reported to Congress earlier this year that third party testing is a part of its efforts to increase supplier diversity and innovation. However, GAO found that TSA has not established metrics to determine third party testing’s contribution toward the goal of increasing efficiency. Further, GAO found no link between third party testing and supplier diversity and innovation. Some TSA officials and industry representatives also questioned third party testing’s relevance to these efforts.
Beginning in September 2019, TSA expanded the use of third party testing to include site acceptance testing for its Capabilities Acceptance Process. This process allows donors—such as airports, airlines, or industry stakeholders—to purchase TSA-qualified security screening equipment and donate it to TSA. As a part of this process, donors are responsible for the procurement of equipment, installation, acceptance testing, and mitigation of any acceptance test failures.
Donors use a TSA-approved contractor to conduct these activities in coordination with authorized TSA representatives. Once the equipment is installed at the airport, the contractor conducts acceptance testing in accordance with the TSA-approved test plans, thereby supplementing TSA’s test and evaluation capabilities. The contractor provides the test results to TSA officials, who review and analyze the data to determine if the system passes or fails.
As of April 2020, third party testing under the Capabilities Acceptance Process was being used for site acceptance testing of the automated screening lanes technology at the San Francisco, California and LaGuardia, New York airports. The first use of this process started in the San Francisco Airport in December 2019. In a memorandum of understanding with TSA, the airport commission agreed to contract with a TSA-approved third party tester to provide testing support for its donation. TSA officials explained that in this instance, the third party test data were accepted in place of TSA test data.
TSA officials acknowledged that the anticipated benefits of third party testing have not been realized, in part due to the lack of incentives for vendors to participate prior to the test and evaluation process. TSA tests are free to vendors, while third party testing costs are borne exclusively by the vendors of the systems, which can be particularly discouraging for small businesses. Industry representatives agreed that it may be difficult for small businesses to proactively use a third party tester because of the expense. Third party testers told GAO that costs varied widely depending on the scope of testing activities and provided estimates that ranged from tens of thousands to hundreds of thousands of dollars.
In addition, vendors said they may be more incentivized to invest in third party testing in advance of submitting technologies to TSA if they had assurances that third party test data would be accepted and thus reduce the amount of TSA testing. According to TSA officials and industry representatives, after third party testing was complete, TSA testing officials did not accept the results from the third party tester in lieu of further TSA testing. Instead, TSA fully retested the systems. This retesting can help address any potential concerns with national security, conflicts of interest, or fraudulent activities related to third party testing; it can also create a disincentive for vendors to proactively use third party testing.
In April 2019, TSA announced the establishment of next generation requirements for explosives detection systems. This calls for potential vendors to consider using third party testing prior to submitting their system to TSA to determine if it is ready. If the system experiences more than one failure in TSA testing, the vendor must use a third party tester, at the vendor’s expense, before TSA will continue testing or the system will be removed from the process. Failure is defined as at least one significant failure, an unacceptable cluster of nonsignificant failures, or a TSA determination that the system has a significant issue that will prevent it from passing testing. The vendor then has six months to demonstrate the failure has been resolved using third party testing. TSA has a similar requirement for advanced imaging technology as of January 2020.
If TSA is serious about its intention to use more third party testing, an overhaul is needed. To help it meet this goal, GAO recommends that TSA establish metrics to measure the effects of third party testing on the efficiency of the test and evaluation process. Once these metrics have been established, TSA should assess gains in efficiency resulting from third party testing, and assess whether third party testing contributes to its goals of increasing supplier diversity and innovation.
TSA concurred and indicated it expects to complete actions to implement them by August 2021. The results from the latter recommendation will be especially crucial to small businesses as third party testing may, in some cases, discourage their participation as testing can be costly.