It’s 7:45 on a Wednesday morning in May at Hartsfield-Jackson Atlanta International Airport and passengers are boarding Delta Air Lines flight 334 to Mexico City. One by one the passengers scan their boarding passes and approach a camera that’s set up on a jetway where they have their pictures taken before they board the flight.
The photos are being matched through biometric facial recognition technology to photos that were previously taken of the passengers for their passports, visas, or other government documentation. All is moving smoothly until the U.S. Customs and Border Protection officers assisting the passengers are alerted that they need to check one of the travelers.
It’s a 28-year old woman, a Mexican national with a Mexican passport. The biometric system alerted the officers because when preflight information was gathered on the woman, no historical photos to match against her could be found.
A CBP officer took the woman aside and looked at her passport. No visa was attached and the woman didn’t have a green card to prove she was a lawful permanent resident. Upon further questioning, the woman admitted that four years ago, she had come into the country illegally.
Using a specially designed, CBP biometric mobile device, the officer took fingerprints of the woman’s two index fingers. “This was the first time that we had captured this individual’s biometrics, her unique physical traits,” said Bianca Frazier, a CBP enforcement officer at the Atlanta Airport. “We didn’t have her biometrics because we had never encountered her before.”
As early as 2002, shortly after the worst terrorist attack in U.S. history, legislation was passed requiring the Department of State and the Department of Homeland Security to use biometric technology to issue visas and screen non-U.S. citizens entering the U.S. Then in 2004, more legislation was passed, authorizing DHS to collect biometric data from non-U.S. citizens exiting the country.
According to Frazier, finding people who have entered the country illegally is common. Since June 2016, when CBP and Delta Air Lines launched a pilot program to test CBP’s biometric facial recognition exit technology, passengers like the young Mexican woman have been found daily. “She was typical of the people who have entered without inspection,” said Frazier. “Most days we find a minimum of two or three undocumented people, but sometimes we find as many as eight to 10 boarding a flight.”
Ultimately, the woman was allowed to board the flight, but when Frazier used CBP’s mobile device to take her fingerprints, it created a fingerprint identification number that is specifically tied to the woman. In the future, if she applies for a visa to return to the U.S. or is encountered crossing the border illegally, an alert will be triggered, indicating that the woman had previously entered the U.S. illegally and is on a lookout list. Additionally, when Frazier processed the traveler, the device automatically created a biometric exit record confirming that the woman left the country.
For more than a decade, the U.S. government has been struggling to find a way to develop a practical and cost effective biometric entry/exit system that fulfills a congressional mandate to keep America safe. CBP has partnered with the U.S. air travel industry to meet that goal and is implementing innovative ways of using biometric technology to provide better enforcement and a better experience for travelers.
By 2013, when CBP assumed responsibility for designing and implementing a system that could biometrically track travelers exiting the U.S., the government had been wrestling with the challenge for years. Technology was part of the problem, but how to integrate that technology into the existing infrastructure at airports without driving up costs and negatively impacting airport and airline operations was a conundrum.
CBP had been working with the airlines to verify travelers entering and exiting the country since the mid-1990s, using travelers’ biographic information— date of birth, passport number, document number, country of citizenship, etc. “The airlines sent us the manifest information in advance of the flight’s departure,” said John Wagner, deputy executive assistant commissioner of CBP’s Office of Field Operations. “We did law enforcement work based on that data.”
But then, after September 11, biographic information wasn’t enough. To increase security, Congress passed legislation that added biometric requirements for tracking travelers. “Inbound passengers were easier to track because we already had a process,” said Wagner. “When travelers come off of an international flight, they are funneled through a secure pathway to the CBP inspection area. The airline transmits the biographic data to us. We verify that information when we read a traveler’s passport and we make sure it’s accurate. That’s when we also collect fingerprints from most non-U.S. citizens.”
With outbound flights, collecting passengers’ biometrics is much more difficult. “We’ve never constrained departures to be able to do that,” said Wagner. “We don’t have specific departure areas for outbound flights. International flights depart from all over the airport, so it was difficult to figure out where we could collect biometrics and what technology we would use.”
Added to that, CBP lacked support. “The travel industry stakeholders were vehemently opposed to any of this because they thought it would cost money and it would slow people down,” said Wagner. The challenges seemed insurmountable. “We were focused on where is the magic technology that is going to make this work and address all of these concerns. No one had been able to find it because it didn’t exist,” he said.
Wagner and his team took a fresh start. They reached out to the DHS Science and Technology Directorate, the department’s research and development arm, to learn more about the biometric technology that was available and which methods of collection would work best. Shortly thereafter, in 2014, a demonstration test lab was set up in Landover, Maryland. “One of the things we learned from previous pilots in airports is that airports are chaotic places. It’s hard to do a really good controlled test when anything can go wrong and you don’t know why. Was it because there were lots of delays? Were there weather incidents? Or did people miss their flights? Any number of factors could affect the performance of the biometric system, so we set up a test space where we could carefully control different variables to see how well our biometric concepts worked,” said Arun Vemury, director of the DHS Science and Technology Directorate’s Apex Air Entry/Exit Re-engineering and Port of Entry People Screening programs.
“We evaluated more than 150 different biometric devices and algorithms. We put them together in different configurations and then brought in test volunteers to actually run through the process to figure out how long it took, what kind of throughput we were able to get, how well the biometrics matched, and what their performance ultimately was,” said Vemury. “Over time, we brought in more than 2,000 people from 53 different countries of origin, who varied in age from 18-85. We were trying to mimic the demographics of travelers coming to the U.S.”
One of the things that Vemury learned was that the algorithms used in facial recognition technology have become much more advanced. The algorithm is the formula that identifies the unique biometric features in a finger, iris, or face and then compares those points to corresponding areas in previously collected biometrics. “Because of the improvements in facial recognition technology, we can verify people’s identities with facial recognition much more effectively today than we could even just two years ago,” said Vemury.
After nearly two years of rigorous testing and evaluation, DHS Science and Technology gave its findings to CBP in December 2015. “We turned over all of our test reports, economic analyses, quantitative analyses, concepts of operation, and staffing estimates,” said Vemury. “The last thing we wanted was to have any unanswered questions. We knew we needed a biometric process that would work.”
Concurrently, CBP was doing its own laboratory tests and conducted a series of pilots. “We ran several pilots to help us learn about the different types of biometric technology in the different environments where we work,” said Wagner. For example, CBP was aware that U.S. passports were vulnerable to fraud and thought a biometric tool could help. After months of testing algorithms and cameras, CBP developed a one-to-one facial recognition technology that compared travelers against their passport photos. The pilot, which was tested on inbound flights, initially ran for two months, from March to May 2015, at Washington Dulles International Airport in Dulles, Virginia. At that point, more lab testing and analysis were done to improve the algorithm, and then a second pilot, which continues today, was set-up at Dulles and John F. Kennedy International Airport in New York City.
“The pilots showed us that the facial recognition technology was accurate,” said Wagner. “We grew confident that the algorithms were good enough to use and rely on.”
One of the many examples that illustrates this occurred at JFK in May 2016, when a traveler with a U.S. passport arrived on a flight from Accra, Ghana, and presented herself as a returning U.S. citizen. All of her biographical information was processed successfully, but the CBP officer who interviewed the woman had a suspicion she might be an imposter. The officer referred the traveler to a booth equipped with the facial recognition technology where her photo was taken and compared to the photo in her passport. The match score was very low and she was referred for further inspection.
The woman was fingerprinted and the officers confirmed her true identity, uncovering that she was an imposter. In actuality, the woman was a Liberian citizen who had been denied a diversity visa from a green card lottery in 2015. She admitted that she found the U.S. passport in a marketplace and didn’t know the true owner. The woman was then turned over to U.S. Immigration and Customs Enforcement authorities and sent to a detention center to await a credible fear hearing to determine whether she would be able to seek asylum. Without the suspicions of an astute officer and CBP’s biometric technology, the woman could have entered the country through fraudulent means.
In another pilot at the land border, in Otay Mesa, California, CBP tested face and iris scans to biometrically record the entry and exit of pedestrians. “From these tests, we learned a lot about how travelers react to various biometric technologies,” said Wagner.
CBP also built a handheld, mobile device that allowed officers to run fingerprints on departing travelers. “We tested the Biometric Exit Mobile in 2015 at 10 airports around the country,” said Wagner. “It showed us we could accurately take fingerprints from a mobile device and gave our officers the capability to do law enforcement and biometric queries on a smart phone if they saw that an individual requires further investigation.”
Biometric success story
As a law enforcement tool, the Biometric Exit Mobile has produced stunning results. Case in point is an incident that occurred in May at Chicago O’Hare International Airport involving a Polish national couple who were boarding a flight to Berlin, Germany. When the couple presented their passports at the departure gate, the CBP officers didn’t find any U.S. visas or country entry stamps, so they decided to run a check and swiped the couple’s passports. The biographical information didn’t reveal anything derogatory, but as a precautionary check, the officers used the Biometric Exit Mobile device to take the couple’s fingerprints. The officers took the index prints of the woman first and within seconds, she came back as a watchlist hit. The same occurred with the man. Both had been ordered deported by an immigration judge, but they didn’t leave the country.
The officers wanted to clarify what they discovered, so they reached out to a colleague. “I pulled up the woman’s name and nothing came up. There was no record on her whatsoever,” said Jonathan Cichy, a CBP enforcement officer who works outbound operations at O’Hare Airport. “However, when I checked her fingerprints, there was a hit, but for a woman with a different date of birth and a different identity, which she had been arrested and deported under.”
Then Cichy looked at the manifest for the flight. “I saw they weren’t on it. There was no record of the identities they were using to get on the plane,” he said. After checking further, Cichy found that both of the Polish nationals had criminal histories with multiple identities. “But none that came up in our systems because they weren’t leaving under any of those identities. Biographics alone did not tell us the full story,” said Cichy, who quickly rushed to meet the flight that was leaving in 20 minutes.
The couple was allowed to board the flight, but not until Cichy had served them with legal papers to verify their departure and close out the deportation case. “If either one of them is found attempting to return to the U.S. without permission, they could be prosecuted for reentry after deportation, a felony that carries a sentence of two to 20 years,” said Cichy.
CBP’s biometric exit tests culminated in June 2016 with a pilot program at the Atlanta Airport. Wagner and his team had a breakthrough. All the work they had done for the past several years was finally coming to fruition. “We came up with a way of taking the information we receive about passengers from the airlines and matching it against information we already have in our government databases,” said Wagner.
Based on their research, Wagner and his team decided to use facial recognition technology. “We found that facial recognition was intuitive for people. Everybody knows how to stand in front of a camera and have his or her picture taken. Not so with iris scans and fingerprints. Every time a traveler does the process wrong, someone has to instruct him or her the right way to do it,” said Wagner.
Aside from being quicker than other biometric methods, facial recognition has additional pluses. The physical design of the camera doesn’t take up much space, and the equipment isn’t costly. Furthermore, CBP already has a collection of photos for biometric comparison. “People have already provided their photographs to the government for travel purposes,” said Wagner.
But the real feat was when CBP found a way to speed up the photo matching process. “As soon as a passenger checks in with the airline, the airline tells us who is getting on the plane. At that point, we find all the photographs we have of the people on the flight and we pool them, and then segment them into individual photo galleries for each passenger,” said Wagner. “If there are 300 people on the flight, we find every photograph we have of those 300 people. Generally, that means we will have about 1,500 pictures because we have multiple photos of each passenger.”
Then, as the passenger boards the flight, he or she has his or her picture taken. That photo is compared to his or her individual gallery of photos rather than comparing it to a billion photos that are in DHS’s biometric database. “The matching is done in real-time because it’s a small file and it’s accurate,” said Wagner.
The Atlanta pilot also was designed with certain parameters. “We did not want to add another layer onto the travel process,” said Wagner. “We told our stakeholders, ‘We want to design something that fits within your existing operations and infrastructure. We’re trying to make things easier for travelers. We don’t want to add additional steps or processes.’”
In a discussion with Delta Air Lines, Wagner asked if the airline would be interested in participating in a biometric pilot. “We have a very strong, long-standing, collaborative relationship with CBP,” said Jason Hausner, Delta Air Lines’ director of passenger facilitation. “Normally, when they approach us to do something, we’re in. We like to be in on the front end to provide our expertise and help shape things.”
Delta also had a long range vision of using biometrics for its own operational purposes. “When we heard the proposal from CBP to test biometric exit technology, it resonated with us because one of the elements we were looking at is biometric boarding,” said Hausner.
In February 2016, Delta met with CBP to develop a project plan and decided to test a flight from Atlanta to Tokyo, Japan. The pilot, which began in June, was successful, so by September, CBP decided to test another flight. This time the flight was to Mexico City. “As a further test to the technology, we chose a flight with a different demographic to ensure the matching capability was still successful,” said Kevin Pfeifer, CBP’s assistant port director of tactical operations at the port of Atlanta.
After more than a year of testing, the facial recognition technology has consistently shown a high rate of accuracy. “Our percent of successful matches is in the high 90s. It’s even moved up a notch in terms of quality and accuracy,” said Nael Samha, CBP’s director of passenger systems who built the architecture for the pilot’s operating system.
Operationally, the pilot has performed well too. “One of the things we wanted to evaluate was the impact on our operations. Would it delay boarding? Would it impact our on-time performance? We’re very metrics oriented,” said Hausner. “So far, this test has not impacted us in any manner, and part of it is because of the approach that CBP has taken. They know that in order for their program to be successful, they need to partner with us.”
During the summer of 2017, CBP conducted technical demonstrations of the biometric exit facial recognition technology with various airlines and airports throughout the country. “We wanted to show stakeholders and the public what this technology is, how it works, and explore how biometric exit technology can fit into airline and airport business models and modernization plans while addressing privacy requirements,” said Wagner.
Some airlines are already making headway. At JFK and in Atlanta, Delta is testing ways to combine the facial recognition technology with its boarding pass procedures. “The CBP pilot is a two-step process by design, but it seemed to us that when this is implemented across the country, it should be a one-step process,” said Hausner.
In June, JetBlue Airways transformed this goal into a reality and was the first airline to board passengers using biometric facial recognition instead of boarding passes. Unlike the technical demonstrations that CBP was conducting with other carriers, JetBlue proposed the pilot. The airline wanted to design its own technology and incorporate it with CBP’s facial recognition matching system. “CBP was very open-minded with what we wanted to accomplish,” said Liliana Petrova, JetBlue Airways’ director of customer experience. “They flew out to Boston and spent several hours with us and took the time to listen. We wanted them to know exactly how we wanted to integrate the biometric technology with the experience at our gate.”
The pilot, which was tested at Logan International Airport in Boston, was assembled very quickly. “CBP gave it priority and helped us do a very fast buildout,” said Petrova. “Not many partnerships, even private partnerships, function as smoothly.”
According to Petrova, the biometric system is part of JetBlue’s strategy to remove the hassle from the traveling experience. “Passengers don’t have to stop, look for their boarding passes or their IDs. The line moves faster and they don’t have to wait as long,” she said. “We’re trying to take the anxiety out of flying and allow our crew members to interact more with customers.”
JetBlue’s customer feedback was positive. “The customers are really delighted by it. They think it’s cool and they’re having fun,” said Petrova. As a result, JetBlue has decided to expand the pilot in late 2017 with additional flights departing from Boston and JFK.
CBP’s future vision for biometric exit is to build the technology nationwide using cloud computing. “There are hundreds of airports throughout the U.S. where we provide services for international travelers and we still need to work through the deployment schedule and timeline,” said Wagner. “We also need to determine the technology we’ll use. We’ve been working with airports and airlines to arrive at some of those answers. We want them to tell us what the equipment should look like, so that it fits in with their operational needs.”
Plans are also underway to update CBP’s biometric inbound technology. “We’ll be using the same system for our arrivals processing as we do for biometric exit,” Wagner explained.
But that’s not all that CBP has in store. “We’re also looking at communicating with people on their mobile devices as they deplane,” said Wagner. “If we can give travelers better guidance on how to navigate customs and the maze at the airport, we can increase efficiency and give them peace of mind.”