The Department of Homeland Security (DHS) is required by law to report annually on 43 specific measures of border security effectiveness.
The Government Accountability Office (GAO) regularly assesses the quality of its reports and recently stated that DHS reported on more of the metrics—37 out of the 43—in its 2019 report than it has previously. GAO found that DHS also disclosed more data limitations than before—such as important context about how certain data is collected—to help Congress and the public better understand the report.
However, the watchdog reminded DHS of its outstanding recommendations to improve the report’s quality, which include improving its process of ensuring data in the report is reliable and disclosing all data limitations.
As part of GAO’s review, the office found that DHS used a statistical model to estimate four metrics, including the estimated number of undetected unlawful entries between ports of entry. In response to one of GAO’s prior recommendations, DHS, in its fiscal year 2019 Border Security Metrics Report, showed how the sensitivity of three assumptions affected the precision of the model’s estimates. For example, the model assumed that DHS apprehends all families unlawfully crossing the border, and the fiscal year 2019 report outlined how this assumption affected the model-based metrics in 2018. GAO said this sensitivity analysis will allow Congress and the public to better understand the limitations of DHS’s model and better evaluate the validity of border security metrics derived from it.
In response to another GAO recommendation, DHS conveyed statistical uncertainty for one of the unlawful entry metrics in its fiscal year 2019 report. In particular, DHS outlined how this uncertainty might affect the model-based apprehension rate. However, GAO found that DHS did not report how uncertainty would affect the other three metrics that rely on its statistical model.
DHS reported on two new metrics in its fiscal year 2019 Border Security Metrics Report: potentially high-risk cargo containers scanned upon arrival at a U.S. port of entry and potentially high-risk cargo containers scanned before arrival at a U.S. port of entry. DHS is required to report on the number of potentially high-risk containers scanned. For these metrics, DHS instead provided the number of potentially high-risk cargo containers that CBP reviewed, assessed, or scanned upon arrival and before arrival, respectively, at U.S. ports of entry. DHS explained that CBP data systems cannot distinguish between cargo that are reviewed, scanned, or assessed.
In March 2019, GAO reported that Border Patrol contracted with the Johns Hopkins University’s Applied Physics Laboratory to undertake a project that aims to use a combination of statistical modeling and data from sensors along the border to estimate the total number of unlawful border entries between land ports of entry, including entries both detected by Border Patrol and those not detected by Border Patrol. A simulation-based estimate is designed to rely on fewer assumptions about the types of individuals who unlawfully cross the border as compared with the statistical model. In February 2021, Border Patrol officials stated they could use the model to assess performance and as a planning capability for DHS, if the data are found to be valid. Border Patrol has made progress developing this model since GAO’s March 2019 report, but it is too early to tell whether DHS will use the simulation-based model to address limitations associated with the model-based apprehension rate. DHS noted in the fiscal year 2019 Border Security Metrics Report that Border Patrol completed the pilot analysis for all stations along the southwest border and is beginning to conduct a sensitivity analysis with the data and incorporate additional operational elements into the model. In July 2021, Border Patrol officials stated they are planning to expand the modeling and simulation capability to the northern border and complete a validation effort for the total flow estimate by the end of fiscal year 2022.