Congress’ Investigative Arm Finds Troubling Justification

WASHINGTON, DC, OCT. 17, 2006 - During the five years since 9/11, as fears of nuclear terrorism have steadily grown, the White House, Congress and Department of Homeland Security (DHS) all have managed to not only fumble legislation and technology for deterring and detecting radiological materials terrorists might try to bring into the country, but they’ve also inadequately secured domestic nuclear facilities and failed to prepare the nation’s emergency medical community to respond to a catastrophic nuclear bomb detonation, least of all explosion of a much less deadly dirty bomb.

In July 2006, DHS announced that it had awarded contracts to three vendors to further develop and purchase $1.2 billion worth of new portal monitors over five years. DHS plans to deploy these monitors at US ports of entry. For fiscal year 2007, the Domestic Nuclear Detection Office (DNDO) plans to acquire the first installment of 104 new portal monitors that use new identification technology at a cost of $80.2 million.

However, Congress curtailed DNDO’s ability to do so by restricting the availability of funding for full scale procurement of new radiation detection portal monitors until DHS certifies that a significant increase in operational effectiveness will be achieved.

It was because congressional budget staffers were given a "heads-up" on the GAO findings that a provision was added to the $35 billion FY07 homeland security spending bill President Bush signed into law.

According to a new report from Congress’ investigative branch, the Government Accountability Office (GAO), DHS’s cost-benefit analysis to support the purchase of new radiation detection portal monitors was not based on available performance data and did not fully evaluate all the monitors’ costs and benefits.

GAO’s audit could result in a delay in full-scale production funding for the contractors providing the monitors.

Congress' auditor of government performance said it found DHS’s analysis assumed the detection portals would have 95 percent accuracy in detecting highly enriched uranium, even though the performance tests conducted for DNDO showed the monitors could not meet that goal - the machines were able to detect highly enriched uranium that was “masked” with other substances only about half the time.

“Nothing could be more important than getting it right when it comes detecting nuclear materials coming into our country, yet the news from [DHS] keeps getting worse and worse,” said Sen. Chuck Schumer in an emailed statement following release of the GAO report Tuesday.

“Their incompetence would be laughable if the consequences weren’t so dire,” Schumer scolded. “They are ignoring their own test results and instead of trying to get it right, they are barging ahead spending $1.2 billion on radiation monitors that only work half of the time. Wherever these monitors are installed, instead of getting real protection, we will just get a false sense of security.”

Despite the problems cited by GAO, DNDO announced on Sept. 27 $3.2 million in grant awards for the deployment of radiation detection systems at interstate weigh stations throughout the southeast. The awards represent the first phase of DNDO’s Southeast Transportation Corridor Pilot (SETCP) program, a two-year initiative involving federal, state and local governments in nine states and the District of Columbia in the development of nuclear and radiological detection and interdiction capabilities on our nation’s highways.

After a competitive application process, initial grant awards will be provided to the Kentucky Transportation Cabinet, South Carolina Department of Public Safety, South Carolina State Transport Police, Tennessee Emergency Management Agency, Virginia Department of Emergency Management, and Georgia Emergency Management Agency. Funds provided will support the deployment of fixed, hand-held and mobile radiation detection equipment at interstate weigh stations.

“We are intensely focused on preventing high consequence threats, such as a radiological or nuclear attack,” said Vayl S. Oxford, DNDO Director. “The southeast transportation corridor sees some of the largest concentrations of truck traffic in the country. The work that we are doing in the southeast will ultimately lead to a web of radiation detection systems on our nation’s highways.”

The SETCP program will integrate existing and new systems and develop a regional threat detection and interdiction architecture. DNDO will also provide training for state and local partners on equipment operations, alarm resolution protocols, information sharing, and the shipment of radioactive materials from nearby nuclear production and waste treatment facilities.

Several weeks earlier, DDNO announced the award of the Cargo Advanced Automated Radiography System (CAARS) program contracts totaling an estimated $1.35 billion to SAIC of San Diego, Calif., American Science & Engineering of Billerica, Mass., and L-3 Communications of Woburn, Mass. The CAARS award, which comes following a fully competitive acquisition, will complement the nation’s radiological and nuclear detection capabilities.

“CAARS is the latest effort in the department’s broader strategy to enhance our nation’s security by harnessing cutting-edge, automated technologies to better detect radiological and nuclear threats at our nation’s points of entry,” said Oxford. “This advanced equipment will greatly enhance our ability to detect these greater threats while also improving our ability to scan greater portions of cargo in a timely, effective manner.”

The CAARS program will deliver an advanced imaging system that will automatically detect high density shielding that could be used to hide special nuclear material such as highly enriched uranium or weapons grade plutonium. CAARS will improve throughput rates providing more effective scanning of a higher portion of cargo at the nation’s points of entry.

“Detection of special nuclear material is accomplished with both passive radiation portal monitors and active automated x-ray imaging,” said Oxford. “Deploying both Advanced Spectroscopic Portals (ASP) and CAARS systems to work in tandem provide the nation with the capability to detect unshielded, lightly shielded and shielded materials across the threat spectrum without slowing down commerce.”

But, GAO has just rained on DNDO’s parade. “To ensure that DHS’s substantial investment in radiation detection technology yields the greatest possible level of detection capability at the lowest possible cost, we recommended [in a March 2006 report] that once the costs and capabilities of these new portal monitors were well understood, and before any of the new equipment was purchased, the Secretary of DHS work with the Director of DNDO to analyze the benefits and costs of deploying new portal monitors. Further, we recommended that this analysis focus on determining whether any additional detection capability provided by the new portal monitors were worth the additional costs.”

In response to GAO’s recommendations, DNDO issued a cost-benefit analysis for the acquisition and deployment of new portal monitors in which it stated the purpose of its analysis was to help provide a “robust” defense against nuclear smuggling, to limit the negative impacts to legitimate trade and travel between nations, and to provide a sound financial investment for the United States government.

GAO concluded that “DNDO’s cost-benefit analysis does not provide a sound analytical basis for DNDO’s decision to purchase and deploy new portal monitor technology. DNDO did not use the results of its own performance tests in its cost-benefit analysis and instead relied on assumptions of the new technology’s anticipated performance level.”

GAO stated “performance tests also showed that the ability of new radiation detection portal monitors to correctly identify masked HEU (placed next to or within another, usually more benign, radiological substance) was even more limited.8 According to the cost-benefit analysis and radiation detection experts to whom we spoke, masked HEU is a significant concern because it is difficult to detect. DNDO also focused the analysis exclusively on identifying HEU and did not consider in the analysis how well (either as a goal or in testing) new portal monitor technology can correctly detect or identify other dangerous radiological or nuclear materials. Furthermore, the analysis did not include the results from side-by-side tests that DNDO conducted of the advanced portal monitors and current portal monitors.”

Continuing, GAO determined that “the cost-benefit analysis for acquiring and deploying portal monitors is also incomplete because it does not include all of the major costs and benefits required by DHS guidelines. In particular, DNDO did not assess the likelihood that radiation detection equipment would either misidentify or fail to detect nuclear or radiological material. Rather, it focused its analysis on reducing the time necessary to screen traffic at border check points and reduce the impact of any delays on commerce. DNDO also used questionable assumptions about the procurement costs of portal monitor technology. DNDO assumed a purchase price for current portal monitor technology that is more than twice what CBP typically pays.”

GAO found that DHS’s “performance tests of [advanced spectroscopic portal Monitors (ASP)] showed that they did not meet DNDO’s main performance assumption in the cost-benefit analysis of correctly identifying HEU 95 percent of the time it passes through portal monitors. The 95 percent performance assumption included ASPs’ ability to both detect bare, or unmasked, HEU in a container and HEU masked in a container with a more benign radiological material.”

Based on an assessment of the performance data by the National Institute of Standards and Technology (NIST), the ASP prototypes manufactured by the three companies that won DNDO’s recent ASP procurement contract tested at the Nevada Test Site (NTS) identified bare HEU only 70 to 88 percent of time. Performance tests also showed that ASPs’ ability to identify masked HEU fell far short of meeting the 95 percent level of performance assumed in the cost-benefit analysis.

According to DNDO’s cost-benefit analysis, identifying masked HEU is the most difficult case to address. DOE officials told GAO benign radiological materials could be used to hide the presence of HEU.

NIST reported that the best ASP prototype DNDO tested last year in Nevada that won a procurement contract was able to correctly identify masked HEU and depleted uranium (DU) only 53 percent of the time. Similarly, the ASP prototypes submitted by the other two companies that won DNDO’s ASP procurement were able to identify masked HEU and DU only 45 percent and 17 percent of the time.

“Despite these results, DNDO did not use the information from these tests in its cost benefit analysis,” GAO said, adding, “instead, DNDO officials told us that since new portal monitors cannot meet the 95 percent level of performance, they relied on the assumption that they will reach that level of performance sometime in the future. DNDO officials asserted that the current performance levels of the ASPs will improve, but they provided no additional information as to how the 95 percent performance goal will be achieved or an estimate of when the technology will attain this level.”

“Moreover, DNDO’s cost-benefit analysis only considered the benefits of ASPs’ ability to detect and identify HEU and did not consider ASPs ability to detect and identify other nuclear and radiological materials,” GAO stated. “The ability of an ASP to identify specific nuclear or radiological materials depends on whether the ASP contains software that is specific to those materials. In our view, a complete cost-benefit analysis would include an assessment of ASPs’ ability to detect and identify a variety of nuclear and radiological material, not just HEU. By excluding radiological and nuclear materials other than HEU, DNDO’s analysis did not consider the number of secondary inspections that would be related to these materials and hence it likely underestimated the costs of ASP use.”

Although in recent congressional testimony DNDO’s director has cited the primary benefit of deploying ASP monitors as reducing unnecessary secondary inspections, GAO found “DNDO did not assess the likelihood that radiation detection equipment would either misidentify or fail to detect nuclear and radiological materials. Rather, DNDO’s cost-benefit analysis focuses on the ability of ASPs to reduce false alarms — alarms that indicate nuclear or radiological material is in a container when, in fact, the material is actually non-threatening, such as ceramic tile. Reducing false alarms would reduce the number of secondary inspections of non-threatening nuclear and radiological materials and therefore the costs of those inspections.”

There are many more damning findings in GAO’s report. DHS neither agreed nor disagreed with GAO’s recommendation, “but continues to stand behind the basic conclusions of its cost-benefit analysis that the ASP program is a sound investment for the US Government,” GAO said, adding, “in our view, the cost-benefit analysis cannot be used as a reliable basis for making a major procurement decision.”

DNDO's Oxford said in statement Tuesday DHS plans to comply with GAO's recommended certification methodology so that production can go forward.

Click here for the main Kimery Report page.