I have to laugh to myself when I read treatises on “big data” and how we live in a world newly dominated by it. Big data has existed since the earliest humans rendered pictures on the inside of a cave wall. Flash forward 15,000 plus years – anyone who managed a set of government contract files in a Lektriever in the 1970s was dealing with big data. However, for most of human history, information existed in physical paper and other analog forms (stone, papyrus, tape) and was largely inaccessible.
Things are much different today… driven by two consequential changes. First, there was the advent of computers in the 1930s which sparked a digital revolution (unbeknownst to us at the time). Information started taking on numeric forms which were easier to filter, parse and sort. In more recent times, even the most difficult forms of information – like video and audio – have taken to digitization. Second, the birth of the commercial Internet created a platform for ubiquitous access and exponential creation of new forms of commerce and social media. The result is unprecedented access to information – in “open source.” I recall hearing how we’ve doubled the amount of data in the history of humanity in just the last three years!
The real challenge now is, what can we do with all this data to improve our homeland and national security posture? We often see the potential of the information as we discover missed clues after significant events. However, through newer technology advances, we now have an opportunity to shift our view from the rearview mirror to a forward-looking view. On the whole, I think we want to aim to evolve more consistently from forensics and post-tragedy investigations to foresight.
Complicating the situation is an iconic change in the world order. We now have a vast number of transnational players operating independently on the world stage. The fall of the wall in 1989 eerily parallels the exponential rise of digital information and the broad adaptation of the commercial internet.
The good news is technology that can respond to all these factors is within our reach. We can harness streaming and near realtime data to create information fast enough to indicate what might be on the horizon. Such platforms are capable of fusing traditional law enforcement and intelligence sources with new open sources. The right assembly of tools allows automated analysis to relieve the analyst’s need to keep a 24×7 bird’s eye view when exploiting large volumes of information.
Dynamic machine-learning algorithms can help to uncover hidden patterns and associations. Graph databases can provide the ability to query and explore the networks of entities and their relationships in order to understand how the adversary operates. Tools for geospatial analysis can help analysts understand how events and groupings are related in space and time.
Read the complete report in the Feb/March 2016 issue of Homeland Security Today.
