Currently, threat actors possess limited access to the technology required to conduct disruptive operations against financial artificial intelligence (AI) systems and the risk of this targeting type remains low. However, there is a high risk of threat actors leveraging AI as part of disinformation campaigns to cause financial panic. As AI financial tools become more commonplace, adversarial methods to exploit these tools will also become more available, and operations targeting the financial industry will be increasingly likely in the future.
Financial entities increasingly rely on AI-enabled applications to streamline daily operations, assess client risk, and detect insider trading. However, researchers have demonstrated how exploiting vulnerabilities in certain AI models can adversely affect the final performance of a system. Cyber threat actors can potentially leverage these weaknesses for financial disruption or economic gain in the future.
Recent advances in adversarial AI research highlights the vulnerabilities in some AI techniques used by the financial sector. Data poisoning attacks, or manipulating a model’s training data, can affect the end performance of a system by leading the model to generate inaccurate outputs or assessments. Manipulating the data used to train a model can be particularly powerful if it remains undetected, since “finished” models are often trusted implicitly. It should be noted that adversarial AI research demonstrates how anomalies in a model do not necessarily point users toward a wrong answer, but redirect users away from the more correct output. Additionally some cases of compromise require threat actors to obtain a copy of the model itself, through reverse engineering or compromising the machine learning pipeline of the target.
