The Department of Defense (DoD) Chief Digital and Artificial Intelligence Office (CDAO) launched the first of two AI Bias Bounty exercises. Bias bounties are new crowdsourced efforts to help detect bias in AI systems. The CDAO is sponsoring two exercises: the first exercise is open to the public today, and a second exercise will soon follow.
The CDAO Responsible AI (RAI) Division is spearheading the two AI Bias Bounties, developed and executed through partnerships with ConductorAI-Bugcrowd and BiasBounty.AI and advised by the CDAO Defense Digital Services Directorate. The exercises are conducted to generate novel approaches to algorithmically auditing and red teaming AI models, facilitating experimentation with addressing identified risks, and ensuring the systems are unbiased, given their particular deployment context. “The RAI team is thrilled to lead these AI Bias Bounties, as we are strongly committed to ensuring that the Department’s AI-enabled systems – and the contexts in which they run – are safe, secure, reliable, and bias free,” remarked Dr. Matthew Johnson, Acting Chief of the DoD’s RAI Division.
The goal of the first bounty exercise is specifically to identify unknown areas of risk in Large Language Models (LLMs), beginning with open source chatbots, so this work can support the thoughtful mitigation and control of such risks. This exercise encourages public involvement (no coding experience necessary) to detect bias, and participants can earn monetary bounties based on scoring and evaluation by ConductorAI-Bugcrowd, funded by the DoD.
The DoD may consider the outcome of the bounty exercises as the basis for further research, analysis, best practices, and policy recommendations. Chief Digital and Artificial Intelligence Officer Craig Martell disclosed, “Given the Department’s current focus on risks associated with LLMs, the CDAO is actively monitoring this area; the outcome of the AI Bias Bounties could powerfully impact future DoD AI policies and adoption.”
The first (public) bounty will run January 29 – February 27, 2024. Please register here to participate in the first bounty: https://osdbiasbounty.com/. You may also contact [email protected] to learn more about this valuable program.
About the CDAO
The CDAO became operational in June 2022 and is dedicated to integrating and optimizing artificial intelligence capabilities across the DoD. The office is responsible for accelerating the DoD’s adoption of data, analytics, and AI, enabling the Department’s digital infrastructure and policy adoption to deliver scalable AI-driven solutions for enterprise and joint use cases, safeguarding the nation against current and emerging threats.
