The Rapid Explanation, Analysis, and Sourcing Online (REASON) Program aims to develop technology that will enable intelligence analysts to substantially increase the quality of argumentation in their analytic reports through more effective use of evidence and reasoning. In the context of an analytic report, evidence is information that supports or opposes a judgment, while reasoning is the stated justification for the judgment. Furthermore, strong reasoning is reasoning that logically substantiates the judgments while weak reasoning is reasoning that either fails to substantiate the judgments or contains logical flaws. The technology developed by the REASON Program will automatically produce comments (feedback and recommendations) on a draft report, highlighting additional relevant evidence, and identify strengths and weaknesses in the draft’s reasoning. Analysts can use the comments to improve their reports.
Argumentation is central to the Intelligence Community (IC) Analytic Standards, which are listed in Intelligence Community Directive (ICD) 203. The standards are intended to guide IC analysis and analytic production. The Analytic Tradecraft Standards focus on several aspects of evidence and reasoning, including sourcing, explaining uncertainty, distinguishing between underlying information and assumptions, and logical argumentation. Because evidence and reasoning are crucial components of every analytic report, REASON will have broader application than previous research efforts aimed at helping the IC make accurate forecasts.
Currently, intelligence analysts are encouraged to use structured analytic techniques to boost the quality of argumentation in their reports. Many of these methods require substantial additional quantities of analysts’ time and are therefore not widely used. As contrasted with current applications of structured analytic techniques, REASON technology will automatically produce comments with no additional effort from analysts, who can use any comments they find valuable.
Some of these comments might be based on the automated application of effective structured analytic techniques, along with additional innovations.
By making specific comments on draft analytic reports, REASON technology will fit into the existing intelligence analysts’ workflow. The comments will be analogous to those made by automated spelling and grammar checks, except that REASON’s comments will focus on improving argumentation instead of writing.
Offerors shall address all three technical Task Areas (TAs) to meet REASON’s goal of developing automated methods to produce comments on draft analytic reports that enable analysts to substantially increase the report’s quality of argumentation:
- Task Area 1 (TA1) – Identify Additional Evidence: Automatically find relevant supporting and contrary evidence in addition to the evidence used in a draft report.
- Task Area 2 (TA2) – Identify Reasoning Strengths and Weaknesses: Automatically find strengths and weaknesses in the reasoning of a draft report.
- Task Area 3 (TA3) – Produce Comments to Increase Quality of Argumentation: Based in part on the output of TA1 and TA2, automatically produce comments that enable analysts to substantially improve the argumentation in their reports.
Offerors must propose novel approaches to each of these three TAs, and if selected as a Performer, will be required to create an end-to-end technology that incorporates software components from each TA. Developed capabilities must be compatible with a provided Application Programming Interface (API) to facilitate assessment by independent test and evaluation (T&E) according to program metrics described in Section II.A.7, Program Metrics.
