Chapter 12 - The Champion-Challenger Project

The request for information or RFI most likely went unnoticed by the majority of federal contract seekers. Likely, the thought of solving one of DARPA's problems was too much for most to consider. However, a select group of companies poured through the RFI with a fine toothed comb. This was it. DARPA was finally looking for a bake-off of the competing technologies being used in the field trials. The energy around this program will be climaxing soon.

The notice (FedBizOpps May 17, 2010):

https://www.fbo.gov/download/56c/56c82971c27d3d3a067639e1606e06a8/SMITE_RFI_17May10.pdf

Information systems security personnel are drowning in ever expanding oceans of observational data from heterogeneous sources and sensors from which they must extract indicators of increasingly sophisticated malicious insider behavior. The Defense Advanced Research Projects Agency (DARPA) Information Processing Techniques Office (IPTO) is requesting information on areas of research related to the development of methods, tools, and techniques to reduce these enormous volumes of data to actionable information. Such technology must be flexible, scalable and highly interactive in order to cope with the dynamic nature of the insider threat. For the purposes of this RFI, we define insider threat as malevolent (or possibly inadvertent) actions by an already trusted person with access to sensitive information and information systems and sources.

The fundamental challenge is one of finding a poorly understood, subtle, or hidden signal (indicators of malicious behavior) buried in enormous amounts of noise (observational data of no immediate relevance) under the constraint that the measures of significance are themselves moving targets (based on dynamic context) that must be continually monitored and updated. The first step in meeting this challenge is to create a scalable, distributed infrastructure to securely collect, store, access, process, and correlate relevant data from heterogeneous sources over extended periods of time. The next step is to determine whether an individual or group of individuals is exhibiting anomalous behavior that is also malicious. However, this analysis is very heavily dependent on the context of the individual, groups of individuals and any data involved. Furthermore, context (e.g., location, time, roles and relations) is dynamic and so must be continually inferred, managed and applied automatically. Part of the challenge is detecting deceptive behavior. Deceptive behavior is characteristic of malicious intent which leads to the problem of assigning intent to observed behaviors.

Looking for clues that suggest an insider attack 1) can be anticipated, 2) is underway or 3) has already taken place could potentially be easier than recognizing explicit attacks. On the other hand, in both the real and virtual world, it is very difficult to do anything without leaving some evidence behind. Attempts to conceal or remove evidence generally create new evidence that, if detected, could be a strong indication of the perpetrator’s intent. Security is often difficult because the defenses must be perfect, while the attacker needs to find only one flaw. An emphasis on forensics could reverse the burden by requiring the attacker and his tools to be perfect, while the defender needs only a few clues to recognize an intrusion is underway.

Forensic-like techniques can be used to find clues, gather and evaluate evidence and combine them deductively. Many attacks are combinations of directly observable and inferred events. Topics of interest to this RFI include, but are not limited to, techniques to (a) derive information about the relationship between deductions, the likely intent of inferred actions, and suggestions about what evidence might mean and (b) dynamically forecast context-dependent behaviors –2 both malicious and non-malicious. Also of interest are on-line and off-line algorithms for feature extraction and detection in enormous graphs (as in billions of nodes) as well as hybrid engines where deduction and feature detection mutually inform one another.

DARPA is requesting white papers in the following three broad areas relating to malicious insider threat detection.........