Leveraging Predictive Data Models to Strike Mobile Targets at Scale

Read Summary

Reading Time: 6 minutes

In an era defined by the rapid obsolescence of emerging technology, battles will be won by those who can create and communicate novel solutions the fastest. Success in the evolving concept of joint all domain operations (JADO) revolves around the ability to target and strike a growing number of highly mobile targets. The volume of data being collected in support of that mission has surpassed the capability of human analysts to simply streamline legacy processes, necessitating a move towards algorithmic support in future collection and targeting strategies. This paper examines the Markov Decision Process (MDP), a sequential process that uses a mathematical framework to model dynamic systems in scenarios where the results are either random or controlled by a decision-maker, as a solution.[1] Using the MDP presents an opportunity for algorithms to develop probabilistic equations for use in optimizing targeting collection at a speed and scale required on future battlefields.

Signature Analysis and Custody

The Joint Force has improved its targeting processes over time by increasing the number of unique signatures collected that can be used for detecting and identifying a threat system, but the need to maintain episodic fused target custody (FTC) complicates the ability to prosecute these targets at scale. FTC is a process centered on using multi-source analysis to maintain accountability of a target system in support of joint fires.[2] When a unique signature is detected by a collection sensor, the threat associated to that signature is considered “found.” Various sensors are then orchestrated and tasked to maintain custody. However, when this process is expanded to a larger number of threat systems in an expansive area of operations, the scale and need for resource allocation and management becomes too large to oversee. Tomorrow’s battlespace demands we explore alternatives for efficiency. Instead of waiting for a signature to be detected, how can intelligence professionals anticipate the location of a threat to optimize the use of limited collection resources?

Leaning into Prediction

Amos Tversky, a prominent psychologist in probability and prediction, argued “Man is a deterministic device thrown into a probabilistic universe.”[3] Humans prefer to view outcomes in terms of black and white. In reality, decisions are based on the probability of a predicted future outcome. Analysts in diverse fields rely on their abilities to sort through large volumes of information to generate predictive analysis. Today, there is an ever-expanding amount of information available to an analyst, leading to a myriad of products and theories on how to best optimize the process of data analytics.

U.S. Air Force intelligence analysts rely on descriptive analytics rather than predictive analytics to inform the Commander’s Critical Information Requirements (CCIRs). Therefore, intelligence analysts tend to rely on past events to understand the adversary and establish requirements to confirm, or deny, a trend. Analysts have tools to display historical or near-real-time intelligence data, but no predictive model designed to inform future probabilities of distinct outcomes. The MDP can fill this void. It consists of four key elements: an agent, state, actions, and policy.[4]  The agent is the system or individual that is making decisions. There are multiple states, or environments, in which an agent is capable of operating in at any given time. Based on the current state, an agent determines a fixed set of actions that it can take to either remain in their current state or transition to a different state. The policy helps to determine the agent’s next action based on its current state.[5]

In theory, the likelihood of an agent transitioning from one state to another is predicted by the current state and a probability of which state the agent can go to next.[6]  In the example below, there are two states the agent can exist in: State 1 (S1) and State 2 (S2). The green lines are associated with Action 1 (A1) while the red lines are associated with Action 2 (A2). Assume you are in S1 using A1. This model shows a 30% probability the action you take keeps you to S1 and a 70% probability that leads you to S2. Therefore, the odds are if in S1 and take A1 the agent will most likely move to S2.

Figure 1. MDP Example[7]

For example, a human subject may have states of “At Home, “In Transit,” and “At Work.” By knowing the current state, we can use past observations to determine the likelihood of a transition to a new state. Given a one-week observation period, the subject transitions from “At Home” to “In Transit” on weekdays, and remains in the state of “At Home” on weekends. If the subject is known to be “At Home,” we can assign a high probability that a transition to “In Transit” is likely to occur as it has in 5 of the last 7 days (71%). There remains a smaller yet distinct probability that the state will not change, and the subject will remain “At Home,” perhaps due to it being a weekend (29%).

Figure 2. State Transition Probability

Application to Mobile Threats

In the wars of tomorrow, we will not have the ability to know and monitor the state of every target and its actions in the battlespace. Thus, when looking to apply the MDP model to mobile threat systems, their partially observable nature must be considered. The Partially Observable MDP (POMDP) is a generalization of the MDP for when there is missing information. In the POMDP, analysts are unsure which state the agent is currently in while observing actions of the agent.[8] These observations provide insights towards predicting which state could be next, based on the probabilities associated with returning to the same state or moving on to separate states.[9]

More specifically, the POMDP could allow analysts to apply this model to target sets, such as an aircraft. Each target can be thought of as an agent (a plane) capable of operating in multiple states (On the ground or Airborne). Under the primary state of being “On the Ground” or “Airborne,” aircraft can be in any one of several sub-states; maintenance, preparing for operations, in-transit to operating area, or conducting operations. Signatures associated with each sub-state can then be used to provide insight into its current and future state. Figure 3 contains a graphical depiction of this concept.

Figure 3. Aircraft POMDP Example

Assuming the state and sub-state are unknown, an analyst can use observable signatures to deduce the probability of the current state of the aircraft. For example, if an aircraft is observed in a hangar, the probability of the aircraft being in the maintenance sub-state is higher than “preparing for operations.” When primed to use this information for predictive behaviors, the value of state-related observations increases. If an analyst observes weapons loading with the historical context to know that 80% of previous weapons loading observations led to an aircraft departing in 12 hours, we can predict with higher confidence that the aircraft will soon transition to “Airborne.” There remains a 20% chance that the aircraft is not preparing to transit, either returning to maintenance of staying static on the airfield, as illustrated in Figure 4.

Figure 4. State Transition Probability

The ability to identify consequential state transitions will enable a new level of efficiency for high demand, low density collection resources. Rather than trying to maintain awareness on every target in each theater, sensors can be prioritized against enemy systems likely to pose a near term threat to blue forces. Collection orchestration efforts then become a series of probabilistic “if, then” statements able to be solved and prioritized by an algorithm at a scale that exceeds human capacity. If a threat is in State 2, then custody must be maintained. If one of the observable signatures of the threat operating in State 2 is the use of its Radar, then SIGINT collection assets are required. Although similar processes can be done by humans building collection strategies, the scale and speed of peer conflict paired with historical timelines will render legacy processes ineffective.

Way Ahead

Threat system mobility and battlefield complexity will only continue to increase, further hindering the joint force’s ability to effectively target adversaries. A small group of planners and analysts have already begun generating nascent POMDP models for real-world target sets. This effort must be expanded, and greater emphasis placed on integrating MDP. As these efforts take place, planners and analysts must codify their lessons learned and begin to apply their POMDP models against additional target sets throughout multiple areas of responsibility (AOR). Concurrently, Joint leaders must advocate for applying this refined predictive modeling in their area of operations. After the POMDP models have demonstrated their effectiveness, the DoD must provide adequate resourcing for POMDP models through a program objective memorandum, incorporate them into targeting strategies, and conduct operational tests on a growing scale until a globally viable solution is operationalized.

---

Disclaimer: The views expressed are those of the author and do not necessarily reflect the official policy or position of the Department of the Air Force or the U.S. Government.

---

[1] Martin L. Puterman, “Chapter 8 Markov Decision Processes,” in Handbooks in Operations Research and Management Science, vol. 2, Stochastic Models (Elsevier, 1990), 331–434, https://doi.org/10.1016/S0927-0507(05)80172-0.

[2] Air Combat Command, Air Force Tactics Techniques and Procedures (AFTTP) 3-1.DCGS

[3] Lewis, Michael. 2017. The Undoing Project. Harlow, England: Penguin Books.