Regain EMS Dominance By Measuring Effectiveness Right

Estimated Time to Read: 12 Minutes
By: Jon-Michael Chombeau

Excerpt: Non-Kinetic Effects (NKE) do not have accepted measures of Battle Damage Assessment (BDA) and never will. New ways of measuring effectiveness are required to ensure Joint Electromagnetic Spectrum (EMS) dominance. Time to Effect (TTE) revitalizes Boyd’s Observe-Orient-Decide-Act (OODA) loop to inform modern kill chains. Sensor Rating (SR) borrows from the National Football League’s Passer Rating to measure sensor effectiveness. TTE and SR fill a growing gap and will enable better integration and execution of NKE.

“We have lost the electromagnetic spectrum.” This blunt statement by Alan Shaffer, the Pentagon’s then research and engineering chief in 2014, was the start of a rapid prioritization of Electromagnetic Warfare (EW) by the United States. This shift was a response to our adversaries’ investment in EW capabilities and tactics, as evidenced by Russia in Ukraine. The Department of Defense continues to struggle to integrate joint NKE, including EW. Universal, objective, and accessible metrics are required to evaluate performance, capability, and effectiveness if the Department of Defense (DOD) is to optimize and integrate NKE across the Joint Force.

Understandably, the focus of recent EW investment has been improving governance and technology. While technological superiority is a priority, without trained operators, validated tactics, and deep integration technology alone has no hope of success. The Joint Force needs to define what superiority in the EMS domain looks like, not just from a strategic level but at the tactical level as well. Kinetic Effects (KE) quantify capabilities through a combination of predictive measures and BDA. NKE capabilities remain unquantified in many cases or require expertise to understand. True NKE BDA continues to elude the Joint Force because NKE by their nature can never meet the same standards for causality that KE BDA requires. Creating NKE BDA and predictive measures requires a new paradigm that accepts greater levels of uncertainty to determine effectiveness. A new paradigm opens the way for new metrics which will bring parity to KE and NKE measurement.

KE BDA holds primacy over NKE BDA for two reasons: they are easily quantifiable and they have a standing historical bias. KE are easily quantified, communicated, and understood. While there are variations, most metrics used to measure KE boil down to defining ratios. These ratios typically describe how many systems, platforms, or weapons friendly forces expect to use to destroy or disable one of the adversary’s systems. Modeling and testing inform these expectations and combat validates them. Common examples include: (1) salvo requirements, which define the quantity of ordnance required to destroy a threat; (2) fighter exchange ratios, which quantifies the number of enemy fighters destroyed for every friendly fighter; and (3) probability of kill (Pk). This information informs commanders in planning and sets a standard to measure execution performance. For example, salvo requirements are a predictive measure a commander will use to determine how much ordnance should be used to gain the desired effect. BDA quantifies the effect that occurred and informs if the reality matched predictions. Quantifying and predicting NKE is much more difficult because the subtlety and breadth of variables that determine performance are dynamic and complex. For example, EA BDA requires determining the effectiveness of radio transmissions on the system and the operator as well as whether it occurred for a tactically relevant amount of time. Most importantly, this NKE quantification must also be accomplished without the “gratification” of a visible explosion.

The second reason for a KE bias is historical. Aircrew developed Battle Damage Assessment (BDA) in World War I and have continuously refined it in subsequent conflicts. Information warfare may be as old as traditional warfare, but most current EW practices have origins in World War II, and cyber warfare is at most a generation or two old. As a result, there is no generalized measure for determining an operator’s performance or effectiveness when employing NKE. A product of this bias toward KE is that in both training and execution, NKE are not well communicated, understood, or held to an equal standard for successful employment. For example, it is still an open question whether Stuxnet, a cyber attack perpetrated on Iranian nuclear facilities, was effective.

New NKE metrics must be objective, as subjective measures are only as useful as the individual providing them. It is difficult enough to train a subject matter expert in a single weapon system. The breadth and depth of capabilities in NKE means any person with general knowledge of all NKE will have little depth in specific capabilities. Therefore, inadequate information, assumptions, and simplifications will always inform subjective opinions. This will vary from person to person. Objective measures provide methods for truthfully comparing the varied capabilities available with NKE. Moving away from subjective opinions to objective measures create the following benefits for operators and decisions makers:

  • NKE operator training and readiness optimization: When an operator employs NKE, sometimes it is a simple switch or button press. Other times, it requires constant operator attention, skill, and finesse based on countless hours of training and study. Objective metrics enable better training and readiness assessment through accurate measurement of operator, unit, and force capability.
  • Accurate mission planning expectations and lessons learned: When mission planning, it is critical to express and understand each system’s capabilities and limitations. Planners cannot define or communicate such expectations without objective measures of performance (MOP) and measures of effectiveness (MOE). Unclear or subjective expectations lead to NKE often being considered a “bonus,” with mission planners assuming zero NKE or inefficiently “layering” redundant effects. Such planning limits capability and imposes unnecessary risks. Even worse, it trains operators that mission success does not require NKE.
  • Better acquisition decisions: Current acquisition decisions are made using limited information about the actual capability of a system in a complex environment because no universal method exists to objectively measure capability. It is likely millions if not billions of dollars are misappropriated due to unexposed risks and redundant, stove-piped lines of effort.
  • Informed tactics, techniques, and procedures (TTP) development: Developing TTPs requires knowing which tactic works best. Without objective measures, it is impossible to determine and optimize the best TTP.

The problem of NKE quantification can be solved now; the first step of which is to develop a common language informed by trusted, useful metrics. Any metrics developed must be:

  • Objective to ensure trust and consistency.
  • Accessible to operators and planners across the joint environment.
  • Universal or applicable to more than a single subset, platform, or capability.
  • Measurable in all training environments, including live execution and modeling and simulation.

The Joint Force must adopt universal, objective, accessible NKE metrics to facilitate tactics optimization, acquisition decisions, and create a better understanding of combined capability in the EMS. The NKE community should implement two metrics now: Change in Time to Effect and Sensor Rating. Both metrics serve as a starting point and the crucial first step in bringing parity between understanding KE and NKE effectiveness.  

Change in Time to Effect (∆TTE)

Put simply, Time to Effect (TTE) is the time required for a system, platform, or entity to recognize a change in the environment and implement the proper response. TTE is a modern remix of Boyd’s OODA (Observer, Orient, Decide, Act) loop. TTE looks at the OODA loop for an operator and measures how long it will take to accomplish the desired task in each environment. TTE provides a measurable, objective metric of capability for a system in a complex environment and is consistent across various KE and NKE. As an example, Surface to Air Missile (SAM) systems requires a certain amount of time to find, fix, track, and engage an adversary’s aircraft. The measured time is the SAM’s TTE.

NKE which focus on affecting systems prior to weapons employment, or “left of launch,” are generally trying to exploit operator or equipment vulnerabilities. Examples of impacts on an adversary’s TTE include forcing an operator to: implement a different TTP, activate an electronic protect (EP) capability, work through numerous false targets, or simply deny the system any detections. If NKE are attempting to influence TTE, then a measurement of a change in TTE would be an objective and accurate way to measure the NKE’s impact. Change in Time to Effect (∆TTE ) creates a valuable MOE for NKE.

∆TTE requires measuring the “standard” TTE and the “actual” TTE. Both are simple to measure but difficult to define. Consider a SAM system that is trying to shoot down a friendly fighter. The SAM operator is trying to produce an effect of shooting down the blue aircraft, while the aircraft is trying to increase the SAM’s TTE using jamming (NKE). The start of TTE measurement should occur at the point at which the SAM would “normally” detect the friendly fighter in an un-jammed environment. TTE measurement ends with a successful missile engagement. The difference in SAM TTE between the fighter jamming and not jamming would provide a measure of the NKE effectiveness. ∆TTE creates an MOE, which can act as real time NKE BDA, but only when a greater degree of uncertainty is accepted. If ∆TTE increases concurrent with NKE execution, NKE is likely the cause, but certainty is not and will never be available.

Image Created by Author

∆TTE can also capture synergistic effects. Increasing the complexity of the environment does not change the utility or ease of measure of ∆TTE. In the above example, adding a second jammer or cyber capability, may complicate determining an individual NKE’s effect; however, it does not complicate measuring the combined effect. ∆TTE utility in a complex environment provides the ability to measure multiple NKE in a controlled environment to determine if they are complementary, competitive, or redundant. When approached correctly, ∆TTE provides a metric to measure the efficacy of individual NKE capabilities, or different NKE TTPs, providing a more objective method to determine which capability is most effective.

Placing ∆TTE in every operator’s lexicon will provide a common MOE to discuss, plan, and execute with NKE. ∆TTE simplifies mission planning, leads to better acquisition decisions, and allows for NKE TTP refinement.

Sensor Rating (SR)

At present, there is no universal, objective metric to measure or predict a sensor’s effectiveness on a given day. Operators lose vital planning fidelity when forced to use oversimplified range numbers or 1-vs.-1 models to communicate predicted sensor performance. The only method available to measure a sensors performance after an event would often be subjective, e.g., “the sensor worked good/fair/poor today.” Detection ranges, sensitivity, number of simultaneous tracks, scan volume, and scan rate are among a few metrics that exist that can help define a sensor’s potential capability. To measure the performance accurately on a given day, operators must account for NKE, electromagnetic interference (EMI), and operator tactics. A method to combine information into a single metric is required to capture all these effects and maintain accessibility. Professional sports, specifically Passer Rating in American football, provides an example metric that can be applied to NKE.

In the National Football League (NFL), multiple different metrics: touchdown passes, completed passes, interceptions, and incompletions are used to measure a quarterback’s performance. In an effort to simplify quarterback comparison the NFL developed Passer Rating. The relevant statistics are used to calculate Passer Rating which is a single number between 0 and 158.3. For two quarterbacks, the individual with the higher Passer Rating generally performed better than the quarterback with the lower Passer Rating. There are exceptions, of course, as well as statistical aberrations, but neither has significantly diminished Passer Rating’s widespread acceptance.

An objective, accessible metric for measuring sensor performance utilizes the same methodology for creating Passer Rating. Capturing a sensor’s performance in a complex threat environment can provide a way to measure NKE. For targeted sensors, an operator’s NKE performance inversely relates to the sensor performance. The better NKE operators, systems, and tactics will create lower sensor performance in targeted systems. Therefore, a measurement of the sensor’s performance helps determine the relative NKE effectiveness. This metric should be called Sensor Rating (SR). SR is another MOE which provides real time BDA and predictive metrics. SR does not meet the current BDA causality requirements, but under a new paradigm SR will shape TTP’s, acquisitions, and execution.

The core statistics used to create Sensor Rating should be based on the primary objectives of NKE when applied to a sensor: Denial, Deception, Degradation, and Disruption. To measure SR requires comparing deviations from a sensor’s ideal capability. Comparing the difference between the present reality and the ideal creates information on the sensor’s performance in a complex environment. Variation in a sensor’s performance provides a useful measure of NKE effectiveness. As with Passer Rating, Sensor Rating either compares the performance of a sensor over time or compares two sensors to determine which is more effective.

Image Created by Author

As an example, consider multiple friendly fighters flying inbound towards an adversary SAM system being targeted by friendly NKE. To target an aircraft the SAM system requires a sensor. Current metrics provide that sensor’s maximum range and maximum number of tracked targets, but neither is useful for determining NKE effectiveness. In our example, the more false tracks, track denials, and track degradations the NKE creates in the targeted SAM sensor, the lower the SR. Measuring and tracking the SR provides an MOE of the NKE applied to the sensor. Theoretically a skilled operator may be able to work through the NKE with little or no adverse effects. Therefore, SR can only provide correlative evidence for BDA, not causation. Still, observing SR as a function of time provides useful information for measuring NKE effectiveness in a single number, applicable to all sensors, or even the entire Integrated Air Defense System. Currently no such metric exists but SR can fill the gap.

In football, Passer Rating is not a perfect measure. Only performance, and not causality, is considered. An interception is an interception, regardless of what caused it. Despite Passer Rating’s flaws, it is the primary measurement of a quarterback’s performance in the NFL. There are zero metrics to measure sensor or NKE effectiveness effectively. The initial step does not need to be perfect; it just needs to be truthful, accessible, and useful. Sensor Rating is a tool that can raise everyone’s awareness and understanding of NKE and optimize operator and Joint Force performance.

Conclusion

The United States must regain ground lost to its adversaries in the EMS domain and the foundation of these efforts requires effective assessment methodologies. ∆TTE and SR are separate metrics which, when compared, can help explain the outcome of an event in greater detail as relates to NKE. For example, if a system has a lower than expected SR, it should be expected that their ∆TTE was adversely affected.  If ∆TTE is large, a lower SR should be expected. When the previous examples are not the case, operators have an indicator to dig deeper and determine what happened and why.

Application of ∆TTE and SR in training and test is simple once the proper data capture tools are put in place. These two metrics apply to almost all EW and many cyber effects. Having a method to measure capability across all or most EMS operations provides warfighters with the tools needed to pick the right effect at the right time and evaluate subsequent performance. Clear and objective assessment is critical to developing new warfare doctrines and capabilities which will only become more dependent on precise, synchronized application of NKE and KE to win on the dynamic and complex future battlefield.

NKE are critical to mission success, yet to date there is no comprehensive method of measuring operator, force, or sensor performance. We can no longer afford to rely on antiquated, anecdotal, and subjective measures of stove-piped systems.Universal, objective, and accessible metrics are required if the DOD is to optimize and integrate NKE. Two metrics to implement immediately are Change in Time to Effect (∆TTE) and Sensor Rating (SR). Acquisition, Training, Tactics, and the nation’s overall capability as a fighting force hinge on our ability to accurately measure effectiveness. With useful metrics, the fog and confusion around a critical warfighting ability will dissipate as operators truly integrate and optimize NKE and KE.

Time To Effect analytic framework developed in concert with Renato DePaolis.

Jon-Michael “Malibeux” Chombeau is a former combat EA-18G pilot and Tactics Instructor. He currently works for Advanced Strategic Insight as Director, Mission Systems Training and provides Electromagnetic Warfare Subject Matter Expertise to Boeing’s Virtual Warfare Center. j.chombeau@advancedstrategicinsight.com

Disclaimer: The views expressed are those of the author and do not necessarily reflect the official policy or position of the Department of the Air Force or the United States Government.

Featured Image provided by author and at Link.

Print Friendly, PDF & Email

Leave a Reply