Human-led Machine Learning & Advanced Threats: Case Method Inquiry and Visual Analytics Applied to COVID-19

Thomas A. Drohan
Approximate Reading Time: 20 minutes


In a pervasive and complex information environment, analytics are vital to understanding advanced threats. As we rely more on machine-learnt results, asking the right questions and visualizing deep analysis are key to grasping and solving problems. These skills are also vital 21st century leadership tools that can forge a common focus among otherwise stove-piped specialists.

The COVID-19 pandemic presents an urgent threat that requires both scientific understanding and decisive leadership. The cause of the disease is SARS CoV-2, a mutating virus that thrives in conditions difficult to control at scale. To counter this threat, this article demonstrates in detail the potential of human-led case method and machine-provided visual analytics.


In a processed data-rich environment, analytics are vital to understanding advanced threats. As we increasingly rely on machine-learnt results, key questions and visual analytics can solve complex problems. These skills are also leadership tools to forge common focus among analysts, planners and operators in a complex information environment.

The COVID-19 pandemic is an advanced threat that requires such scientific understanding and decisive leadership. The cause of the disease is a mutating virus, SARS CoV-2 (Severe Acute Respiratory Syndrome Coronavirus-2), that thrives in conditions that are difficult to control at scale. To counter this threat, this article advocates human-led case method inquiry and machine-provided visual analytics.

Our case method approach uses nine basic questions. This problem-solving process can be adapted to any threat in any domain for any desired effect. In particular, National Security Strategy Pillar One threats also are transnational and share a formidable characteristic. In addition to bio-threats, terrorism, cyber campaigns, and weapons of mass destruction require a single weapon to cause disproportionate, cascading effects. Add criminal networks and narrative warfare to exploit the uncertainty, and our ability to defend the homeland, sustain joint force advantages, deter aggression, and fulfill other objectives in the National Defense Strategy, becomes acute.

Our visual analytics tools that help us see what we otherwise would not, are GraphXR and SavantX. The latter is a new video-capable version of the program we introduced in a previous OTH Journal article.

By using this general-purpose method of questioning and visualizing our problem, we can pull in details that matter and find holistic solutions to systemic threats.

We begin with a fundamental question about advanced threats in complex environments.
How do we discover what data and information are relevant to our problem?

Analysis and Analytics

Our basic solution is to use what we know with best practices of how we learn. This approach is relevant to our ongoing explosion of data and information. That volume of data and information overwhelms what any person can know, so how to learn is increasingly important. We can use analysis and analytics to learn what to know. What is the basic difference between these two terms?

Analysis is a process that breaks a whole into component parts. It’s an intellectual partner of synthesis a cognitive activity that combines various pieces to form a coherent whole.

Analytics are methods of analysis to help us make decisions—such as how to make changes to achieve desired effects.

A good first step toward understanding an ambiguous IE is to step back and look. Try to see the whole mess, then break that down into clarifiable pieces. When machine-and-I rearrange those pieces later, we discover and sometimes create new relationships.

In our Information Environment Advanced Analysis Course, we refer to this thinking as decomposition, recomposition and synthesis. We begin to understand a complex problem by deconstructing our IE into elemental components. We do not start with the SARS-CoV-2 threat or specific tools looking for a problem to solve. We start by identifying major systems, sub-systems, objects, and attributes. Why? Because all of the relevant systems and sub-systems are outside our organization’s lane, not apparent, or deliberately hidden. We treat systems as actors themselves, which helps identify them via behaviors.

Framework for Analysis

Here are our nine case method questions. We will briefly answer the first three and focus in depth on question #4 because we want to visualize the complex linkages of our problem.

  1. Who are the major actors and systems?
  2. What do the major actors and systems want, and what resources are available?
  3. What are the actors’ and systems’ strategies to get what they want?
  4. What are the linkages among the major actors and systems?
  5. What are the patterns, trends and anomalies?
  6. What are your goals and conditions to change, with respect to viral threats?
  7. What incentives/capabilities do you want to influence to achieve goals/conditions?
  8. What activities can you generate to influence incentives/capabilities?
  9. What are your strengths/weaknesses compared to this threat and any competitors?

Our answers to questions 1 through 3 are limited to a few significant international, China, and US examples. They are organized so that each lettered sub-topic (a, b, c, d) underneath each question (1, 2, 3) correlates to one another.

To demonstrate the case method of problem-solving, imagine that we are members of a cross-functional team connected by physical (conference room) and virtual (Zoom, etc.) facilities. You the facilitator orchestrating a learning process that asks questions of the participants. Participants, ideally a mix of subject matter experts, are doing the substantive thinking, and come up with the following answers.

1.Who are the major actors and systems? A variety of actors and interconnected systems are involved at global, state and community levels of interaction.

a. Health-related organizations: World Health Organization (WHO); China National Health Commission (NHC) and Center for Disease Control and Prevention (CDCP); US Centers for Disease Control (CDC); US Department of Health and Human Services (HHS); US National Institutes of Health; US Corona Virus Task Force.

b. Leading health professionals: WHO Director-General Tedros Ashanom Ghebreyesus; China NHC Minister Minister Ma Xiaowei; China CDCP leaders leaders George Gao, Li Xinhua, Liu Jianjun, Feng Zijian; US CDC Director Robert Redfield; US HHS Secretary Alex Azar, US NIH Director Francis Collins; and US Corona Virus Task Force members.

c. Heads of international entities threatened by outbreaks: United Nations Secretary-General Antonio Guterres; China Chairman Xi Jinping; US President Donald Trump.

d. State and non-state information agencies: China’s news is state-controlled by Xinhua News Agency, China Radio International, China Global Television Network, China Daily, and People’s Daily; US news is privately-owned and the top ten newspapers are The Washington Post, Tampa Bay Times,New York Post, Los Angeles Times, Seattle Times, Boston Globe, Denver Post, Wall Street Journal, and Chicago Tribune; US top ten news outlets are Yahoo, Google, Huffington Post, CNN, New York Times, Fox News, NBC News, Mail Online, Washington Post, and The Guardian. Approximately 90% of US news outlets involve six private corporations—AT&T, Comcast, Disney, 21st Century Fox, Viacom, and CBS.

2.What do the major actors and systems want, and what resources are available? Common and competing interests among the previously noted actors and systems include these corresponding samples.

a. Collaboration and transparency: WHO-China interactions reflect a balance. Director General Ghebreyesus praised Chairman Xi’s efforts to contain the virus, despite China’s systematic suppression of information. The international health system arguably wants to integrate China’s resources into a global effort, an incentive that sometimes tolerates non-transparency.

b. Treatment and prevention: Health professionals operate under policy constraints to treat existing patients and prevent outbreaks. Chairman Xi quarantined entire cities, while President Trump banned and limited inbound travel from China and other deeply infected countries. Both policies shaped a common strategy of treatment and prevention: containment.

c. Global awareness and national calm: political authorities attempt to maintain global awareness of threats while managing domestic calm. Chinese officials emphasize the latter to mitigate economic disruption and political unrest. US officials call for balance, but the simultaneous of both outcomes is politicized. When awareness leads to preparations (e.g., testing), the corresponding call for calm is portrayed as a mixed message.

d. Health-related and political agendas: health issues are prone to competing political priorities. Consider two US examples. Health officials initially discouraged the public from buying protective masks so that medical authorities could distribute more to patients, even though some well-sealed masks can provide some protection. The initial low rate of person-to-person spread of this high virality-low fatality virus was a political talking point for national calm, even as local transmission was broadening the impact.

3.What are the actors’ and systems’ strategies to get what they want? Global, state, and community-level actors have different strategies.

a. Collaboration and transparency among health-related organizations: WHO strategy seeks to increase collaboration by: (a) sharing details without domestic restrictions; and (b) fighting disinformation. China’s strategy works to contain COVID-19 by implementing mass quarantines, relying on non-ideological experts, and reasserting authoritarian controls. US strategy aims to contain the virus with external travel restrictions and domestic quarantine bases, without triggering socio-economic disruption.

b. Treatment and prevention among leading health professionals: WHO strategy involves orchestrating collaboration among government agencies, private companies, and research universities to inform practices and accelerate development of a vaccine. China’s strategy encompasses improving sovereign capabilities to contain, reduce and control viral threats with limited external assistance. US strategy is similar to that of the WHO, with Chinese characteristics: collaboration among scientists and a desire for more independent capabilities.

c. Global awareness and national calm among political authorities: WHO strategy pursues increased awareness by organizing the international expertise and assistance. China’s strategy emphasizes national calm and equates that with loyalty to Party directives. US strategy promotes global awareness through transparency, and national calm…somehow. How to achieve the latter is unclear as critiques and counterattacks attribute wrongful intent and attract polarizing disinformation.

d. Health-related and political agendas among health-related organizations, professionals and political authorities: the WHO agenda is an open book of international partnerships, cooperative research, and consensus-building among national and non-state agendas. China’s agenda is an opaque blend of secretive Party functionaries and problem-solving non-partisan experts. The US agenda is a translucent brew of overlapping responsibilities among federal, state and local governments, and research opportunities among academia and business.

4. What are the linkages among the major actors and ideas? This is the question that GraphXR and SavantX will help us answer in some depth. So, get ready for details but pay attention to the questions we ask.

Visualizing Data and Linkages       

GraphXR is a visual analytics platform used in domains ranging from counterterrorism to business intelligence. It bridges the workflows of data scientists and subject matter experts by enabling transformations, filtering, and algorithms to be performed on high dimensional and connected data in a browser-based graphical user interface (GUI).

SavantX is a HyDRA-based analytic that looks for relationships among people, places, things, and ideas in multiple geo-spatial dimensions: Hyper-Dimensional Relationship Analysis. The program is an interactive, supervised machine learning platform that ingests unstructured data from videos with English captions, and from any form of text.

GraphXR and SavantX can be used together to characterize the IE in complementary ways. In our following applications, we describe online applications of Graph XR and demonstrate SavantX.

GraphXR Application

The GraphXR application involves publicly available data on from The Johns Hopkins University’s Center for Systems Science and Engineering.

This online tracker uses GraphXR’s dynamic modeling capabilities to present different perspectives on the CSSE’s COVID-19 data. Multiple instances of the GraphXR interface are embedded within the page alongside contextual information:

OTH, Emerging Security Environment, Multi-Domain Operations

“Daily change in number” maps the locales of confirmed outbreaks to nodes. The number of new cases is indicated by node size, with each date represented as a separate property. Outbreak locations, magnitudes, and directions are immediately apparent. These vectors can be overlaid by transportation hubs and routes to get ahead of further coronavirus transmissions.

OTH, Emerging Security Environment, Multi-Domain Operations

“Stats by region” maps the same dataset to a different model. Each node indicates the number of COVID-19 related deaths reported in a given location on a given date. Compared to the at-a-glance insight revealed by “daily change in number,” this model enables a time series playback for trend analysis. Different nations’ capabilities can be modeled as well, which is particularly useful in building out our answers to Framework Questions 1 through 3 above.

For instance, China’s centralized control of extensive surveillance at the local level may be able to anticipate vectors that are driving community spread. While we are not doing this in the US due to legal protections of our civil liberties, knowing what other actors can do (Singapore, South Korea) is relevant to the IE and therefore our strategy.

OTH, Emerging Security Environment, Multi-Domain Operations

Based on publicly available information, trends in “epicenters” of outbreak can be displayed to indicate viral growth rates and death rates. Iran’s death rate, depicted by the size of each node below, is currently the world’s highest at 7.6%. This compares to 3.6% in China, 2.6% in Italy, and .5% in South Korea:

OTH, Emerging Security Environment, Multi-Domain Operations

Exploratory analysis of the same model from multiple perspectives can reveal anomalies and underlying issues in the data’s reliability. For instance, the “death per 100 cases” score could be understood to reflect a higher mortality rate of COVID-19 in certain countries. However, given the coronavirus’ mechanisms for transmission and lethality are not influenced by national boundaries, it is unlikely that COVID-19 is deadlier in the US and Iran. More probably, detection efforts have been less successful in these countries, inflating the apparent ratio of deaths to reported cases.

OTH, Emerging Security Environment, Multi-Domain Operations

OTH, Emerging Security Environment, Multi-Domain Operations

The image above shows yellow, green and blue nodes. Yellow nodes show test results from recent COVID-19 cases, with their height reflecting different test dates (lower samples are more recent). Green nodes are virus mutations, with their size reflecting how different they are from the first reported strain. Blue nodes represent test result collection locations. Further analysis conducted by Kineviz on similar data provided by Nextstrain and GISAID shows extensive viral spread despite testing and quarantining.

In this application, all data is drawn from a single source. GraphXR’s dynamic modeling also makes it possible to ingest and compare data from multiple sources, including databases and live streams. This enables subject matter experts to create bioinformatics on COVID-19 as needed, without relying on technical users to perform extract-transform-load (ETL). Correlations among symptoms and diagnoses across communities and regions can inform decisions about where to place resources and expertise.

SavantX Demonstration

This application of SavantX demonstrates how to retain our human prerogative of making decisions about machine-processed relationships. We chose a YouTube video for SavantX to process, “Experts discuss COVID-19 at Johns Hopkins Carey Business School.”

In making these decisions, we ended up conducting three approaches to a target of influence (TOI): singular search; leverage search; and radial search.

A singular search is a focused look at one TOI; a leverage search looks for a countervailing relationship to leverage against your TOI; and a radial search begins with one target but expands until you find another TOI of interest.

The demonstration provides a description of each search type and goal, followed by a brief description of the results.

1.Singular search: finding relationships related to a target of influence.

The visual analysis of the video began at the lowest scale of displayed relationships (note the sliding bar on the left side of the image below, which currently is all the way at the bottom of the vertical scale).

The first three relationships displayed were among the nodes named virus, risk and countries.
OTH, Emerging Security Environment, Multi-Domain Operations

OTH, Emerging Security Environment, Multi-Domain Operations

We were interested in more relationships, so we increased the fidelity of the program (sliding the vertical scale on the left, upward). This proliferated the number of nodes shown to over 60.

Of these nodes, we selected risk, testing, and severe illness. Why? We were interested in relationships between risk and testing, and between risk and severe illness, for three reasons. First, we need to manage risk. Second, testing is something we can influence. Third, we should prevent severe illness.

Directing SavantX to find relationships among those nodes yielded the following information.

With respect to risk and testing, testing is done only on those who seek treatment. This led to a recommendation that high-risk individuals ought to be tested. For risk and severe illness, the need to develop a vaccine for individuals with higher risks of developing severe illness led to that being considered as a priority.

The usefulness of the singular search approach here is that we are able to:

(a) find related conditions that we want to influence
(b) discover relationships that could influence those conditions

Targets in the IE, therefore, can be conditions (risk, testing, severe illness, etc.), not just actors.

2.Leverage search: finding countervailing relationships against a target of influence.

Having selected our targets to influence — risk, testing, and severe illness — we again increased the fidelity of the program. We noted infected was a node, which got our attention because it adversely influences risk, testing, and those with severe illness.

OTH, Emerging Security Environment, Multi-Domain Operations
Selecting infected for analysis yielded the following information: the majority of people infected have cold-like symptoms that would not suggest COVID-19 and would not even lead them to necessarily seek medical attention. Therefore we need to test people with a connection to the origin of the virus (China). One person in Singapore, for instance, infected 11 others.

The usefulness of the singular search approach is that we are able to:

(a) find related conditions that we want to influence
(b) discover relationships that could influence those conditions

Targets in the IE can be conditions (e.g., risk, testing, and severe illness).

3.Radial search: finding new linkages, beginning with a primary target of influence.

We directed SavantX to find relationships among our three nodes of interest — risk, testing, and severe illness — which again generated dozens of nodes. None of of these nodes, when selected, displayed common relationships with the risk, testing, and severe illness. The unproductive nodes were named provider, surveillance, ill, virus, cases, and influenza. At this point, we needed to find new linkages.

OTH, Emerging Security Environment, Multi-Domain Operations

So we deselected “severe illness” and selected “symptoms” instead, increasing the fidelity of the program to see if there were four-node relationships. The nodes that showed up were named infected, viruses, ill, and potentially.

OTH, Emerging Security Environment, Multi-Domain Operations

Of these, the only node that yielded a relationship was infected, which contained the following information:

Viruses that cause non-specific symptoms make it difficult to identify COVID-19 unless people seek medical attention or have other risk factors that would cause their medical providers (assuming they have one) to contact them.

The usefulness of this radial search was that it revealed the limits of relying on the following information:

(a) symptoms
(b) people arriving for medical attention
(c) people with a known risk factor

We also know with reasonable certainty that the risks of CoV-2 include:

(a) insidious symptoms which often preclude testing
(b) unknown personal risks to infections that can increase the severity of illness

Next Steps in the Analysis

While the scope of this article precludes going through the remaining steps, here is a brief outline. Having identified linkages, the next step is to find patterns, trends and anomalies (framework question #5). Infectious disease researchers, for instance, have identified a COVID-19 pattern in patients’ CT-scans that confirms diagnosis of the virus. From there, we can develop actors’ goals and desired changes in conditions (Step 6), incentives and capabilities to influence (Step 7), activities to generate (Step 8), and strengths and weaknesses among the now-specified strategies (Step 9).

Two Recommendations for COVID-19

Based on the preceding linkage analysis, we can generate two recommendations.

First, COVID-19 is an advanced threat and actor itself, one whose mutating behavioral characteristics need to be fully understood. Researchers are collaborating in a global effort to streamline the trials of vaccine development. Defense Support to Civil Authority is also being proactive as US Northern Command mission prepares plans to contain the disease. Given what we know about how COVID-19 behaves, treating the virus as an insurgent may have its merits. Countering this threat, for instance, requires good governance and legitimacy. This relates to our next recommendation.

Second, the US Corona Virus Task Force can influence conditions that empower COVID-19’s effects. A communications campaign can convey to the public what we know and do not know. We know that the virus is highly infectious, more fatal as the common flu, yet may be accompanied by common cold-type symptoms. The virus is mutating and spreading very quickly, despite efforts to contain further outbreaks. Therefore self-quarantine, seeking medical attention, and communicating risk factors to care providers, are vitally important but insufficient. Risk factors include vulnerable immune systems, upper respiratory issues, obesity, and diabetes. Systemic solutions are needed to defeat the disease.

Three Recommendations for Data Analytics

Be a human in charge of machine-learning. Ask different types of analytic questions within and beyond a machine’s data set. This may be outside your job jar, so cross-functional teams are vital to understanding the whole problem. Inquiry should include cause-and-effect questions based on historical data (machines excel at this), and proactive questions with future possibilities and probabilities in mind (humans do this).

Fit and refute analytics’ results. What are assumptions, evidence, and logic that fit the results, AND, what are assumptions, evidence and logic that would refute the results? Similar to nurturing an effective red team, leaders must generate such hard questions to ask of themselves, responsible organizations, and key individuals.

Scrutinize constructs of thinking. Constructs include any framework at any level of responsibility or technology that structures thinking. How a video, speech, or reading is organized may not be readily apparent without the assistance of machine learning that can break it down. Narrow AI can also process simulations and exercises; a way to future-cast desired changes in conditions.

Visualized data can help integrate the sometimes separated efforts of analysts, planners, and operators. This requires human-led, information environment-wide, mission-oriented leadership.

Brig Gen (ret) Thomas Drohan is Director of the International Center for Security and Leadership, JMark Services Inc ( He formerly headed the Department of Military & Strategic Studies at the United States Air Force (USAF) Academy. He holds a PhD from Princeton University, an MA from the University of Hawaii, and a BS from the USAF Academy. Brig Gen Drohan’s publications include A New Strategy for Complex Warfare: Combined Effects in East Asia, and articles in journals such as Joint Force Quarterly and Defense Studies. His career includes combat rescue, airlift and anti-terrorism in East Asia, the Middle East, and Afghanistan. He is a Council on Foreign Relations Japan fellow and Reischauer Center for East Asian Studies scholar.

Disclaimer: The views expressed are those of the author and do not necessarily reflect the official policy or position of the Department of the Air Force or the United States Government.

Feature Image Source: Nodal Analysis/, COVID-19/, SavantX/ User Generated

OTH, Emerging Security Environment, Multi-Domain Operations

Print Friendly, PDF & Email

One thought on “Human-led Machine Learning & Advanced Threats: Case Method Inquiry and Visual Analytics Applied to COVID-19

  • March 26, 2020 at 10:37 am

    An outstanding and timely publication. Thank you.


Leave a Reply