Publicly Available Information: The Secret to Unclassified Data, Part I

Estimated Read Time: 10 minutes

By David R. Lands

Abstract
The information age represents both challenges and opportunities for the Intelligence Community (IC). Access to data via hand-held devices and personal computers has increased the amount of information available to all who possess the means to retrieve the vast amounts of data; social media, text messages, images, news, government reports, scientific and technical research, blogs, academic information, and video that are shared publicly. This paper intends to increase awareness of what publicly available information (PAI) is, and what it is not, within the Air Force intelligence, surveillance and reconnaissance (ISR) Enterprise and determine its value to enhance our ability to solve our most challenging problems. PAI’s value will be realized by incorporating unclassified data into existing dynamic collection management processes, and when combined with classified information, will increase confidence of ISR fusion analysis. Considering PAI is unclassified data that resides primarily on the internet, we cannot assume it is accurate. We must also consider the challenges posed by deception and disinformation campaigns of adversaries who recognize data as a US center of gravity. Acknowledging PAI as foundational data that potentially leads to valuable open sources intelligence (OSINT), the IC, Defense Intelligence Agency, and Air Force ISR Enterprise need to work together to raise awareness and prepare analysts to take advantage of the opportunities PAI presents.

“In the age of information, ignorance is a choice.”
Donald Miller, Author

The Need to Know
The information age present both challenges and opportunities for the IC. Access to data via hand-held devices and personal computers has increased the amount of information available to all who possess the means to retrieve the vast amounts of data, social media, text messages, images, news, government reports, scientific and technical research, blogs, academic information, and video that are shared publicly. Lieutenant General VeraLinn “Dash” Jamieson, Deputy Chief of Staff for ISR, recently stated that PAI…“will constitute between 60 – 80 percent of our intelligence assessments at the speed and scale of modern warfare.” She further recommended Air War College’s Vigilance Horizons ISR Research Task Force consider “Reaching Next Generation ISR Dominance through exploitation of Publicly Available Information” as a research topic. This paper intends to increase awareness of what PAI is, and what it is not, within the Air Force ISR community and determine its value to enhance our ability to solve our most challenging problems. PAI’s value will be realized by incorporating unclassified data into existing dynamic collection management processes, and when combined with classified information, will increase confidence of ISR fusion analysis. Considering PAI is unclassified data that resides primarily on the internet, we cannot assume it is accurate. We must also consider the challenges posed by deception and disinformation campaigns of adversaries who recognize data as a United States (US) center of gravity.

PAI is defined by DoD Manual 5240.01 as “Information that has been published or broadcast for public consumption, is available on request to the public, is accessible on-line or otherwise to the public, is available to the public by subscription or purchase, could be seen or heard by any casual observer, is made available at a meeting open to the public, or is obtained by visiting any place or attending any event that is open to the public. Publicly available information includes information generally available to persons in a military community even though the military community is not open to the civilian general public.” PAI is unclassified data. It is not intelligence; it is raw unanalyzed data. PAI should not be confused with OSINT.

The National Defense Authorization Act for Fiscal Year 2006 states, “OSINT is intelligence that is produced from publicly available information and is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement.” The distinction between PAI and OSINT is best understood when comparing the meaning of “I” in PAI – information, to the “I” in OSINT – intelligence. PAI is a single component of information or data used to produce OSINT, but by itself, is only information. Only after PAI is exploited and determined to possess value that will result in addressing a specific intelligence requirement, does it become intelligence. Not all PAI will become OSINT. PAI that doesn’t possess intelligence value will remain information.

 “Ninety percent of intelligence comes from open sources. The other 10 percent, the clandestine work, is just the more dramatic. The real intelligence hero is Sherlock Holmes, not James Bond.”
Lieutenant General (ret) Samuel V. Wilson, former director DIA

The December 2017 National Security Strategy (NSS) of the United States of America recognizes the importance of open sources to our nation. The third pillar of the NSS is “Preserve Peace Through Strength,” and identifies intelligence as the final component in the “Renew Capabilities” section of the strategy. Specifically referring to open sources of information in the “Priority Actions” subsection of intelligence, the strategy states, “HARNESS ALL INFORMATION AT OUR DISPOSAL: The United States will, in concert with allies and partners, use the information-rich open-source environment to deny the ability of state and non-state actors to attack our citizens, conduct offensive intelligence activities, and degrade America’s democratic institutions.” The inclusion of allies and partners when considering open sources makes strategic sense. Classification becomes irrelevant when the information being shared is publicly available unclassified data. We can share PAI without concern for disclosing the sources and methods used to obtain the information.

Radio – Our Quest for Information Begins
Formal analysis of PAI and open sources of information was being conducted before creation of the IC. Eighteen months after the Second World War began, on 26 February 1941 the Foreign Broadcast Monitoring Service (FBMS) was established in the US when the Federal Communications Commission (FCC) was directed to monitor foreign radio broadcasts. Analysis capability was originally initiated by academic institutions at Princeton and Stanford Universities, whose purpose was to interpret and understand what was being said, and perhaps more importantly, to consider the intent of the communicator. The fundamental questions being considered was whether the information intended to inform, persuade or influence the audience. The functions tasked to FBMS were to record, translate, report and analyze foreign broadcasts for the purpose of monitoring propaganda, policy, news and information that was not available by other means.

Less than two months later, FBMS’ purpose changed to provide government “up-to-the-minute” news, intelligence information to defense agencies, and foreign propaganda strategies. Priorities quickly changed, most notably diminishing the focus on propaganda and emphasizing a transition from information to intelligence. On 26 July 1942, FBMS was re-designated as the Foreign Broadcast Intelligence Service (FBIS), only to change names again four years later on 31 July 1946 to the Foreign Broadcast Information Service. The previous transition from information to intelligence was reversed as the name changed from intelligence service to information service. 1946 also marked the assumption of the FBIS by the Central Intelligence Group, the precursor to today’s Central Intelligence Agency (CIA).

The most significant twenty-first century change to the FBIS was the creation of the Director of National Intelligence’s Open Source Center, established 1 November 2005. A CIA press release announced, “The Center will build on the established expertise of the CIA’s Foreign Broadcast Information Service (FBIS) — an organization that enjoys a long history of providing the US government highly valued open source products and services. The Center’s functions will include collection, analysis and research, training, and information technology management to facilitate government-wide access and use.” On 1 October 2015, the Open Source Center was re-designated the Open Source Enterprise and incorporated into CIA’s new Directorate of Digital Innovation (DDI). Recognizing the connection that PAI and open sources have with innovation in the digital age is vital to the IC’s ability to embrace and leverage the vastness of PAI data available for analysis.

Start With the Obvious
Open sources of information have historically represented a fundamental consideration for the IC in our pursuit of knowledge. PAI represents the point of departure for additional, more traditional intelligence collection requirements. In The Five Disciplines of Intelligence Collection, Eliot A. Jardines, the first Assistant Deputy Director of National Intelligence for Open Source (ADDNI/OS), explains that OSINT “is frequently described within intelligence circles as ‘the source of first resort’ due to its ubiquitous nature and its ability to be shared broadly.” By definition, OSINT is publicly available information that represents intelligence value and therefor transitions from information to intelligence. PAI data, as the foundation of knowledge and basis for open sources, should be the first place analysts look for answers to our most challenging problems. The amount of data available publicly is astonishing. Digital technology that connects the world and the exponential growth of PAI data has become such a vast resource of potential intelligence that it has received considerable attention within the IC and AF ISR Enterprise. The personal computer, smart phones, smart watches, tablets, personal assistant devices, the infrastructure upon which they communicate, and the resulting connectivity they provide, has dramatically increased the amount of PAI data available for analysis. Acknowledging PAI as foundational data that potentially leads to valuable OSINT, the IC and AF ISR need to work together to raise awareness and prepare analysts to take advantage of the opportunities PAI represents.

 “We may not know where we are or where we are going, but we have to go.”
Dr. Debasish Mridha, Author

The majority of society isn’t trained or aware of Operations Security (OPSEC) considerations when sharing or posting information on social media sites and the internet. Information shared in the public domain is not protected or private and thus becomes available for manipulation and aggregation. The information age and technology have combined to provide analysts the capability to search and synthesize PAI to improve analysis efficiency of processing, exploitation and fusion of data into actionable intelligence. PAI as an initial single source of information provides the ability to tip and cue analysts to influence collection requirements tasked to traditional ISR platforms and sensors to gather amplifying data that will provide high confidence intelligence assessments.

A North Atlantic Treaty Organization (NATO) Research Paper states, “In the collection process, open sources set the information stage, whereas secret (classified) intelligence tends to explain the behavior of the main actors.” Consider a theatrical production as a metaphor for the collection process explained above. Open sources of information represent what we see on-stage when the curtain raises; furniture, elaborate backgrounds, props, lighting effects, and actors in costume. We gain insight into the dimension of time (historical, present or future), location (bedroom, kitchen, dining room, street corner, office, factory), and culture (European, American, Asian) – contextual information that “sets the stage.” Before the first line is recited or physical movement of an actor is gestured, we instinctively wonder and search for additional information that will provide perspective. Classified intelligence represents the words spoken, the facial expressions as characters interact, non-verbal gestures, and movement of actors upon the stage – amplifying information that would “explain the behavior of the main actors.” Unclassified data and open sources provide a foundation of knowledge and serves as a point of departure for our desire to know more – more than is available from PAI alone. We need to supplement our knowledge with classified information to truly understand what is currently happening, and most importantly, to manage collection of additional information that will provide the means to assess what may happen in the future.

PAI represents the ability to identify gaps in traditional ISR collection and direct limited assets to meet emerging intelligence and USAF mission requirements. Developing a comprehensive training program for ISR analysts to recognize PAI as a primary source of data will enable us to better prioritize tasking of traditional ISR collection platforms and sensors to more efficiently deliver decision advantage. The software tools we need to aggregate and analyze PAI are available now. Our potential ability to systematically incorporate PAI data into existing collection management and fusion processes will illustrate its value and deserves our attention.

Fusion, defined for the joint intelligence community in Joint Publication 2-0, is “the process of managing information to conduct all-source analysis and derive a complete assessment of activity.” The volume of classified intelligence data provided by traditional ISR collection platforms is abundant and often considered overwhelming for analysts to absorb and fuse into actionable intelligence. Inclusion of PAI will increase the volume, variety and velocity of potential intelligence data. More important than its value as complementary intelligence data, PAI has the ability to help direct our collection efforts and focus our need for additional intelligence sources that will enable efficient fusion. NATO recognizes the value of PAI to fusion when acknowledging, “Fusion of open source and social media analysis with existing intelligence from other disciplines guarantees all-source available products.”

Dr. Joseph Nye, former Chairman of the National Intelligence Council, is referenced in the Open Source Intelligence Professional Handbook as using a jig-saw puzzle analogy to explain the relationship between open and classified sources. “He likened open sources to the outer pieces of the puzzle, without which you can neither begin nor complete the puzzle. Classified sources, however, were essential to fill in the hardest to understand middle of the puzzle, and to complete the picture.” Considering the puzzle analogy, PAI (open sources) can be used as the foundation (outer pieces of the puzzle) upon which classified information (middle pieces of the puzzle) is gathered and fused to conduct analysis and produce intelligence (completion of the puzzle / picture). Some of the most valuable PAI available to the IC is data posted by billions of users utilizing the internet and social networking sites.

 

OTH, multi-domain operations, emerging security environment

 Data, Tools and Tradecraft
The Pew Research Center has identified the most popular social media sites based upon number of active users, as Facebook, Instagram, Pinterest, LinkedIn, and Twitter. PAI is more than social media. Multiple other sources of data includes text messages, images, news, government reports, scientific and technical research, blogs, academic information, and video. To provide perspective, consider these conversation data statistics provided by Bernard Marr:

  • More than a billion tweets are sent every 48 hours.
  • One million accounts are added to Twitter every day.
  • Every sixty seconds, 293,000 status updates are posted on Facebook.
  • 72% of online adults use social networking sites.
  • 25% of Facebook users never bother with any kind of privacy control.
  • The average Facebook user creates 90 pieces of content including links, news stories, photo albums, notes, and videos each month.

The most efficient means to search and consolidate the vast amounts of data generated by today’s most popular sites and digital communication forums is to team with existing commercial software providers. Many commercially available internet search software tools exist that are capable of providing very specific data based on user defined search criteria. The USAF Intelligence Systems Support Office (ISSO) produced Aztek Warrior Assessment 3.0 in May 2017. Findings were based on a rapid lab assessment of 31 software technologies and their ability to respond to 97 questions that illustrate their capacity to support military intelligence requirements. Specifically, software providers were compared to assess their capabilities to support Indications and Warning, Battle Damage Assessment, Force Protection, Operations Security, Targeting, Information Operations, Language, and Data Specifications. Some of the criteria considered included data quality / variety, data import / export capability, advanced network and pattern analysis, model development, location inference, retroactive search capability, and image exploitation and analysis. As technology advances and software capabilities improve, continuous assessment of tools and their capability to meet military requirements will be increasingly important. The ISSO Assessment has narrowed the field of available tools by identifying the most effective and efficient. Moving from a lab environment to the field has illustrated both function and value to analysts and users.

 “There is no avoiding the realities of the information age. Its effects manifest differently in different sectors, but the drivers of speed and interdependence will impact us all. Organizations that continue to use 20th-century tools in today’s complex environment do so at their own peril.”
General (ret) Stanley A. McChrystal, USA

Utilizing the results of Aztec Warrior Assessment 3.0, the AF Open Source Working Group (AFOSWG) and its Tools and Tradecraft Community of Practice has worked closely with analysts to confirm which software tools are most effective. Negotiations are currently being conducted to facilitate purchase of bulk licenses for the best software considering data aggregation, search, and filter capabilities that meet AF ISR mission requirements. Selection of a comprehensive toolset will provide consistent training across the entire AF ISR enterprise, providing focus and consolidating analysts’ efforts to develop tactics, techniques, and procedures (TTP). AFOSWG’s recommendation is to purchase licenses for Department of Defense approved Army Research Lab designed software tools that are currently being used by Special Operations Command, the US Army, and the Marine Corps. Software selection allows us to focus on tool integration and training.

Our youngest Airmen are familiar with the most popular social networks because they use them, daily. They have been instrumental in their development and are familiar with the connectedness they provide. Analyst training will require awareness of approved tools and tradecraft. Recent examples or vignettes that illustrate PAI’s functionality as beneficial to fusion analysis will provide increased confidence in assessments. Standardized training for specific tools and comprehensive tradecraft will provide knowledge and skills required by analysts to efficiently request, filter and analyze data effectively. We must also provide training to raise awareness of adversary capabilities to use disinformation and deception techniques, “fake news,” to decrease our confidence in PAI as valid and accurate. AF ISR analysts utilizing PAI must become sensitive to adversary spoofing and information operations, remaining ever mindful of the old adage: trust but verify.

 “Tell me and I forget. Teach me and I remember. Involve me and I learn.”
Benjamin Franklin

The next generation of ISR dominance will require an ability to conduct operations in contested environments, under fiscal constraints with reduced operating budgets, and in cooperation with joint, coalition and allied partners. Considering the environment that is described in much of the Air Force future operating scenarios, we should not expect to have the autonomy of uncontested airspace that we have enjoyed for the past 17 years of conflict. Without freedom of flight to collect information with traditional ISR platforms and sensors, we will have to rely on space based systems. Unless we consider the possibility that space also becomes a contested environment. Assuming our persistent collection capability is disrupted by adversary air and space defenses, we should prepare now to establish and enhance our PAI analysis capability. An adversary’s integrated air defense system will not prevent data from being generated, collected or analyzed. Reduced military budgets will likely prevent manpower growth, so we must efficiently improve analysis with existing end strength.

Improving PAI and open source analysis is not a cheap endeavor, but it is likely much cheaper than new or additional airborne platforms and sensors. Redirecting Airmen from core mission responsibilities for formal training comes with opportunity costs. We should consider new or additional training requirements as an investment in improved capability that will prepare us to provide improved analysis and collection management. A good first step to improving analysis capability is awareness and training. All Airmen should be familiar with PAI as a potential data source for US and adversary manipulation. Ignorance is a poor excuse for operations security (OPSEC) risk to US forces as our digital interaction continues to expand with improving technology.

Lieutenant Colonel David R. Lands is an intelligence officer for the Virginia Air National Guard. Lieutenant Colonel Lands has commanded 204 combat missions as an Intelligence, Surveillance, and Reconnaissance Mission Commander in support of Operations IRAQI FREEDOM, ENDURING FREEDOM, NEW DAWN, and INHERENT RESOLVE. 

 

The views expressed are those of the author and do not necessarily reflect the official policy or position of the Department of the Air Force or any organization of the US government.

Further Reading:
Elliot A. Jardines, “Open Source Intelligence,” in The Five Disciplines of Intelligence Collection, ed. Mark M. Lowenthal and Robert M. Clark (Los Angeles, CAL CQ Press, 2016).

Barnard Marr, Big Data: Using Smart Big Data, Analytics and Metrics to Make Better Decisions and Improve Performance (West Sussex, UK: Wiley, 2015).

Agnia Grigas, Beyond Crimea: The New Russian Empire (New Haven, CT: Yale University Press, 2016).

OTH, multi-domain operations, emerging security environment

Leave a Reply