Machine Intelligence to Detect, Characterise, and Defend against Influence Operations in the Information Environment

Abstract:

Deceptive content—misleading, falsified, and fabricated—is routinely created and spread online with the intent to create confusion and widen political and social divides. This study presents a comprehensive overview of content intelligence capabilities (WatchOwl– https://watchowl. pnnl.gov/) to detect, describe, and defend against information operations on Twitter as an example social platform to explain the influence of misleading content diffusion and enable those charged with defending against such manipulation and responsive parties to counter it. We first present deep learning models for misinformation and disinformation detection in multilingual and multimodal settings followed by psycho-linguistic analysis across broad deception categories. We then analyse user behaviour and spread patterns while engaging with deceptive content, characterise the vulnerable subpopulations and their demographics, and explicitly measure speed and scale of diffusion to uncover who shares content, how quickly, how much, and how evenly. We measure audience reactions to misinformation and disinformation, distinguishing the reactions of users identified as bots versus humans. Finally, we take advantage of deep machine translation and text generation models to create unique solutions for real-time defence against digital deception and discuss how to apply causal discovery and inference to intervene into information campaigns jointly across information, cognitive, and physical domains.


AUTHORS

Photo of Dr. Svitlana Volkova

Data Sciences and Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States

Dr. Svitlana Volkova is a recognized leader in the field of computational linguistics, machine learning and computational social science. She leads the development of human-centered analytics to explain, predict and prescribe human social systems and behaviors as they relate to national security challenges in the human domain. Solutions developed by Svitlana and her team advance understanding, analysis, and effective reasoning about extreme volumes of dynamic, multilingual, multimodal real-world social data. Since joining PNNL in 2015, Dr. Volkova has led more than ten projects, including two DARPA efforts. She authored more than 50 peer-previewed conference and journal publications. Svitlana was a Vice Chair of the ACM Future of Computing Academy between 2017 and 2019. She received her PhD in Computer Science in 2015 from Johns Hopkins University where she was affiliated with the Center for Language and Speech Processing and the Human Language Technology Center of Excellence.

Photo of Dr. Maria Glenski

Data Sciences and Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States

Dr. Maria Glenski is a Data Scientist in the Data Science and Analytics Group, National Security Directorate at the Pacific Northwest National Laboratory. Her research focuses include computational social science approaches to behavior analysis, characterization, and modeling of complex social behavior in diverse online social environments and explainable artificial intelligence (XAI) evaluating the impacts of algorithmic biases in machine learning models. Dr. Glenski received her Ph.D. in Computer Science from the University of Notre Dame where she was an Arthur J Schmitt Leadership in Science and Engineering Fellow. Dr. Glenski’s research has been published in top tier venues including WWW, ACL, ACM TIST, and CSCW and she regularly serves on the program committee of several international conferences and journals.

Photo of Ellyn Ayton

Data Sciences and Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States

Ellyn Ayton is a Data Scientist at PNNL. She received her Master’s in Computer Science from Western Washington University. Her research areas of interest include deep learning and its many applications, such as detecting digital deception. She contributes to projects that use NLP to extract predictive signals from open source data, evaluates the effectiveness of causal mechanisms in machine learning models, and develops methods of interpretability and explain ability of black-box deep learning models.

Photo of Dr. Emily Saldanha

Data Sciences and Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States

Dr. Emily Saldanha is a research scientist in the area of data science at PNNL where her work focuses on applying machine learning and deep learning techniques to identify and understand patterns in complex data. She has specific interests in the development of robust methods for application areas ranging from energy technologies to computational social science. She received her Ph.D. in physics from Princeton University in 2016, where her work focused on the development and application of calibration algorithms for microwave sensors for cosmological observations.

Photo of Joshua Mendoza

Data Sciences and Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States

Joshua Mendoza is currently a Data Scientist in the National Security Directorate at Pacific Northwest National Laboratory. His work focuses on the application of deep learning for natural language processing and computer vision problems. Joshua received his bachelor’s degree with honors in 2015 from the University of Washington.

Photo of Dr. Dustin Arendt

Visual Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States

Dr. Dustin Arendt is a Senior Research Scientist and Team Lead in the Visual Analytics Group at Pacific Northwest National Laboratory, joining the lab in October 2014. He received his Ph.D. from Virginia Tech in 2012 and completed a postdoc at the Air Force Research Laboratory from 2012 to 2014 where he researched graph visualization and applied machine learning. Since joining PNNL he has worked in several domains including visualization for cybersecurity, streaming data visual analytics, visual abstraction, dynamic graph visualization, visualization for natural language processing, interactive machine learning, explainable machine learning, and visualization for machine learning model validation and comparison. Currently, his interests are at the intersection of human-computer interaction, data science, and visual analytics, with a focus on validating machine learning models through explanations and exploratory data analysis. His research involves rapid prototyping and empirical evaluation of tools that blend machine learning, data science, and visualization.

Photo of Zhuanyi H Shaw

Visual Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States

Zhuanyi H Shaw is a user experience researcher and designer in PNNL's Visual Analytics group. Using her MS in Design, she focuses on human factors, conducts user research to identify target users and gather user requirements; transforms the requirements into user experience design; bring the concept back to users for concept evaluation. While at PNNL, she has worked on various projects for information analysts and visualization within the domains of Bio surveillance, social media, smart grid, smart building and infrastructure protection.

Photo of Kevin Cronk

Mission Alignment - Pacific Northwest National Laboratory
Richland, Washington, United States

Kevin Cronk is a Nuclear Engineer and MBA, serving as a Technical Advisor within the National Security Directorate at Pacific Northwest National Laboratory.  Prior to joining PNNL in 2018, Kevin served as a program manager at the Department of Homeland Security’s Domestic Nuclear Detection Office (now CWMD) and was a project engineer at the US Nuclear Regulatory Commission.  Kevin served as a Nuclear Engineer in the United States Navy. 

Photo of Sara A. Smith

Operations and Analysis - Pacific Northwest National Laboratory
Richland, Washington, United States

Sara A. Smith is a cyber security analyst at PNNL. Her current work focuses on research applications, advanced analysis, and operational expertise in defense, counterterrorism, open source intelligence, and computational propaganda issues. Ms. Smith holds a Bachelor of Science in Life Science from the United States Military Academy. Prior to joining PNNL, Sara served as an Intelligence Officer in the U.S. Army.

Photo of Dr. Mark Greaves

Mission Alignment - Pacific Northwest National Laboratory
Richland, Washington, United States

Dr. Mark Greaves is currently Technical Director for Analytics in the National Security Directorate at PNNL. Prior to coming to PNNL, Mark was Director of Knowledge Systems at Vulcan Inc., Director of DARPA’s Joint Logistics Technology Office, and Program Manager in DARPA’s Information Exploitation Office.  Mark began his career at Boeing, where he worked on advanced programs in software agent technology. He has published two books and over 40 papers, holds two patents, chaired the FIPA technical committee on agent communications languages, and was awarded the Office of the Secretary of Defense Medal for Exceptional Public Service for his contributions to US national security while serving at DARPA.  Mark holds a PhD from Stanford University.

Journal of Information Warfare

The definitive publication for the best and latest research and analysis on information warfare, information operations, and cyber crime. Available in traditional hard copy or online.

Quick Links

View the latest issue of JIW.

Latest Edition

Purchase a subscription to JIW.

Subscribe

Keywords

C

C2
C2S
CDX
CIA
CIP
CPS

D

DNS
DoD
DoS

I

IA
ICS

S

SOA

X

XRY

Quill Logo

The definitive publication for the best and latest research and analysis on information warfare, information operations, and cyber crime. Available in traditional hard copy or online.

SUBSCRIBE NOW

Get in touch

  • Journal of Information Warfare
    21 North Broad Street
    Suite 2-H
    Luray, VA 
    22835 
  • 757.581.9550
  • JIW@ArmisteadTec.com