Machine Intelligence to Detect, Characterise, and Defend against Influence Operations in the Information Environment
Abstract:
Deceptive content—misleading, falsified, and fabricated—is routinely created and spread online with the intent to create confusion and widen political and social divides. This study presents a comprehensive overview of content intelligence capabilities (WatchOwl– https://watchowl. pnnl.gov/) to detect, describe, and defend against information operations on Twitter as an example social platform to explain the influence of misleading content diffusion and enable those charged with defending against such manipulation and responsive parties to counter it. We first present deep learning models for misinformation and disinformation detection in multilingual and multimodal settings followed by psycho-linguistic analysis across broad deception categories. We then analyse user behaviour and spread patterns while engaging with deceptive content, characterise the vulnerable subpopulations and their demographics, and explicitly measure speed and scale of diffusion to uncover who shares content, how quickly, how much, and how evenly. We measure audience reactions to misinformation and disinformation, distinguishing the reactions of users identified as bots versus humans. Finally, we take advantage of deep machine translation and text generation models to create unique solutions for real-time defence against digital deception and discuss how to apply causal discovery and inference to intervene into information campaigns jointly across information, cognitive, and physical domains.
AUTHORS
Aptima, Inc.
Woburn, Massachusetts United States of America
Svitlana Volkova is Chief of AI in the Office of Science and Technology at Aptima, Inc. She spearheads the company's initiatives to develop trustworthy and human-centric compound AI systems that tackle complex real-world problems for the Department of Defense and other government agencies. Volkova's research focuses on advancing natural language processing and machine learning techniques, with an emphasis on graph neural networks, causal inference, and multimodal frontier models. Her work has pioneered methods for AI-powered analytics to explain complex social systems and behaviors. She received her PhD in computer science from Johns Hopkins University, and MS degrees in computer science from Petro Mohyla Black Sea State University, Ukraine, and Kansas State University.
Data Sciences and Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States
Dr. Maria Glenski is a Data Scientist in the Data Science and Analytics Group, National Security Directorate at the Pacific Northwest National Laboratory. Her research focuses include computational social science approaches to behavior analysis, characterization, and modeling of complex social behavior in diverse online social environments and explainable artificial intelligence (XAI) evaluating the impacts of algorithmic biases in machine learning models. Dr. Glenski received her Ph.D. in Computer Science from the University of Notre Dame where she was an Arthur J Schmitt Leadership in Science and Engineering Fellow. Dr. Glenski’s research has been published in top tier venues including WWW, ACL, ACM TIST, and CSCW and she regularly serves on the program committee of several international conferences and journals.
Data Sciences and Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States
Ellyn Ayton is a Data Scientist at PNNL. She received her Master’s in Computer Science from Western Washington University. Her research areas of interest include deep learning and its many applications, such as detecting digital deception. She contributes to projects that use NLP to extract predictive signals from open source data, evaluates the effectiveness of causal mechanisms in machine learning models, and develops methods of interpretability and explain ability of black-box deep learning models.
Data Sciences and Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States
Dr. Emily Saldanha is a research scientist in the area of data science at PNNL where her work focuses on applying machine learning and deep learning techniques to identify and understand patterns in complex data. She has specific interests in the development of robust methods for application areas ranging from energy technologies to computational social science. She received her Ph.D. in physics from Princeton University in 2016, where her work focused on the development and application of calibration algorithms for microwave sensors for cosmological observations.
Data Sciences and Analytics -Pacific Northwest National Laboratory
Richland, Washington, United States
Joshua Mendoza is currently a Data Scientist in the National Security Directorate at Pacific Northwest National Laboratory. His work focuses on the application of deep learning for natural language processing and computer vision problems. Joshua received his bachelor’s degree with honors in 2015 from the University of Washington.
Pacific Northwest NationalLaboratory Foundational Data, Science Group
Richland, Washington, United States
Dustin Arendt received his Ph.D. from Virginia Tech in 2012 where he researched social network analysis and modeling and simulation of complex systems. Challenges inherent to understanding complex systems led him to a career in visual analytics. Since joining PNNL, he has worked in several domains including visualization for cybersecurity, streaming data visual analytics, visual abstraction, dynamic graph visualization, visualization for natural language processing, interactive machine learning, explainable machine learning, and visualization for machine learning model validation and comparison. Currently, his interests are at the intersection of human-computer interaction, data science, and visual analytics with a focus on validating machine learning models through explanations and exploratory data analysis. His research involves rapid prototyping and empirical evaluation of tools that blend machine learning, data science, and visualization.
Pacific Northwest National LaboratoryHuman Centered Computing Group
Richland, Washington, United States
Zhuanyi H Shaw is a user experience designer. Using her MS in Design, she focuses on human computer interaction, conducts user research to identify target users and to gather user requirements, transforms the requirements into user experience design, and brings the concept back to users for concept evaluation. She specializes in the UX of AI and data visualization.
Mission Alignment - Pacific Northwest National Laboratory
Richland, Washington, United States
Kevin Cronk is a Nuclear Engineer and MBA, serving as a Technical Advisor within the National Security Directorate at Pacific Northwest National Laboratory. Prior to joining PNNL in 2018, Kevin served as a program manager at the Department of Homeland Security’s Domestic Nuclear Detection Office (now CWMD) and was a project engineer at the US Nuclear Regulatory Commission. Kevin served as a Nuclear Engineer in the United States Navy.
Operations and Analysis - Pacific Northwest National Laboratory
Richland, Washington, United States
Sara A. Smith is a cyber security analyst at PNNL. Her current work focuses on research applications, advanced analysis, and operational expertise in defense, counterterrorism, open source intelligence, and computational propaganda issues. Ms. Smith holds a Bachelor of Science in Life Science from the United States Military Academy. Prior to joining PNNL, Sara served as an Intelligence Officer in the U.S. Army.
Mission Alignment - Pacific Northwest National Laboratory
Richland, Washington, United States
Dr. Mark Greaves is currently Technical Director for Analytics in the National Security Directorate at PNNL. Prior to coming to PNNL, Mark was Director of Knowledge Systems at Vulcan Inc., Director of DARPA’s Joint Logistics Technology Office, and Program Manager in DARPA’s Information Exploitation Office. Mark began his career at Boeing, where he worked on advanced programs in software agent technology. He has published two books and over 40 papers, holds two patents, chaired the FIPA technical committee on agent communications languages, and was awarded the Office of the Secretary of Defense Medal for Exceptional Public Service for his contributions to US national security while serving at DARPA. Mark holds a PhD from Stanford University.
Published In
Journal of Information Warfare
The definitive publication for the best and latest research and analysis on information warfare, information operations, and cyber crime. Available in traditional hard copy or online.
Quick Links
Archive
