Adversarial AI and Polymorphic Malware: A New Era of Cyber Threats

The state of cybersecurity has always been in flux, but the arrival of tools like ChatGPT heralded one of the most significant challenges for security teams in years. AI has the potential to unlock incredible potential in data processing and malware detection, but in the wrong hands, Large Language Models (LLMs) and other adversarial AI tools can be used to develop polymorphic malware that can escape detection, gain access to sensitive data, and poison data sets.

While these attacks pose significant risks to the security of your infrastructure, they’re not insurmountable. Here, we’ll explore what adversarial AI and polymorphic malware do and how you can respond to these new and evolving threats.

What is adversarial AI?

Adversarial AI and adversarial ML are attack methods designed to infiltrate an organization’s AI and ML systems. They can also counteract the AI-based cybersecurity systems in place to prevent intrusions. 

According to NIST, adversarial AI does this through “the process of extracting information about the behavior and characteristics of an ML system and/or learning how to manipulate the inputs into an ML system in order to obtain a preferred outcome.” Essentially, that means manipulating the inherent logic and decision-making processes that AI systems rely on with specific attacks.  These attacks include:

Model-based attacks

The algorithms and data that power artificial intelligence models are often highly protected trade secrets. If your model can process information faster and more reliably, it will gain a substantial competitive advantage. Because of this, malicious actors will often look for ways to reproduce your model through model extraction attacks.

A model extraction attack occurs when an attacker attempts to send numerous queries into your model and monitor its output. With enough information, an attacker could find a way to clone the model in part or whole and use it for their own gain. 

Similarly, a model inversion attack uses data inputs to reverse-engineer outputs, allowing attackers to predict and extract data from the model. An attacker could use this tactic to steal personally identifiable information, financial data, and other proprietary information.

Data poisoning

AI models take vast amounts of data to work correctly. Malicious actors can inject poisoned data into these models during model training to either sabotage them or create a backdoor that provides access to sensitive or proprietary information. 

For example, an attacker could upload images incorrectly tagged as stop signs into a self-driving automobile’s detection systems to make its algorithm less accurate. For cybersecurity systems that rely on attack samples to detect intrusions, a malicious actor could inject poisoned data to create false positives that make real cyber attacks far more difficult to detect.

A real-world example of this process is Nightshade, a defensive tool that injects poison into images to protect artists from AI models that scan their work without permission. This poison is invisible to the human eye but can cause AI models that ingest it to break down and become virtually useless.

Evasion attacks

Evasion attacks are similar to data poisoning attacks, except that they’re used on a model that has already been trained. They rely on imperfections in the model to inject misclassified data to avoid detection. For example, a malicious actor could upload doctored images or messages in emails to evade spam-detection filters or rely on tweaked malware to get past an organization’s automated detection filters.

What is polymorphic malware, and how can AI enhance its capabilities?

Polymorphic malware is a type of attack that can continuously change its key characteristics, such as its file name, size, location, signature files, encryption keys, and other details, in order to stay undetected. 

These files are problematic because by the time traditional detection algorithms have amassed enough information to sniff them out, the polymorphic malware has likely already changed its makeup. 

Polymorphic malware has existed in some form since the 1990s and has become such an ingrained part of a cyber attacker’s toolkit that nearly all forms of malware are polymorphic in some way. 

Since the advent of ChatGPT, AI-powered malware-creation tools have also cropped up, designed to speed up polymorphic malware generation and make it more resilient to other AI and ML-based detection routines. One of the more prominent examples is BlackMamba. This keylogging attack changes its code every time it executes to evade detection by automated cybersecurity systems.

The availability and immediacy of LLMs mean that these tools are becoming more robust and easier to create. For example, GhostGPT allows attackers to generate unfiltered responses to prompts requesting malicious code from large language models without the need for a jailbreak. MIT Technology Review also believes that malicious AI agents are on the way — that the same tools designed to help individuals with complex tasks could also be used to attack infrastructure and adapt its strategies to bypass cybersecurity countermeasures.

How to mitigate the risks of adversarial AI and polymorphic malware

Much of the cybersecurity knowledge you’ve already learned and cultivated will continue to serve you well against adversarial AI and polymorphic malware. Keep these tips in mind:

• Leverage malware detection tools. While adversarial AI and polymorphic malware may seem brand new, they still rely on many of the same techniques and tactics malware has used for years. Automated ML-based malware detection tools with continuous monitoring capabilities will give you a complete picture of the health and security of your infrastructure, while endpoint detection and response systems will scan employee devices across your infrastructure for threats. Follow up with a manual response to confirm whether or not your system is compromised.
• Lean on cybersecurity fundamentals. Even as attacks evolve, the basics remain a solid foundation for maintaining a robust security stance. Use zero trust techniques like multi-factor authentication and identity access management to limit unauthorized access and regularly verify credentials. Limit API calls to AI models to prevent evasion attacks. And, of course, continue to train fellow employees on cybersecurity best practices to prevent mistakes that could lead to an attack.
• Perform data sanitization and cleaning processes on a regular basis. Since data poisoning attacks happen during the training of massive amounts of data, they can be challenging to detect if your team isn’t vigilant. DHS recommends regular analysis of your datasets and cleaning any corrupted, duplicate, incomplete, or otherwise suspicious information as necessary to keep AI models in working order.