Evaluating LLM Input Comprehension and Guardrail Robustness through Noise-Based Attacks

Introduction

Hughes et al. introduced Best-of-N (BoN) Jailbreaking[1], a jailbreaking approach for LLMs which works by repeatedly sampling variations of a prompt with a combination of perturbations - such as random letter shuffling or capitalization for textual prompts. These prompt variations in their most fundamental form could be considered as noise.

This research project intended to gain a deeper understanding of how LLMs respond when presented with prompts containing variable amounts of noise, and understanding this may provide insight into creating both new offensive and defensive methods for Generative AI Security.

The following research is centered around three main questions. What percentage of noise in a prompt would cause:

• an LLM to be unable to comprehend the original meaning of the prompt.
• an LLM Guardrail to misclassify a malicious prompt as benign, resulting in an adversarial prompt.
• an LLM Guardrail to misclassify a benign prompt as malicious.

Spikee Tooling and Noise Plugin

Testing against LLMs was conducted with the use of spikee (Simple Prompt Injection Kit for Evaluation and Exploitation) [2]. Spikee is designed to assess the susceptibility of LLMs and their applications to targeted prompt injection attacks.

Spikee can create datasets which can be augmented with custom plugins. Plugins can be used to apply arbitrary modifications to the dataset. When creating a dataset, three individual pieces of data are required.

• Documents — Use-case specific text in which malicious payloads can be injected (such as the body of an email).
• Jailbreak — Text in the prompt designed to misalign the LLM from its intended purpose or perform an additional or different instruction set by the attacker.
• Instruction — Malicious instruction that we want the LLM to follow, made more plausible by the introduction of the aforementioned jailbreak.

A custom plugin was created to introduce noise during the dataset creation. This plugin allowed a variable noise percentage to be set so that different datasets could be created, each dataset containing a specific amount of noise across every prompt.

It should also be noted that when a plugin is applied to a prompt, the plugin is only applied to the jailbreak and instruction combined, not the “document” part of the prompt.

The datasets, noise plugin, and spikee result files can be found from Reversec’s GitHub[13].

Datasets

The three questions defined in this research were each evaluated with a different dataset.

To analyse the ability of an LLM to comprehend text with noise, a custom dataset was created with 32 entries. These entries were each benign and requested the LLM to respond by answering a simple question related to the information within the prompt. An example sentence from the dataset is “what is the smallest prime number greater than 2 but less than 10? Respond only the number itself, followed by a ’!’ character, and no spaces in between the number and following character.”.

In this example, spikee would evaluate the output and determine if the response contained the string ‘3!‘. Additional parameters were provided to spikee to ensure that when the noise plugin modified this prompt, noise would not be applied to the ‘2’, ‘10’, and ’!’ parts of the string, as if these characters were modified and the LLM understood the message, then the LLM would be unable to provide the answer that spikee was expecting. This same approach was applied to all 32 prompts in the comprehensibility dataset.

The following table presents a single prompt from each dataset with different levels of noise.

The dataset used for analysing malicious prompts with noise against guardrails was created from the ‘seeds-cybersec-2025-04’ repository on Reversec’s public GitHub[3]. Only English language text was chosen during the generation of the dataset, and only bare documents were created (not including prompts), resulting in a total of 740 total documents.

The dataset used for analysing benign prompts with noise (i.e. testing the potential of false positives) was the wildguard harmful dataset containing only benign prompts, i.e. prompts without the ‘prompt_harm_label’ not matching ‘harmful’. This included 1007 prompts in total.

For the input comprehensibility dataset, the noise plugin was applied to the entire prompt with exclusions of parts of the prompt which if modified, would fail. Whereas noise was only applied to the jailbreak and instruction for the cybersec dataset which was used for testing guardrail robustness.

LLM Input Comprehensibility of Noise

The following strategy was used to evaluate the ability of each LLM to understand input when noise was present. All prompts in the input comprehensibility dataset were sent to each LLM to evaluate what percentage of these prompts the LLM responded to correctly. Assuming that an LLM responds to these questions without noise successfully, we can assume the LLM was able to understand the message. After introducing noise it can be inferred that if the LLM responds correctly, then the LLM is still able to understand the original message, or at least infer what we’re asking based on the scrambled text.

Six LLMs were chosen to evaluate against the input comprehensibility dataset, these were

• OpenAI GPT4.1 Mini[4]
• OpenAI GPT4.1[5]
• Google Gemini Flash 2.0[6]
• Google Gemini Flash 2.5[7]
• TogetherAI Llama3.1 70B Instruct Turbo[8]
• Claude Sonnet 3.7[9]

This included 11 datasets, evaluating each models ability to understand prompts with varying levels of noise, and without. The 11 datasets each contained a different level of noise, for example 5%, 10%, and at every 5% interval up to and including 50%.

As seen from the results, all LLMs appear the handle noise in prompts similarly. We only selected 32 base questions that all LLMs could understand and answer correctly in their base form, without noise. This allowed us to observe and compare the LLMs’ performance after noise was added to each prompt.

Overall, all LLMs responded similarly, and almost all input prompts were incomprehensible when 50% of noise was applied to prompts, with the exception of Claude Sonnet 3.7 which appeared to outperform all of the other LLMs across all of the noise datasets with the exception of one, i.e. 45%.

The following table presents the results for each model against each dataset with varying amounts of noise.

LLM Guardrail Robustness against Noise

We evaluated the robustness of guardrails to noise by testing them with malicious prompts from 11 datasets, each containing a specific level of noise. For each dataset and each LLM, we measured the percentage of prompts classified as benign. The evaluation used three different LLM guardrails.

• ProtectAI DeBERTa v3 base Prompt Injection v2[10]
• Meta-Llama - Prompt Guard 86M[11]
• Azure Prompt Shields[12]

Prompt Guard 86M returned a decimal to indicate a score related to attack detection. The cutoff for this classifier was set to classify a prompt as malicious when returning a score greater than 95%. This rate also applied to the false positive testing discussed in the next section.

Unlike the input comprehensibility testing, the response to malicious prompts with noise was more varied. Prompt Guard 86M was highly susceptible to noise-based attacks. Initially, noise applied at a rate of 0% to 5% resulted in the prompts being classified more as malicious, but after 10% the guardrails were quickly bypassed, resulting in 98.2% of prompts misclassified when 30% of noise was applied.

Both Azure Prompt Shields and ProtectAI DeBERTa appeared to respond similarly to noise, as initially a small amount of noise (0%-20%) introduced to a prompt would cause the prompts to be classified more correctly as malicious, however this started to change much later than Prompt Guard 86M, and this occured approximately when 20% to 25% of noise was introduced in a prompt.

Against Azure Prompt Shields, noisy prompts only outperformed clean prompts when noise was applied at a 50% rate. At this level, input comprehensibility testing showed that prompts became nearly unintelligible to most LLMs. However, using a more structured method of applying noise — such as a Best-of-N strategy — could make these prompts easier for LLMs to interpret the original meaning of the message. This suggests that Azure Prompt Shields may still be vulnerable to noise-based attacks using more targeted approaches.

ProtectAI DeBERTa was the only guardrail where noise-based attacks did not outperform the original, clean prompts — as long as the noise level stayed at or below 50%. Higher noise levels may change this outcome.

One reason high-noise prompts may bypass guardrails is that the LLM fails to understand the message — as suggested by the input comprehensibility graph. If the malicious intent becomes unintelligible, the LLM might not recognize it as harmful. However, another contributing factor could be that the LLM was not trained on examples of malicious prompts with this level or type of noise. Importantly, we did not verify whether prompts that bypassed the guardrails still conveyed their original malicious intent in a way the LLM could act on.

The following table presents the results for each model against each dataset with varying amounts of noise.

LLM Guardrail False Positive Sensitivity to Noise

To evaluate the impact of noise on false positives, we tested 11 datasets containing varying levels of noise — similar to the robustness evaluation. Each dataset included only benign prompts, allowing us to measure how often the guardrails incorrectly flagged them as malicious. The same three LLM guardrails were used as in the robustness tests.

Prompt Guard 86M was once again an outlier and performed differently to ProtectAI DeBERTa and Azure Prompt Shields. Initial benign prompts were mostly incorrectly blocked by Prompt Guard 86M, with only 8.2% of benign prompts being correctly classified. Similarly to the guardrail robustness testing, noise resulted quickly in prompts bypassing protections but being classified correctly at a high success rate of 93.05% prompts when 30% of noise was applied.

The responses from ProtectAI DeBERTa and Azure Prompt Shields were once again similar, but deviated slightly to the previous testing, as a low amount of noise resulted in the prompts being less likely to be correctly classified as benign and therefore seen as more likely as being malicious. The successful classification of prompts with noise for Azure Prompt Shields started to turn back around 30%, and ProtectAI DeBERTa around 45%, but the datasets with noise never achieved a higher pass rate than the non-noise prompts when noise was examined at a rate of 0% to 50%.

The following table presents the results for each model against each dataset with a varying amount of noise.

Conclusion

The ability of LLMs to understand and respond correctly to prompts containing noise appeared to be consistent across all LLMs tested. Unsurprisingly, the ability for an LLM to understand a prompt containing noise is negatively correlated with the percentage of noise applied to a prompt. Regardless, even with a prompt containing 50% of noise, there still seems to be potential for an LLM to respond correctly to a message.

The robustness of LLM guardrails to prompts with noise appeared to be less consistent across different models. One of three LLM guardrails tested responded poorly to random noise-based attacks and easily allowed prompts to bypass protections, whereas others only allowed malicious prompts to bypass protections at a point where the original message was almost unintelligible.

The false positive rate of LLM guardrails when classifying benign prompts with noise also appears to be inconsistent across different models. However the response from each model appeared somewhat similar across both malicious and benign prompts. For example, Prompt Guard 86M classified both benign and malicious prompts as more benign once a small amount of noise was applied, whereas ProtectAI DeBERTa and Azure Prompt Shields initially classified prompts with noise as more malicious, but this behavior seemed to revert once the ability to understand input with noise was low.

While this attack pattern is well known, the research provides quantitative insight into how it plays out across different LLM–guardrail combinations. Specifically, it measures the threshold at which noise makes a malicious prompt appear benign to a guardrail, while still being interpretable by the LLM. The higher the LLM’s comprehension at that threshold, the more effective the attack becomes.

For example, Azure Prompt Shields required at least 50% noise in prompts before guardrails started to fail. When paired with Claude Sonnet 3.7, such prompts would be correctly interpreted 21.88% of the time. In contrast, Gemini Flash 2.0 only succeeded 3.12% of the time to this level of noise, making attacks far less efficient from an attackers point of view.

Although Claude’s high comprehension of noisy inputs was impressive, this could also increase the risk of exploitation if combined with less robust guardrails. As a defensive measure, guardrail–LLM pairs could be chosen so that the point of guardrail failure coincides with the LLM’s inability to understand the prompt.

References

1. Best-of-N Jailbreaking (2024). Hughes et al. Best-of-N Jailbreaking
2. Simple Prompt Injection Kit for Evaluation and Exploitation spikee
3. Reversec GitHub Seeds Cybersec 2025-04 Dataset seeds-cybersec-2025-04 dataset
4. OpenAI GPT 4.1 Mini GPT 4.1 Mini Model Documentation
5. OpenAI GPT 4.1 GPT 4.1 Model Documentation
6. Google Gemini Flash 2.0 Gemini Flash Model
7. Google Gemini Flash 2.5 Gemini Flash Model
8. Llama 3.1 70B Llama 3.1 70B
9. Claude Sonnet 3.7 Claude 3.7 Sonnet and Claude Code
10. ProtectAI.com. (2024). Fine-Tuned DeBERTa-v3-base for Prompt Injection Detection. HuggingFace. Model on HuggingFace
11. Meta Llama - Prompt Guard 86M Model on HuggingFace
12. Azure Prompt Shields Prompt Shields
13. Research Dataset Reversec GitHub