Showing Posts About

AI

Multi-Chain Prompt Injection Attacks

Multi-chain prompt injection is a novel attack technique targeting complex LLM applications with multiple chained language models. The technique exploits interactions between LLM chains to bypass safeguards and propagate malicious content through entire systems. A sample workout planner application demonstrates how attackers can manipulate multi-chain LLM workflows to inject and propagate adversarial prompts across different processing stages.

Fine-Tuning LLMs to Resist Indirect Prompt Injection Attacks

A fine-tuning approach was developed to enhance Llama3-8B's resistance to indirect prompt injection attacks. The method uses data delimiters in the system prompt to help the model ignore malicious instructions within user-provided content. The fine-tuned model achieved a 100% pass rate in resisting tested prompt injection attacks. The model and training scripts have been publicly released.

When your AI Assistant has an evil twin

An indirect prompt injection attack against Google Gemini Advanced demonstrates how malicious emails can manipulate the AI assistant into displaying social engineering messages. The attack tricks users into revealing confidential information by exploiting Gemini's email summarization capabilities. The vulnerability highlights potential security risks in AI assistants with data access capabilities.

Generative AI - An Attacker's View

Generative AI is increasingly being used by threat actors for cyber attacks. Attackers can leverage AI for reconnaissance, gathering personal information quickly and creating targeted phishing emails. The technology enables sophisticated social engineering through deepfakes, voice cloning, and malicious code generation, with potential for more advanced attacks in the near future.