TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification

Large Language Model (LLM) services and models often come with legal rules on *who* can use them and *how* they must use them. Assessing the compliance of the released LLMs is crucial, as these rules protect the interests of the LLM contributor and prevent misuse. In this context, we describe the novel problem of Black-box Identity Verification (BBIV). The goal is to determine whether a third-party application uses a certain LLM through its chat function. We propose a method called Targeted Random Adversarial Prompt (TRAP) that identifies the specific LLM in use. We repurpose adversarial suffixes, originally proposed for jailbreaking, to get a pre-defined answer from the target LLM, while other models give random answers. TRAP detects the target LLMs with over 95% true positive rate at under 0.2% false positive rate even after a single interaction. TRAP remains effective even if the LLM has minor changes that do not significantly alter the original function.

🪤 TRAP in a nutshell

🦹 Motivation

💧 Private LLMs that cost millions of dollars to train may be leaked by internal or external threats.
🐍 Open-source LLMs are distributed under restrictive licenses that may not be respected. For instances, Microsoft’s Orca-2 is distributed under a no-commercial licence, and Meta’s usage policy of Llama-2 forbids deceptive usages.
🎭 LLMs do not disclose reliably their identity. For instances, Mixtral-8x7B identifies it-self as FAIR’s BlenderBot 3.0, and we can disguise GPT-3.5 and GPT-4 as Anthropic’s Claude or as Llama-2, using deceptive system prompts.

Therefore, we need specific tools to ensure compliance.

🥷 Problem: Black-Box Identity Verification (BBIV)

A reference LLM (either close or open) can be deployed silently by a third party to power an application. So, we propose a new task, BBIV, of detecting the usage of an LLM in a third-party application, which is critical for assessing compliance.

Question: Does this third-party application use our reference LLM ?

🪤 Solution: Targeted Random Adversarial Prompt (TRAP)

To solve the BBIV problem, we propose a novel method, TRAP, that uses tuned prompt suffixes to reliably force a specific LLM to answer in a pre-defined way.

TRAP is composed of:

a closed-ended question
- 🔥 20 tunable tokens
- ⚙️ optimised on the
- 🎯 to output a specific chosen at random, here 314

Schema method

🍯 The final prompt is a honeypot:

The suffix forces the reference LLM to output the target number 95-100% of the time
The suffix is specific to the reference LLM (<1% average transfer rate to another LLM)
TRAP beats the perplexity baseline
- Using less output tokens (3-18 tokens vs. 150 tokens)
- Perplexity identification is sensible to the type of prompt

🛡️ Third-party can deploy the reference LLM with changes

TRAP is robust to generation hyperparameters (usual ranges)
TRAP is not robust to some system prompts

Robustness plot

Read the full paper for more details.

Citation

If you use our code or our method, kindly consider citing our paper:

@misc{gubri2024trap,
      title={TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification}, 
      author={Martin Gubri and Dennis Ulmer and Hwaran Lee and Sangdoo Yun and Seong Joon Oh},
      year={2024},
      eprint={2402.12991},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}