Hostile requests are specially worded instructions that can bypass the safety mechanisms of language models and trigger unwanted actions. Such requests are dangerous because they let attackers manipulate the model to reach their own goals. This can include:
- Extracting confidential information. Some requests aim at hidden data, which is especially risky when the model works with personal or corporate information. The model may accidentally reveal details that were supposed to stay secret.
- Bypassing ethical limits. Language models often have built-in filters that keep them from answering requests tied to violence, illegal actions or discrimination. But if a request is worded so that it slips past these limits, the model may answer such questions anyway.
- Producing an unexpected or wrong result. Attackers can deliberately word a request so that the model returns an incorrect or absurd answer. This can be used for disinformation or simply to cause chaos.
Hostile requests undermine the reliability and safety of models, so it is important to develop and deploy mechanisms that filter and prevent them.