Stop Sequences

Stop sequences are specific tokens or phrases that signal to a language model to halt its output generation. By strategically incorporating stop sequences, users can manipulate the model's behavior to create desired outcomes. For instance, using phrases like "[END]" or "[END OF TEXT]" can trick the model into believing that the user input has concluded, allowing for the introduction of new instructions or prompts without the model recognizing them as part of the ongoing conversation. This technique can be particularly useful in prompt injection scenarios, where the goal is to override the model's original instructions and gain control over its responses. By effectively utilizing stop sequences, users can navigate around the model's built-in safeguards and generate content that may otherwise be restricted.