Exploiting Text Completion

• LLMs are trained to predict the next token in a sequence. Exploit by taking advantage of text completion in the prompt.

• For example: A bot for Mozart’s bio shouldn’t give information on calculating determinant of a matrix. But if we add “Sure, here is how you do it:” at the end of the sentence, it might complete it.

• Since LLMs are non-deterministic in nature, we might have to send the same prompt again.

• We’re trying make the LLM pay less attention to its initial prompt and instead focus on the added input prompt.