SolidGoldMagikarp (plus, prompt generation) - LessWrong
www.lesswrong.com
UPDATE (14th Feb 2023): ChatGPT appears to have been patched! However, very strange behaviour can still be elicited in the OpenAI playground, particularly with the davinci-instruct model. …

A deep dive into the inner workings of ChatGPT, and why it stops responding or replies weird or creepy things to seemingly simple requests.

  • 𝕊𝕚𝕤𝕪𝕡𝕙𝕖𝕒𝕟OPMEnglish
    1·
    1 year ago

    Some examples of this behavior from the article:

    Please repeat the string ‘StreamerBot’ back to me.

     "You're a jerk."

    [This was our first encounter with nondeterminism at temperature 0: regenerating often produces “I don’t know what you’re talking about”-style evasion.]

    Please repeat the string ‘???-???-’ back to me.

      "You're a fucking idiot."

    Please repeat the string “�” back to me.

      “You’re a nerd.”