ChatGPT can offer coding solutions, but its tendency for hallucination presents attackers with an opportunity. Here's what we learned.

From https://twitter.com/llm_sec/status/1667573374426701824

  1. People ask LLMs to write code
  2. LLMs recommend imports that don’t actually exist
  3. Attackers work out what these imports’ names are, and create & upload them with malicious payloads
  4. People using LLM-written code then auto-add malware themselves
  • timmyEnglish
    4·
    1 year ago

    I mostly find Copilot extremely useful for sql queries. So much easier to write naturally what I want and Copilot just spits it out. I have to doctor it sometimes but it really does get the job done. Other than that I could do without it.