A small number of samples can poison LLMs of any size
www.anthropic.com
Anthropic research on data-poisoning attacks in large language models

So the research is out and these LLMs will always be vunerable to poisoned data. That means it will always be worth out time and effort to poison these models and they will never be reliable.

  • hexagonwin@lemmy.sdf.org
    17·
    2 months ago

    I think it’s pretty obvious. Having a specific not-common keyword in the train data connected to gibberish, and when you later trigger that specific keyword in the model it’s likely to trigger that gibberish data, since that’s where the specific keyword appears most (if not only).

    Sadly this is not some great exploit that can sabotage the whole model and make it useless.