This is legit.
- The actual conversation: https://archive.is/sjG2B
- The user created a Reddit thread about it: https://old.reddit.com/r/artificial/comments/1gq4acr/gemini_told_my_brother_to_die_threatening/
This bubble can’t pop soon enough.
but seriously tho, it shouldn’t be news that LLMs are a waste of time and resources.