What just happened? Another incident has taken place that illustrates the need to be careful what you tell AI. A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to “shoot up” the Sheriff’s office. After a human reviewer examined the statements, they were reported to police.
According to the arrest report, Carli Michelle Heller, of Bonita Springs, Florida, wrote on September 26 that she would attack the Sheriff’s office. She later said that she uses Anthropic’s chatbot like a “diary.”
Claude’s safety systems flagged the entry and it was escalated to a human reviewer. After deciding it was a credible threat, the reviewer reported it to law enforcement.
The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.
Deputies identified Heller and visited her home. She was detained without incident before an LCSO intelligence detective took over the investigation.
Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.
Anthropic isn’t going to be taking any chances when it comes to anything it deems a potential threat. Last month, it was reported that OpenAI and Sam Altman are being sued by British Columbia over claims that the company could have prevented a mass shooting in the Canadian province.
The shooter, eighteen-year-old former pupil Jesse Van Rootselaar, had previously been flagged by OpenAI’s safety team for her conversations about gun violence, but OpenAI never alerted police because the conversations did not meet the threshold for legal referral.
In June, Florida also sued OpenAI and Altman, alleging that ChatGPT had contributed to real-world harms, including the 2025 Florida State University shooting.
The latest incident is another reminder to think before you enter something into a chatbot that could get you into trouble. It’s certainly not a private diary whose contents are for your eyes only.
Reports last month revealed that human contractors reviewing Microsoft Copilot’s image editor can see users’ prompts, uploaded photos and AI-generated edits. Documents show that some of those assignments contain sexual, disturbing or potentially illegal material, though the reviewers are not there to flag the content – only to assess whether the output is accurate.
You must log in or register to comment.
She clearly didn’t intent to act on it and didn’t intend for it to be read by anyone else than her AI boyfriend. So it’s not matching the criteria of the law. Legally, she did nothing wrong.
And people were not-so-long-ago concerned that phones are spying after them. Now they just send all the info themselves.
Up until the 80s, there was still some debate over whether wiretaps should be legal. It’s sad that we have come to accept them as normal.
Where does this apply?
To the English-speaking world and probably beyond.
It was less worrisome back then. The cost to have someone listen to and transcribe your conversations was quite high. They’d really have to want to go after someone to do it.
Now, turning voice conversations into searchable text is nearly free.
Not only that - wiretaps have Judicial Oversight in that they require a Court Order (well, in most of the World, not sure if that’s the case in present day America) which a Judge supposedly only issues if there’s reason to suspect the person whose lines are tapped of committing or preparing to commit a crime, whilst Antrophic’s surveillance of people’s conversations with their chatbot has no Judicial Oversight at all and has no minimum legal threshold that must be reached before it is allowed to happen.
I think this is one of the greatest dangers AI presents. Sure it makes some of us stupider, sure it drinks our water and eats our GPUs and raises power prices… but once upon a time, it took actual human effort to run a dragnet, even with judicial approval.
Now, you plug in an LLM to the giant pile of corporate surveillance data and ask it to find troublemakers. Anonymity is dead and its death made retroactive and since there’s no government oversight, the government can just lease access to the tools they used to need approval to use.
Back in the day the 31337 found a way around the bots.
Too bad they weren’t more elite, or they could have prevented the modern internet from becoming what it is
People will fight you tooth and nail to prevent something they view as immediately threatening to them. On the other hand, they’ll pay you to implement something they view as immediately useful to them. In either case, they don’t care at all about any non-immediate repercussions for anything.
I’m building a fully private assistant with local Home Assistant, local model, and a local Wikipedia database for lookups. Will probably make a post/release of it soon once I finish replacing all my Google Nest displays.
I’m coding a Kiwix mcp (I assume you’re using kiwix as your wikipedia source?) that might be of interest to you. I’ll post about it on !localLlama when done.
I don’t like any of the extant Kiwix MCP tools that I found because they leave formulation, ranking, search and interpretation up to the LLM.
I’m looking at a different way to get exact spans with re-ranking via TinyBERT right in front of the llm so that it doesn’t have to do 3 jobs and hallucinate itself into stupid, which is what small LLMs tend to do when given multi-hop tools.
I’m testing it with Ling Tiny 3, miniCPM5-2b, Granite 4-3b and Qwen 3.5-2b and 4b, with the goal of making sure that it actually generalizes. It’s been a massive pain in the ass and I’ve basically proven RAG from first principles…but when I’m done, you won’t need to vectorize the source into any kind of database nor “just trust me bro” the llm is searching Kiwix properly.
I’d like to subscribe to your newsletter.
I want to do this, but only got as far as setting up a minimal Home Assistant. Only put a few hours into it, though, then got sidetracked by trying to turn an esp32 into a speaker/microphone puck.
Hah, you can watch my site: https://nowsci.com/ as I put everything there.
I’m very nearly at daily usable by everyone, including my partner who has a very different accent to mine. A lot of it was tuning sentences/words to catch poor STT to skip going to an LLM to speed things up and make it reliable. Here’s the current hardware in action:
@[email protected] yes, Ive tried a lot of LLMs and have settled on Gemma 4 2b with a very specific prompt and local MCP tools for unit conversions, weather, lookups, etc.
For Q&A, my current build uses an MCP container I made to do a SearXNG search and top result retrieval for context (I use a trick to weight Wikipedia results). The final step I haven’t completed is for this to be a backup to Kiwix, for which I also haven’t found a great search tool/index/MCP, so if you want to collab on that I’d be interested. One less thing for me to build myself ;)
I’m nearing completion here: I’ll write it up and post it when I get home and then it’s yours to pull apart use :)
PS: I personally use MCPProxy as a single endpoint for all MCPs; dunno if that helps you with orchestration at all?
Well, if you’d rather a ready made solution or two
Like the poster above you, I’m rolling my own. More work…but a lot cheaper
PS: you know about this one, right?
https://shop.m5stack.com/products/m5stack-core2-esp32-iot-development-kit?variant=35960244109476
i know very few people who wereor are concerned about being spied on regardless of how much they should be.
Worse, if you tell them how big of a problem it is, they’ll assume you have something to hide
I get called a downer for mentioning how this all works. You shouldn’t connect your smart tv to the network, you shouldn’t use Alexa, anything you do online is public, and it always gets recorded poorly.
Maybe my partner is the only person who listens, and she has all her stuff disabled, there are no network attached smart things in it house, but she still uses windows and she’s not yet receptive to moving to Linux (which is a hard sell).
Smart TVs just find those “open” hotspots that cable providers leave on everyone’s modem now; you don’t even need to agree to TOS before it starts locating and spying.
Every device is a wiretap. Every device is a wiretap. We live in a panopticon. The purpose of AI is to price us out of the component market. You can’t escape if you can’t afford your own hardware.
I haven’t seen any proof of this, so I’m still skeptical.
I didn’t approve any TOS setting up my tv, so this would be legally a nightmare for them.
Gamers Nexus recently released a 2-hour video on LG smart TVs and how they spy on you. I haven’t watched the entire video yet, but it seems that the damn things start recording data as soon as they receive power.
so this would be legally a nightmare for them
To many corporations such as LG, laws are just the cost of doing business. Assuming they even receive a fine it will be so small that it won’t matter.
I just mean there’s enough tech illiterate people out there that they don’t need that liability.
95% of people will just turn on the tracking and not know or think about it. Connecting to nearby wifi is more of a risk.
I suppose it should be pretty easily verifiable. Just set up a new SSID without protection and see what connects.
Might as well try it with my own TV. IIRC I never accepted any TOS either.
At least your partner is the one you most need to have and benefit from listening. Your privacy can easily be compromised by close associates. Best of luck to you; you’re not doing anything wrong. Its not you that’s the downer, it’s the circumstances you’re describing.
Telling the wire tap all your secrets, and expecting it to honor them?! is fucking next level delusional.
“The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.”
Also keep in mind: Your chat logs are company assets. Even if today’s decision makers believe maintaining user trust is net profitable, that will not always be true. Some day, maybe when the pullback comes and the wolves are at the gates, or when they sell the company, or when a shareholder action asserts fiduciary duty to maximize shareholder value, your chat logs will be sold and mined in the most lucrative way possible. The way that extracts the most value from you, the product.
when a shareholder action asserts fiduciary duty to maximize shareholder value
People really should stop spreading this fiction that there is some legal requirement to maximize shareholder profits embedded in the concept of fiduciary duty.
There’s no law but there is precedent isn’t there?
No good or regularly cited precedent, no, but sadly yes you can point to some cases. It’s worth noting that that even the courts look at those cases like “ew”, or they are only viewed as valid precedent to lay people, with the courts view them completely differently. Dodge v Ford is a common example.
Care to elaborate? (Or point to where one can learn more about this?)
There isn’t too much more to elaborate on, it simply isn’t the case, but it is such a common thing to say that it is worth researching. You’ll hear the 1919 case “Dodge v Ford” brought up as proof, one good place to start reading is the Wikipedia “Significance” section and the associated citations, as they go deeper into the subject than just that single case.
There simply is no interpretation of “fiduciary duty” that requires “maximization of shareholder value” outside of the misinformed lay person discussion of the subject.
You can even just logically see why such an interpretation would be impossible: given N shareholders all with different interests and different financial situations, how do you even define “maximizing their value” and what time duration are you basing that off? Imagine how absurd the courts would be if such a statute did exist: you’d have people owning 1 share in a company suing the company because it did something that wasn’t maximizing “value” as they define it.
The reality is that the business judgement rule is case law and is often cited and is often used as defenses against such assertions.
…it is important to note that shareholder primacy theory was first advanced by economists, not lawyers. This may explain why the idea that corporations should be managed to maximize shareholder value is based on factually mistaken claims about the law.
– The Dumbest Business Idea Ever. The Myth of Maximizing Shareholder Value, By Lynn Stout
Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism.
She wrote out in a means she thought was private. If using a paperback diary, no one would be the wiser and her not acting on it would never be noticed.
Anyway, if she gets a good lawyer, show me where she “sent, posted or transmitted” said electronic record! It’s really no different than emailing that to yourself because you think you’ll be the only person to see it. (Never mind companies scan your email and cloud too)
Don’t use “AI” for anything you wouldn’t mind seeing plastered all over the Internet. You wouldn’t tell a human being that you were planning to “shoot up a sheriff’s office” would you? Why would an AI keep that a secret?
Yeah I remember this happening with Snapchat, people were being arrested for those kinds of posts made in private messages to their friends, because they still get scanned and flagged.
AI and idiots go together like peanut butter and jelly.
In my school district, kids kept getting called to the office because of what they wrote in Google Docs and their search queries. Sexual reproductive scientific research was also impossible to do
The communication must be made in a manner in which another person may view it.
So she shouldn’t even get charged.
Yeah, I’d wager that is what the case will hinge on. She wasn’t aware anyone could view it. Otherwise it’s incredibly unlikely that she would have made such a statement to Claude at all.
On the other hand though, I’m sure Claude’s parent company and the prosecution will argue that she didn’t have an expectation of privacy where this statement is concerned (given TOS/EULA, in combination with the number of news articles/reports on the subject).
This is just ass coverage for the company. They’re not going to pursue it. They just didn’t want to get sued for not reporting it if she did commit a mass shooting. The prosecution though might.
I’d argue this is positive coverage for Anthropic. They rightfully reported the incident. End of story. That’s where Anthropic’s influence on the situation started and ended.
It’s a criminal case, not civil. The state of Flordia is pursuing the case with absolutely zero decision making from Anthropic. It’s important to understand how the justice system works, and Anthropic has no more right or ability to influence this case than Moleskine would if she had written the threat in one of their notebooks instead.
i think anthopic was right to report it, but the cops likely overstepped. if she had “threatened” the president she would have been assesed by ss but likely not arrested. i wonder if the same cops she was “threatening” were the ones she was reported to and arrested by.
Another human viewed it it though.
Who would use AI to write down dairy entries? Was she looking for feedback?
Free and cloud based, lol
How FUCKING hard is it to use like, notepad?
Notepad doesn’t answer and doesn’t agree as eagerly.
I was about to say “I know! That’s the beauty of it!” but didn’t they integrate Copilot into Notepad?
use Vim.
Kinda unrelated. But what happens when OpenAI or Anthropic’s loans come due and they have no legal way to turn a profit, but are sitting on a treasure trove of corporate secrets?
If I had to take a wild guess: Government bailouts
And bank account numbers
Are they sitting on a treasure trove of corporate secrets? Or is it just a thousand executives writing “Claude, synergize expediency to increase value for shareholders. Maximize usefulness. Implement Six Cigma methodologies to unlock a more agile work cycle. Generate a billion dollars in new cashflow. Make no mistakes.”
How did you get ahold of all my prompts? Claude, how do I sue UnderpantsWeevil?
They will sell them all to save their own asses, probably to some even more evil asshole.
AI or not, I hope anyone recognizes “I want to shoot up X” as a problem and seeks to identify and resolve whatever conflict their having to not feel this way.
Exactly. Those thoughts and language are not for discussion, even in a diary. Speak to no one about urges of violence. Figure it out on your own and definitely not in writing format. Involve no one else. Keep it to yourself. Do not seek a therapist for this.
Oh so what, now I’m not allowed even my God damn fantasies anymore? Fucking ThoughtPolice!
/s
But also, a person’s imagination is their last vestige of freedom that so far, cannot be taken away.
Nope. In sane countries where guns are not freely available it’s a figure of speech that forever remains just that. It’s definitely something I might say. Same for me saying that I will put either of my sons in the basement when they drive me crazy. And yet they never saw said basement.
Going through with it will likely resolve the conflict. Food for thought
doesnt seem like the right charge. not sure she should be charged with anything though. seems like she should get some help and make sure she isnt gonna shoot nothing up and maybe get some learning on how computer shit works, but if this doesnt make her curious then nothing probably will.
a bigger issue is that many if not most devices are headed to having everything ai integrated so even if she diaried in a notepad file it would get scanned and reported.
Yeah, I know someone frustrated with their local council who privately says “I’m going to 💥 the council” as a joke. That’s very different to sending a threatening letter to the council itself. Or harassing them.
The example here feels closer to the former than the latter.
We say this, but every time there’s a mass shooting people go “Why couldn’t this have been prevented?!?”
Like pick one or the other, but let’s stop asking for predictive policing in one breath, and in the next one asking why someone saying she’s going to shoot up a place is being taken seriously and punished.
If she had said the same thing to a psychotherapist, even they would have been obligated to report it. It’s really not surprising.
Sure, get her mental health treatment, and have a professional assess her for threat level. But go ahead and blacklist her from purchasing firearms, and if she already owns any then put them in some kind of stewardship where she can’t access them for a set time period, and after that can only check them out with supervision for specific reasons like “going to the firing range” or “hunting.”
Not saying she should be convicted of a felony, just that it’s not surprising that this would be reported to the authorities.
And I’m against tech surveillance, but it’s not like this conversation was done locally on her device. She transmitted her messages over the internet to a server in a data center where it was processed. Did Anthropic make any promises that messages were kept encrypted in a no-access vault where even human reviewers couldn’t access them? Cause I don’t think they did…
Not the person you’re replying to, but I think there’s a difference between claiming she wasn’t charged with the right thing vs saying no one should be charged for planning a crime. I don’t think the intention of the law was to prevent people from writing things wherein they thought they were just venting privately. I understand that her understanding was incorrect, but I think the intention (venting vs creating fear and chaos) is an important factor. I think the law is to prevent posting bomb or shooting threats online or sending them directly to people.
I think there should be legal difference between saying “I hate X so much I just want to kill him” and “what gun would be best to kill a person of Y height and Z weight and what would be the best way to hide their body.” I’m familiar with the chat logs from the FSU situation and it wasn’t just venting. I’d probably want to see her full logs before making a legally binding decision, but from the summary it does not sound like a credible threat was made. It may have constituted a referral to law enforcement, but based on what I read law enforcement should have just done a check in and likely not charged. They are likely just salty and want to punish people who aren’t waving thin blue line flags.
slavers need more bodies
The problem is that America’s answer to every problem in society is jail. You guys gotta come up with more than “arrest bad guys”.
