An FBI agent called a Claude user about his chats

(austinpatrick.substack.com)

28 points | by AustinLikesAI 7 hours ago ago

27 comments

  • breakwaterlabs 5 hours ago

    > but to my fellow Americans, when you’re speaking to the police about an issue, you’re under no obligation to speculate on answers.

    Youre under no obligation to answer them at all, particularly with no warrant, arrest, or attorney.

    > asks him to come down to the station for questioning.

    NAL: I do not believe your legal situation improves if you voluntarily agree to such a thing.

    “No, thank you”, can suffice.

    • frogperson 5 hours ago

      In some states you must actually say "I plead the 5th", it can't just be implied, and remaining silent can be used against you. You can say "I would rather not say, and I plead the 5th."

      • breakwaterlabs an hour ago

        You don’t need to plead the fifth when you are not being held, when there is no probable cause.

      • FireBeyond 3 hours ago

        In theory, it's any state, as it is based on your constitutional rights. But yes, there have been court cases where it was ruled that you need to "assert your fifth amendment rights, ideally on each and every question".

        That exercising your right to remain silent by ... remaining silent ... was able to be characterized by the prosecutor as being uncooperative and obstructing and hindering the investigation of the crime.

        And if you're in Louisiana, be doubly careful:

        Beware anything beyond the simplest statement. "Yo, I want a lawyer, dawg" was successfully argued in the Louisiana Supreme Court as "Defendant asked for a canine attorney. Law enforcement were unable to find one, but had fulfilled their obligation to attempt to provide counsel for the defendant. Therefore, any statements he made after this were done voluntarily, fully aware that he had no counsel and were as a result admissible."

        • bikvfeee 43 minutes ago

          I'm calling out this story as fake.

  • kylecazar 6 hours ago

    "no easy answers here, it seems!"

    Speak to a model as if your words are being sent to a company that has warned you they will escalate to authorities.

    If you need to make threats and express violent thoughts get a notebook. And maybe a therapist.

    • pshirshov 5 hours ago

      > If you need to make threats

      "If you need to shoot monsters ... get ... a therapist"? Why can't I play an FPS?

      • 5 hours ago
        [deleted]
  • 4 hours ago
    [deleted]
  • barbariangrunge 7 hours ago

    Title needs renaming as per hn guidelines, no?

  • ultimafan 3 hours ago

    This seems to be the very definition of play stupid games, win stupid prizes. Never understood what people like this get out of pushing boundaries, it's a complete disconnect between reality in their head vs reality in society, sure you can technically be 100% legally right as far as letter of the law is concerned but that's not how normal people with emotions operate and sooner or later someone is going to end up getting hurt.

  • refulgentis 6 hours ago

    I’m doing legal work this week involving a school shooting case and a missed warning sign. Maybe that's why this doesn't feel like some quirky fun thought experiment to debate. The user knew he was trying to get a rise out of Claude. The person reviewing the chat saw someone saying he planned to kill his neighbors and had already called others to join him. Taking that seriously was reasonable.

    • fn-mote 6 hours ago

      He got a rise out of Anthropic and the FBI. Claude doesn’t ever care, and pretending otherwise obscures the “agency” of the people involved.

      Frankly, just like swatting, this highlights how authorities do little due diligence before charging into a situation.

      It might be a crisis situation, but a single agent doing some surveillance could have figured out it nothing was happening at the old residence.

      This is like getting swatted. Responding less aggressively would have made the situation less dangerous.

      • refulgentis 5 hours ago

        I'm a bit confused, I'm missing something: after they said they're acting on killing their neighbors, there should have been 1 agent surveilling the wrong house, because “agents knock[ing] on the door" is dangerous and aggressive?

    • thisisjeffmaher 6 hours ago

      Agreed, and just saying "no easy answers" with the shooting anecdote at the end of the article kind of undersells it.

    • cyanydeez 6 hours ago

      yeah, reasonable, but an expectation of the user? Should the user reasonably assume chat bots are real life?

    • roosterIllusi0n 5 hours ago

      How many times does a warning system get to be wrong before its warnings are ignored?

      You can say whatever you want to your AI if you run it locally. People can't have a tool that tries to invent crimes out of every benign thing they do by stripping all context. It's an absurd concept.

      The agent's attitude to blame the victim of a false report instead of the false reporter demonstrates how dangerous this is. Agents and officers that lie to arrest you will never get in trouble. When they are angry at the victim, they can make up whatever they want.

      • refulgentis 5 hours ago

        "False alarm” and “false report" aren’t the same thing. By his own account, he said he was "planning to kill my gay neighbors across the street." He knew he was joking. That doesn’t mean a reviewer invented the threat by taking him out of context.

        The post doesn’t say anyone lied to arrest him. He wasn't arrested. And it says the detective told him he wasn’t being detained.

        Flipping a false claim that you're about to commit murder, into a problem about checking if the claim is false is saddening, given the consequences.

        • roosterIllusi0n 4 hours ago

          They are exactly the same thing once you notify the police. Police work off the assumption that it takes concern to contact them and then they treat it all seriously.

          That can't happen if you are false reporting multiple messages a day to police because you lack context or refuse to assess context.

          I am sorry for your position, but false calls to police are not the right of any company. If they find it impossible to moderate without making multiple false calls a day to police, that is proof their business model is illegal. They either pay the money to get it right or they shutdown. This is how businesses are supposed to work.

          • refulgentis 3 hours ago

            A false report is one the reporter knows is false. Anthropic passed along what he actually wrote. He "had specified a target... mentioned a plan... and mentioned that the plan was in action."

            More context wouldn't obviously have cleared him, either. By his own account, "I say a lot of heinous stuff to my chatbot," including asking how to "take down each officer" in a shootout.

            If you think companies report too often, that's a fair argument. It's not something mentioned in the post and it's extremely hard to understand how it applies here.

          • DeluluDon 4 hours ago

            I think they're within every right to report. You should read the article instead of kneejerking at headlines.

  • treebeard901 40 minutes ago

    When you use AI, you're in a group chat with law enforcement, courts probably even doctors.

  • curiousObject 4 hours ago

    Meanwhile in video games, millions of Americans, some of them quite young, are right now playing with imaginary scenarios of mass murder.

    But the FBI is not wasting its time visiting their homes, or blaming them for wasting the agency’s time.

    What’s the difference?

    • bikvfeee 39 minutes ago

      The difference is that in the case of videogames society has already explored that path, and got over it.

  • cucumber3732842 6 hours ago

    I have no commentary on where we should draw the line when it comes to the chat provider narcing to the feds but can we all appreciate that this dude is gonna spend his whole life regretting that time the FBI asked him if he knew any Klansmen and he missed the chance to respond with "well they say the Klan is all feds, and now I know you" or some wisecrack like that.

  • bartleeanderson 5 hours ago

    This belongs on the same footing as Fuck your AI safeguards, but additionally Fuck Anthropic for adding to the police state.

  • strathmeyer 5 hours ago

    [dead]