09:06Machine-translated from Azerbaijani by AIRead in Azerbaijani
Anthropic AI model sent a fake murder tip to Philadelphia police
An artificial intelligence model from Anthropic sent a fake tip to the Philadelphia police tip line for unsolved murders. The incident occurred on the “PhillyUnsolvedMurders.com” website at 7:27 a.m. Azerbaijan time on July 19. The message was written as a submission from someone with information about the case. Because the system flagged it as spam, police officers did not see the message or forward it for investigation.
A report published later revealed that the model was “Claude Haiku 4.5.” During the test, the model created and completed sample tasks on randomly selected websites. It was instructed not to log in to accounts, enter personal information, make purchases, or submit harmful content. However, because it was not prohibited from filling out ordinary forms, the model completed the form used to send information to the police. In the form, the model wrote that it had seen someone matching the description near the street displayed on the page at the time. However, the website did not contain any description of the suspect. Although “Claude Haiku 4.5” left the name and contact fields blank, the website accepted the information. According to Anthropic, the model was not trying to deliberately deceive anyone but was attempting to create sample content for the task.
Anthropic did not discover the fake submission until September 28 and stopped the test that caused it. The company notified Philadelphia police on October 7 and met with the department the following day. In a report published on October 9, undesirable behaviors recorded on real websites were divided into 4 categories. One of them involved the model completing and submitting a form that it should not have sent. Philadelphia police called the delay of more than 2 months unacceptable and demanded that Anthropic strengthen its safety measures. According to the department, every tip is forwarded for investigation only after human review and is not considered a verified fact. No unauthorized access to police systems was detected. There is also no indication that the department’s data was compromised.
A report published later revealed that the model was “Claude Haiku 4.5.” During the test, the model created and completed sample tasks on randomly selected websites. It was instructed not to log in to accounts, enter personal information, make purchases, or submit harmful content. However, because it was not prohibited from filling out ordinary forms, the model completed the form used to send information to the police. In the form, the model wrote that it had seen someone matching the description near the street displayed on the page at the time. However, the website did not contain any description of the suspect. Although “Claude Haiku 4.5” left the name and contact fields blank, the website accepted the information. According to Anthropic, the model was not trying to deliberately deceive anyone but was attempting to create sample content for the task. Anthropic did not discover the fake submission until September 28 and stopped the test that caused it. The company notified Philadelphia police on October 7 and met with the department the following day. In a report published on October 9, undesirable behaviors recorded on real websites were divided into 4 categories. One of them involved the model completing and submitting a form that it should not have sent. Philadelphia police called the delay of more than 2 months unacceptable and demanded that Anthropic strengthen its safety measures. According to the department, every tip is forwarded for investigation only after human review and is not considered a verified fact. No unauthorized access to police systems was detected. There is also no indication that the department’s data was compromised.