State AG demands answers about Meta AI chatbot’s false accusations

State Attorney General Letitia James sent a letter to Meta, the parent company of Facebook, demanding answers about the company’s AI chatbot fabricating allegations of sexual harassment against state lawmakers. The letter is dated April 27, one day after City & State reported that the Meta AI chatbot was informing users about completely made-up allegations of harassment concerning elected officials. 

In the letter, which directly cites City & State’s reporting, James wrote that her office tested the chatbot and found it “yielded similar results, including citations to fabricated source articles.” “We are deeply concerned about the creation and dissemination of such misinformation, which is likely to harm the reputation of New York State officials and mislead New York residents,” James wrote.

The letter offers Meta a May 10 deadline to respond to a series of five requests posed by the attorney general’s office. “We would like to better understand the efforts Meta has made to prevent Meta AI from creating and disseminating false information, particularly concerning serious claims of sexual harassment,” the letter reads. 

The attorney general’s office wants answers from Meta about the steps the company has taken to address the chatbot’s “hallucinations,” or erroneous responses, related to public officials and sexual harassment. James is also asking for details surrounding the chatbot’s fact-checking measures and any disclosures that Meta provides regarding the accuracy of information produced by the chatbot.

In the letter, James noted that her office received responses that cited fake articles from major news outlets like The New York Times and CBS News, and she asked Meta about the steps it is taking to address such false citations. Finally, the attorney general asked Meta about mechanisms for users to report AI hallucinations to the company.

“This is a new technology and we inform users that responses may be inaccurate or inappropriate and allow them to provide feedback to help us improve,” Meta spokesperson Kevin McCalister said in a statement. “We apologize for this issue and are actively working to address it.”