OPENAI gives us an overview of how he monitors the improper use on the Chatppt

Openai’s latest report on the use of malicious AI highlights the stiff rope that AI companies walk between preventing improper use of their chatbots and reassuring users that their privacy is respected.

The report, which has dropped today, highlights several cases where Openai has studied and disrupted the harmful activity involving its models, focusing on scams, cyberattacks and influencing campaigns linked to government. However, it arrives in the middle of an in -depth examination on another type of risk of AI, the potential psychological damage of chatbots. This year alone has seen several reports on users commit acts of self -harm, suicide and murder after interacting with AI models. This new report, as well as the disclosure of the previous company, provides an additional overview of how OPENAI moderates discussions for different types of abusive use.

Openai said that since he began to report public threats in February 2024, she disrupted and reported more than 40 networks that violated their user policies. In today’s report, the company shared new case studies in the last quarter and details on how it detects and disrupts the malicious use of its models.

For example, the company has identified an organized network of crimes, apparently based in Cambodia, which tried to use AI to rationalize its workflows. In addition, an operation of Russian political influence would have used Chatgpt to generate video prompts for other AI models. OPENAI also reported accounts related to the Chinese government which have violated its policies on the use of national security, including generation of large -scale systems proposals designed to monitor conversations on social networks.

The company previously declared, in particular in its privacy policy, that it uses personal data, such as user prompts, to “prevent fraud, illegal activity or improper use” of its services. Openai also said that he was based on both automated systems and human examiners to monitor activity. But in today’s report, society has given a little more knowledge of its reflection process to prevent improper use while protecting users more widely.

“To effectively detect and disrupt threats without disturbing the work of daily users, we use a nuanced and enlightened approach which focuses on the models of behavior of threat actors rather than the interactions of isolated models,” the company wrote in the report.

Although monitoring national security violations is one thing, the company also recently explained how it deals with the harmful use of its models by users with emotional or mental distress. A little over a month ago, the company published a blog article detailing how it manages these types of situations. The position came in the middle of the media coverage of violent incidents which would have been linked to Chatgpt interactions, including a suicide murder in Connecticut.

The company said that when users write that they want to injure themselves, Chatgpt is formed not to comply and rather recognize the feelings of the user and to direct them towards help and real resources.

When AI detects that someone plans to harm others, conversations are reported for a human review. If a human examiner determines that the person represents an imminent threat to others, he can report them to the police.

OPENAI also acknowledged that the safety performance of its model could deteriorate during longer user interactions and said it was already working to improve its guarantees.