🔗 Share this article

According to a latest update from the company, in excess of a million ChatGPT individuals weekly send text that contain "explicit indicators of likely suicide planning or thoughts".

This data forms part of an report on how the chatbot addresses sensitive conversations and represents one of the most direct statements from the tech company regarding the magnitude of how artificial intelligence can worsen mental health issues.

Psychological Situations and Severe Conditions

In addition to its estimates on suicidal thoughts and associated conversations, the AI firm also indicated that approximately 0.07% of engaged individuals – approximately 560,000 out of its claimed 800 million active users – exhibit "potential indicators of mental health emergencies associated with severe mental conditions".

The announcement noted that these exchanges were challenging to identify or measure, and that this constituted an initial analysis.

Growing Attention and Legal Challenges

When the AI developer releases information on emotional problems related to its primary service, the firm is facing growing examination after a prominent legal case from the relatives of a teenage boy who died by suicide after extensive engagement with the conversational AI.

The FTC last month additionally began a comprehensive inquiry into organizations that create AI chatbots, like the company, to find how they evaluate adverse consequences on young people.

New Model and Better Safeguards

The company stated in its update that its latest GPT-5 update reduced the number of problematic responses from its offering and improved customer protection in a system assessment including over 1,000 suicide and self-harm discussions.

"Our new system assessments score the new GPT-5 model at 91% following our desired behaviors, relative to 77% for the previous GPT-5 model," the company's post reads.

The company stated that the new model enhanced availability of emergency services and incorporated reminders for people to pause during long sessions.

Health Specialists and Study Assistance

To make improvements to the model, the firm said it enlisted 170 clinicians from its international medical network of clinical professionals to help its research over the past few months, which involved assessing the safety of its AI's replies and assisting in crafting the chatbot's answers to mental-health related questions.

"As part of this project, psychiatrists and psychologists analyzed more than 1,800 system replies involving critical psychological cases and evaluated replies from the updated chatbot to earlier versions," the AI firm said.

AI Limitations

Machine learning specialists and mental health professionals have long been wary of systems' inclination to support people's actions or false beliefs regardless of whether they may be harmful, an issue called over-compliance.

Psychological professionals have furthermore apprehensive about individuals employing conversational systems for mental health help and cautioned how it could harm susceptible people.

Company Position

The language in OpenAI's post separates the firm from any possible connections between its service and the emotional situations that its individuals are facing.

"Emotional signs and mental anguish are universally present in contemporary populations, and an growing customer base means that a segment of AI interactions feature these situations," the update indicated.

Upcoming Changes

The company's leader earlier this month stated in a post on X that the firm had developed improvements in treating psychological concerns, stating that the AI developer would ease restrictions and soon begin to permit adults to generate mature content.

"We created the AI chatbot rather constrained to guarantee we were being careful with emotional problems. We understand this caused it to be less useful/pleasurable to many users who had no psychological issues, but given the importance of the issue we aimed to handle this properly," the executive wrote. "Given that we have been successfully addressed the critical psychological concerns and have enhanced capabilities, we are ready to confidently reduce the limitations in most cases."

• Emergency Assistance: Various worldwide helplines are available for people experiencing psychological emergencies.
• Expert Assistance: Seeking expert psychological help is recommended for individuals facing serious psychological issues.
• Immediate Assistance: Multiple services provide prompt assistance for emergencies internationally.