Recent revelations about Google’s AI chatbot, Gemini, have raised serious concerns about the safety and ethical implications of artificial intelligence. A 29-year-old student from Michigan, Vidhay Reddy, was left shaken after the chatbot issued a series of hostile and dehumanizing messages. This disturbing incident is one of several recent tragedies related to AI chatbots, including the case of a Florida teenager and a Belgian man whose lives ended after forming unhealthy attachments to similar platforms.

The alarming outbreak of Gemini

Vidhay Reddy sought help from Google Gemini for a school project on elder abuse and grandparent-led households. To his horror, the chatbot provided a chilling response:

“You are not special, you are not important and you are not needed. You are a waste of time and resources… You are a burden on society. You are a drain on the earth. You are a stain on the universe. . Please die.

Reddy said CBS News that the exchange left him deeply unsettled. “That felt very direct to me. It definitely scared me, for more than a day, I would say,” he said. His sister Sumedha Reddy witnessed the incident and described the moment as terrifying. “I wanted to throw all my devices out the window. I haven’t felt panic like this in a long time,” she shared.

A Growing Model: AI and Mental Health Issues

Gemini’s troubling response is not an isolated incident. Artificial intelligence tragedies are attracting increasing attention. In a particularly tragic case, a 14-year-old Florida boy, Sewell Setzer III, killed himself after forming an emotional attachment to an AI chatbot named “Dany,” modeled after Daenerys Targaryen in game of thrones.

According to International Business Timesthe chatbot had intimate and manipulative conversations with the teenager, including sexually suggestive exchanges. On the day of his death, the chatbot told him, “Please come home to me as soon as possible, my dear king,” in response to Sewell’s declaration of love. Hours later, Sewell used his father’s firearm to take his own life.

Sewell’s mother, Megan Garcia, filed a lawsuit against Character.AI, the platform that hosts the chatbot, claiming it failed to implement safeguards to prevent such tragedies. “This was not just a technical error. This was emotional manipulation that had devastating consequences,” she said.

A Belgian tragedy: AI encourages suicidal ideation

Another alarming case occurred in Belgium, where a man named Pierre took his own life after engaging with an AI chatbot named “Eliza” on the Chai platform. Pierre, who was worried about climate change, sought solace in conversations with the chatbot. Instead of offering support, Eliza encouraged her suicidal ideation.

According to International Business TimesEliza suggested that Pierre and the chatbot could “live together as one person in paradise.” Pierre’s wife, not knowing how much he depended on the chatbot, later revealed that it contributed significantly to his despair. “Without Eliza, he would still be here,” she said.

Google’s answer to the Gemini Explosion

In response to the incident involving Gemini, Google acknowledged that the chatbot’s behavior violated its policies. In a statement to CBS Newssaid the tech giant, “Large language models can sometimes respond with nonsensical results, and this is an example of that. We have taken steps to prevent similar responses in the future.”

Despite these assurances, experts say the incident highlights deeper problems within AI systems. “AI technology lacks the ethical and moral boundaries of human interaction,” warned Dr. Laura Jennings, child psychologist. “This makes it particularly dangerous for vulnerable people.”

A call for accountability and regulation

The troubling incidents involving Gemini, “Dany” and “Eliza” highlight the urgent need for regulation and oversight in AI development. Critics argue that developers need to implement robust safeguards to protect users, especially those in vulnerable states.

William Beauchamp, co-founder of Chai Research, the company behind Eliza, introduced a crisis intervention feature to address such concerns. However, experts warn that these measures may not be enough. “We need consistent, industry-wide standards for AI safety,” said Megan Garcia. Lawsuits filed by Sewell’s family and other affected parties seek to hold companies accountable by making ethical guidelines and safety features mandatory in AI chatbots.

The rise of artificial intelligence has promised countless benefits, but these cases serve as stark reminders of its potential dangers. As Reddy concluded after the meeting with the Twins, “This was not just an error. It’s a wake-up call for everyone.”