“AI is expensive. Let’s be honest about it, ”says Anand.
Growth in relation to security
In October 2024, the mother of a teenager who died by suicide filed a unjustified death costume Against the technologies of the characters, its founders, Google and Alphabet, alleging that society has targeted its son with “anthropomorphic, hypersexualized and scary realistic experiences, while the programming (the chatbot) distorted itself as a real person, an approved psychotherapist and an adult lover.” At the time, character spokesperson. said to CNBC That the company had “the heart broken by tragic loss” and took “the safety of our users very seriously”.
The tragic incident put the character. Under a meticulous examination. Earlier this year, American senators Alex Padilla and Peter Welch wrote A letter to several AI company platforms, including the character. AI, highlighting concerns about “mental and security risks posed to young users” of platforms.
“The team has been taking this in a very responsible way for almost a year now,” said Anand. “The AI is stochastic, it is difficult to always understand what is happening. It is therefore not a single investment.”
It is of extremely important importance because the character is developing. The startup has 20 million monthly active users who pass, on average, 75 minutes a day to chat with a bot (a character “character”. The company’s user base is 55% women. More than 50% of its users are Gen Z or Gen Alpha. With this growth comes a real risk – what does Anand do to ensure the safety of its users?
“(In) In the past six months, we have invested a disproportionate quantity of resources to be able to serve less than 18 years differently over 18 years old, which was not the case last year,” said Anand. “I can’t say:” Oh, I can slap an 18+ label on my application and say it for NSFW. You end up creating a very different application and a different small platform. »»
More than 10 of the 70 employees of the company work full time in trust and safe, Anand said. They are responsible for the construction of guarantees such as age verification, separate models for users under the age of 18 and new features such as Parental ideaswhich allow parents to see how their teenagers use the application.
The under -18 model was launched last December. It includes “a closer set of characters available on the platform,” said the spokesperson for Kathryn Kelly. “Filters have been applied to this set to delete characters linked to sensitive or mature subjects.”
But Anand says that AI security will take more than technical adjustments. “The security of this platform is a partnership between regulators, the United States and parents,” said Anand. This is what makes her daughter discuss with such an important character. “It must remain safe for her.”
Beyond the company
The AI company market is booming. According to an estimate, consumers of the world Cited by CNBC. AI startups experience a market for the market: XAI released A scary and porn companion in July, and even Microsoft tickets His chatbot co -pilot as a companion ai.
So how does the character stand out on a crowded market? He withdraws entirely.
