What is SimSimi? A look at the chatbot application accused of enabling cyberbullying

A South Korea-developed chatbot application popular among teenagers which uses artificial intelligence (AI) to "learn" and respond in a life-like conversation is facing mounting criticism – and bans – after being caught up in a slew of allegations that it enables cyberbullying.

The application, SimSimi, was restricted in the Republic of Ireland (RoI) last week (Wednesday 29 March) however anti-bullying campaigners are warning it remains available to users in the UK and Northern Ireland – leaving potentially vulnerable children at risk.

The technology behind the app is simple: it allows users to 'teach' the AI answers to words, phrases or questions by inputting the 'answers' to specific keywords. Then, when another person puts in that keyword for example a name – the learned associated text will be displayed. The worry being groups of children can manipulate the results to reply with malicious replies.

ISMaker, the company behind the app, which is listed on the 'popular' section of Apple's iOS Store, is yet to respond to critics however when Irish users now try to ask SimSimi a question they are met with: "I do not talk in Ireland for a while".

The BBC first reported how schools in Northern Ireland are currently issuing warnings to parents about the app. The Police Service of Northern Ireland (PSNI) is also calling on legal guardians to check their children's phones and potentially remove the service.

According to Journal.ie, parents who have examined SimSimi said their children's names had been linked to cases of "vile abuse". One mother, the outlet reported, was told she would have to wait for 15 days for any action to be taken after she complained.

IBTimes UK downloaded and tested the application. The exchange quickly turned weird. After saying hello, the app responded to this author's first name with "...loves incest." An inputting of a full name brought up the somewhat gender-confused response: "Slut is so annoying hate the girl".

Time for another name. Typing Suzanne was met with: "is gay" and the name Emily brought up: "Slut, who needs to improve her grades big time!!" The app quickly took IBTimes UK to multiple pop-ups and invasive gaming advertisements, and it was swiftly deleted.

On the PSNI Down Facebook page, a warning said the app was a "cause for concern".

A message stated: "Over the last couple of days we have been getting PMs [private messages] asking us about SimSimi on the App Store. It appears to be a computer generated chat room where you talk to a very abusive computer!

"We had only been using the app for two minutes before we got quite the surprise! Please be aware of it and its explicit nature. The app is rated as 17+ but I am twice that age and was really shocked at some of the stuff that came back! It also has in-app purchasing and advertised posts.

"We have reported it to the App Store and will include it in our internet safety talks but parents, carers and internet users should keep an eye out for it and avoid."

Harry McCann, founder of Ireland's Digital Youth Council, who helped in the campaign to restrict access to the app, previously described it as the "home of cyberbullying." In a blog post, he noted: "It's important we understand that SimSimi is being used to viciously attack people online."

"Negative, abusive, sexually graphic"

Speaking to the BBC, Liam Hackett of a UK anti-bullying campaign called Ditch the Label, said the app's developers have a "duty of care" to suspend the service for young people.

"Content is predominantly negative, abusive, sexually graphic or violent," he added. "Essentially, you can go on there and put whatever kind of content you want and there's no repercussions for you.

"It can create things like paranoia, because it isn't a direct person who is actually saying it to you. It is a robot on their behalf, which has purely user-generated content and I am guessing it's pretty much impossible to find out who has posted that about you."

On the app's customer review page, most people do seem to be aware it's not for the faint hearted – backing up the activists' assertions it could be used for web-based bullying. "Don't play if suicidal. It told me to kill myself," wrote one commenter.

Another said: "If you say the right things it starts to say dealing disturbing and disgusting things like kill yourself and stuff like that so if you are sensitive to that stuff I suggest you don't play this game." Others stressed it has a filter for "bad words" which should be switched off by parents.

It's not the first time the app has got into trouble. Back in 2012, SimSimi was reportedly banned from use in Thailand after fuelling political controversy and protests. One government minister, one outlet claimed, contacted the developers after it insulted the country's prime minister.

It remains to be seen if action will be taken outside of Ireland.

If you need to report cyber bullying, you can contact the NSPCC helpline today.