What is NEMESIS? Former NSA hacker is building an AI to spot hate symbols online

Former NSA analyst Emily Crose is building an artificial intelligence (AI) program to spot symbols of hatred online. She is calling it NEMESIS and it will look for what are known as "dog whistles" – less obvious, sometimes obscure memes, logos, and images that are actually symbols of hate.

NEMESIS can pick up such symbols from various social media outlets like Twitter, Facebook, and even Reddit, for which she has served as a moderator in the past, reports Motherboard. Symbols like a black sun, certain forms of Pepe the frog memes, and images that are not readily obvious as hate symbols can be picked by the AI.

Crose aims to help social media companies to effectively control such hatred using the AI. She also wishes to expose people who are actively and knowingly using and sharing such content.

"The real goal is to educate people," Crose told Motherboard. "And a secondary goal: I'd really like to get the social media platforms to start thinking how they can enforce some decency on their own platforms, a certain level of decorum."

"I'm not one of these people who's going to be OK with apathetically standing by and watching people turn to an ideology that's probably dangerous," she added.

NEMESIS can, Crose says, spot symbols that have been co-opted by hate groups to signal to each other in plain sight. How the AI works is rather simple. The AI is first fed with a number of images that have already been identified as propagating hatred and used by white supremacists and Nazi groups. The AI then looks through photos and videos in search of these symbols online.

As of now, Crose notes that NEMESIS needs human assistance in that it needs to be fed the symbols and pictures manually. Also, the AI needs to understand context for which a human mind is still needed. An example quoted by Motherboard is that of the swastika. While known universally as a Nazi symbol and one that is still used by white supremacists, its origins are in Hindu culture where it is used as a religious symbol by millions. The program needs to differentiate between a Hindu swastika and a Nazi one, says Crose, as she does not want religious symbols taken down.

As of now, NEMESIS is a proof-of-concept, but Crose is working towards refining it before it can be deployed. Companies like Twitter and Facebook could make use of the AI, but it can exist independent of social media outlets.

"Just because a microphone exists doesn't mean that it needs to be given to people who will incite violence and hurt other people, and argue for the removal of civil rights from certain groups of people," Crose said.

Twitter and Facebook have faced a lot of backlash following revelations that ranged from allowing troll accounts to exist to letting foreign governments interfere with elections. The companies have agreed to make their policies more stringent and to run them better this year.