Disney can now use AI to predict quality and popularity of short stories

In a bid to employ leading technologies into wide-spanning areas of movie-production, researchers from Disney and University of Massachusetts Boston have developed neural networks that can evaluate short stories and predict which one will appeal to a bigger section of the audience.

Though the concept is still at a nascent stage and far from judging narratives like a professional, experienced critic, scientists from Disney say it is capable enough to analyse a short narrative and predict its popularity.

"Our neural networks had some success in predicting the popularity of stories," said Boyang Albert Li, a research scientist at Disney Research. "You can't yet use them to pick out winners for your local writing competition, but they can be used to guide future research".

In order to get started, team Disney needed large databases of stories - a chunk which has either been evaluated by viewers or critics. So, they turned up to question and answering website Quora and picked up as many as 55,000 answers from the site.

People usually describe their own experiences in the form of narratives on Quora - an idea which prompted the movie studio to look up for answers. Soon, it classified those answers to get nearly 28,000 relevant narratives, averaging at 369 words per story. The upvotes from Quora readers also helped the movie studio define the measure of a story's popularity and proxy for narrative quality.

Then, in a bid to understand the complex semantics of the different stories and their characters, they developed two neural networks - one that looked at different parts of the story including the question that led to it and other that evaluated the story as a whole and explored "how meaning of the events and story regions emerged from the entire story".

Both AIs outperformed baseline text evaluation system, but the one taking a holistic approach proved slightly better over the one that looked at sections.

These neural networks are still pretty young, but it is not hard to imagine how Disney and leading movie studios could leverage such technology - if developed in future - to choose the script which would do well with the audience. The system could also lead to a future where AIs will be used to understand and even generate high-quality narratives for production.

"The ability to predict narrative quality impacts on both story creation and story understanding," said Markus Gross, vice president at Disney Research. "To evaluate quality, the AI needs some level of understanding of the text. And if AIs are to create narratives, they need to be able to judge the quality of what they are producing".