Can Big Data and algorithms find the next Harry Potter?

Big Data analytics might be able to predict the next Harry Potter – like best seller

As Harry Potter turns 20, JK Rowling’s runaway success published by Bloomsbury gave a new lease of life to adult fan fiction. Two decades later, publishers are leveraging big data to uncover characters that could potentially be the next bestseller.

J.K. Rowling’s Harry Potter books caused worldwide Potter mania with 450 million copies sold across the world in 79 languages. The question is can big data replicate the success of Rowling’s fantasy fiction?

That’s where computer algorithms and big data analytics steps in. Today, several researchers and professionals are extensively working on how they can help publishing houses predict the success rate of a book before it’s released.

Can Big Data find a bestseller?

John Grisham

An ex-research lead on literature at Apple, Jodie Archer, and an associate professor of English at the University of Nebraska-Lincoln, Mathew L. Jockers is co-publishing a scholarly book, titled “The Bestseller Code: Anatomy of the Blockbuster Novel.” The subject of the book revolves around “bestseller-ometer,” an algorithm that claims a track record of “predictingNew York Times best sellers when applied retrospectively to novels from the past 30 years.

Around 2008, Jockers worked as a lecturer at the Palo Alto campus. His work revolved around the application of computer-enabled quantitative analysis to text. Archer on the other hand, was doubtful about computers’ ability to be substantive about literature.

Together they demonstrated a computer model’s prowess in picking the genre of Shakespeare’s plays based on textual markers. The model helped them find that the book’s primary subject was “human closeness.”

Archer and Jockers realized that narrative “cohesion” was a common trait among top-selling authors. This was done by cataloging words associated with certain subjects. Furthermore, Danielle Steel devotes one-third of her novels to a signature topic – “domestic life,” while John Grisham writes mostly about “lawyers and law.”

Archer and Jockers made other interesting observations as well, for instance, sex doesn’t sell, or at least it’s confined to a vanishingly small proportion of best-selling material. They run their model on the book Fifty Shades of Grey, from which they drew the conclusion that the chief subject of the book was “human closeness.”

Other algorithm-based initiatives in the publishing arena

Jodie Archer and Mthew L. Jockers

Archer and Jockers were not the first in the arena to apply the power of Big Data to literature. A Berlin-based startup Inkitt is venturing into a similar domain with what’s been known as the “first novel selected by an algorithm.” Essentially, the program tracks reader responses to stories posted to its web platform to identify potential best sellers.

London’s Jellybooks, another firm, founded in 2011, employs a similar approach. The algorithm can measure “reader engagement” later in the literary production cycle, right before the books are published. This is achieved using a software downloaded by readers onto their devices in exchange for advance access to a title.

The algorithms aren’t magic. Matthew Wilkens, Assistant Professor, English, University of Notre Dame comments, “They reflect interpretative and analytical choices reading one book closely; you’re looking for certain repetitions, word usage patterns, thematic emphases, and allusions.”

Use of algorithms in the publishing industry

Dave Eggers’ The Circle

These algorithms furnish instructive data points, to highlight diverse directions in which popular fiction may be taken. The potential application of such tools across at publishing’s “discovery” stage are endless. It not only helps determine best-sellers but also helps towards creating one.

The bestseller-ometer has nominated Dave Eggers’ The Circle as the exemplary best-selling text from the past 30 years. The book was accorded a 100 percent chance of best-selling. The algorithm worked right, as the popular fiction sold 220,000 copies as of June.

The tool combines the processing power of thousands of computers, and to identify the characteristics of best-selling fiction at scale by interrogating a massive body of literature.

Last Words

Differentiate your publications and meetings with Big Data Analytics

Archer and Jockers have no notion at the moment to “disrupt” publishing with their tool. In other words, they have no immediate plans to commercialize the algorithm. This will serve as a prototype for the approach’s potential in tackling literary questions. Besides, the algorithm could sharpen publishers’ ability to identify prospective bestsellers at the manuscript stage.

However, Knopf editor Carole Baron is pretty skeptical of the forecasting power of an algorithm, as it is based on already-published works.

Moreover, publishers are rarely inclined towards diverting funds to unknown writers. This is where the bestseller-ometer algorithm can come really handy, in the form of a tool that ease publishers’ concerns about taking a flier on a rookie author. It wouldn’t come much as a surprise if the tool helps authors in creating the next Harry Potter-like success.



Download our Mobile App

Amit Paul Chowdhury
With a background in Engineering, Amit has assumed the mantle of content analyst at Analytics India Magazine. An audiophile most of the times, with a soul consumed by wanderlust, he strives ahead in the disruptive technology space. In other life, he would invest his time into comics, football, and movies.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

15th June | Online

Building LLM powered applications using LangChain

17th June | Online

Mastering LangChain: A Hands-on Workshop for Building Generative AI Applications

Jun 23, 2023 | Bangalore

MachineCon 2023 India

26th June | Online

Accelerating inference for every workload with TensorRT

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Is Sam Altman a Hypocrite? 

While on the one hand, Altman is advocating for the international community to build strong AI regulations, he is also worried when someone finally decides to regulate it