Marketing ROI Prediction—Our Best Study Yet

It’s been awhile since we’ve made one of these! Expect to hear from us more often this year. There have been some major product updates:

-       You can now target dozens of different audiences on Slayer (from fitness buffs to foodies)

-       Processing time on our AI was cut by more than 50%

-       We validated our model accuracy

That last piece is the purpose of this blog post. One of the most common questions that we get asked these days is: “How do I know your product really works?”… and while we could point to a lot of great anecdotes—clients boosting customer touches 30%, email open rates increasing 70%, users going viral and getting millions of impressions on social media… they were always just anecdotes.

That changes today.

We did a study of 29 different Slayer models, and verified that model accuracy is75-80%.

How did the study work?
We developed a testing database of about 375,000 headlines, along with their engagement rates (comments, likes, click throughs etc.). We showed our AI models only the headline, and asked it to guess the engagement rate.

For all 375,000.

And we found a couple of really interesting things.

1.     Our AI is incredibly predictive.Some models were 85%+ accurate over thousands of headlines. Which brings us to our next point:

2.     The AI’s accuracy was different foreach audience. Certain audiences are either more predictable or care about text more than others. This jives with what we’ve observed in the real world.

Every single one of our models performed above expectations.

For fun, we ran a smaller test with some of our rivals, who prescribe general best practices to writing headlines instead of being AI based—they had less than 50%accuracy on a sample of about 100 headlines... basically the same as flipping a coin. While those tools will help you write with a set of conventions, it won’t help you be engaging the way Slayer can.

If you’d like to use our AI to increase your marketing ROI while spending less time on copy, we’d love to hear from you. Feel free to sign up for the product (it’s free!) or shoot us a message via hi@slayerai.com