Opinion: The Problem With Data/Analytics in Esports: It’s Not Them, It’s You!

If you run around and say, ”TSM is worth $410M,” you are basically saying you trust this data and its sources to an extent that you don’t even question the methodology. When you run around and say, “These numbers are false,” you do the opposite. While the truth might be somewhere in the middle of these two diametrically opposed views, the valuations are not wrong as they follow the outlined methodology, which has been made transparent. If you don’t do your homework on this, it’s your fault, not Forbes’.

Basically Every Market Research Report

Whether it’s Newzoo, Nielsen, YouGov, or anyone else – it doesn’t matter. Did you ever take the time to look at the methodology before interpreting the data or forming an opinion for your decision-making process? Professional reports always walk you through their methodology or at least outline it somewhere in the report. If you can’t find a methodology, you might want to be careful. If you find a methodology, you’ll want to read this carefully. It’s funny, because the true and original source of data is often fully documented with context and clarity. When people interpret these data-points out of context, the blame is often put back on the source, not the person who interpreted it incorrectly. This behavior has led us to headlines like “League of Legends Worlds is bigger than the Super Bowl.”

You can have some fun and ask the next person that pitches you a product or a service with one of the standard infographics in it what is included in the numbers and what isn’t. You shouldn’t just take data out of context because it works for your narrative; yes, it might drive attention and get you a “wow” moment, but if it’s inaccurate, it will do more harm than good long-term.

Hours Watched – An Established Viewership Metric in Esports

We have gotten so used to Hours Watched as a viewership metric in esports, that you will find it everywhere. What does it actually say? It’s a content consumption metric. If I watch for two hours, I generate two hours watched. If six people watch for 20 minutes each, they generate two hours watched. Without other data points, such as total number of viewers, it’s hard to draw deep conclusions from this metric. We are comparing total results. If a stream gets a 20% increase in hours watched, that’s good. And it is. We just don’t know if more people watched for a short duration or if fewer people are watching for a longer time. Again, two very different things. Also, the amount of Hours Streamed can be variable (e.g. between a year-long league and a one month tournament). A mix of multiple metrics like AMA/ACU, Hours Watched, and Peak Viewers would get you a more complete picture.

In an attempt to establish a viewership metric that is closer to TV ratings, Nielsen implemented the AMA (Average Minute Audience). Do you know how this is calculated or what it says? If not, I hope you read up on it before you quote it. 

When I discussed my opinion with Pike, she added: “Research in esports is not easy – I’m the first to admit that. I’m acutely aware of the gaps third-party providers have in our data, the fact publishers have much better data than we ever will, the need for more granularity…etc etc etc. I don’t try to hide this, and anyone who has worked with or taken the time to have a conversation about data with me will vouch for that. I encourage anyone who is skeptical of esports data to engage in dialogue on the topic, and become part of the solution, not just point out the problems.” 

I could go on forever and apply this to more metrics, reports, and even social media followings. Here is my point as the TL;DR: Metrics are often not perfect, but they can provide guidance. You are responsible for making sure you understand the data you are using. Get educated and stop interpreting things without understanding the underlying methodology. More importantly, stop calling bullshit on things you don’t understand, and if you think you know better, become a part of the solution, instead of continuously bashing others.

Source: Read Full Article