Why Measuring User Engagement Is Harder Than You Think
Photograph by Diyosa Carter/Getty Images
If you’ve spent any time in a newsroom, traditional or otherwise, you know that publishers are obsessed with measuring where their Web traffic comes from. Whether it’s Google (GOOG) Analytics or Chartbeat, or comScore, or Omniture, or any one of a dozen other providers, tracking where readers come from is a crucial part of online media—mostly because publishers need to know which channels are worth focusing on, since there are so many to choose from. Is Twitter your biggest source? Then you should tweet more and optimize your content for Twitter. Is Facebook (FB) a big referrer of traffic? Then you need to be aware of changes to the newsfeed and how they affect you.
But what if your biggest source of traffic and readers is something you aren’t even really paying attention to, and something that is extremely hard to track in the same way as Google or Twitter or Facebook? That’s the reality of Web publishing today, according to Alexis Madrigal at the Atlantic—who writes about the influence of what he calls “dark social” on engagement and traffic patterns. While everyone is busy watching Twitter and Facebook because they are easy to track, Madrigal argues that most social traffic still comes from old-fashioned or difficult-to-track sources such as e-mail and chat messages:
“This vast trove of social traffic is essentially invisible to most analytics programs. I call it DARK SOCIAL. It shows up variously in programs as ‘direct’ or ‘typed/bookmarked’ traffic, which implies to many site owners that you actually have a bookmark or typed www.theatlantic.com into your browser,” writes Madrigal. “But that’s not actually what’s happening a lot of the time. Most of the time, someone Gchatted someone a link, or it came in on a big e-mail distribution list, or your dad sent it to you.”
As evidence, Madrigal provides some data from Chartbeat, the Betaworks spin-off that focuses on real-time analytics for publishers, looking at everything from the amount of time readers spend on a page to how far down they got in an article before they decided to click away. Chartbeat, which we have written about a number of times, is one of the few analytics engines that tries to break down that “direct” category into sub-categories, such as e-mail, and what the service calls “direct social”—meaning everything from apps (for chat or other social features) to instant messaging.
For the Atlantic, the impact of this kind of direct social traffic outweighs any other kind of social network or service, such as Facebook, Twitter, or Reddit: According to data from Chartbeat, the magazine’s website gets almost 60 percent of its social traffic from these hard-to-track sources. Facebook is still a large referrer for the site, generating about 21 percent of the social traffic, and Twitter is also fairly large at 11 percent—but the “dark social” category is larger than all the other social services combined and has more than twice the impact that Facebook does.
Chartbeat’s numbers also showed that this direct-social traffic was a large contributor for other sites that the service tracks, according to Madrigal—and Chartbeat is used by some of the largest publishers in the media business, including ESPN and the New York Times. On average, the company’s stats showed that close to 70 percent of the social traffic to these sites came from e-mail, instant messaging, chat apps, and other sources. (As Madrigal points out, at many websites, including the Atlantic, social traffic far outweighs traffic that comes from search, and that gap is still growing.)
The one obvious conclusion to take away from all of this is that measuring user engagement and sources of traffic is probably a lot harder than most publishers think—and they likely already thought it was pretty hard. It’s bad enough that comScore and Compete and Nielsen (NLSN) and Google Analytics all provide different numbers, and it’s almost impossible to tell who is right (especially since all these sources often disagree with a publisher’s internal statistics). Now there is a huge source of traffic that is even harder to measure: E-mail is trackable in the aggregate, but how do you track instant messaging?
This problem is compounded by the shift to mobile content consumption, since chat apps and instant messaging and other direct communication methods are even more prevalent in the mobile world than on the desktop. Links are passed from social network to apps to chat to e-mail, and tracking them quite quickly becomes almost impossible. That’s part of the reason why almost all Web publishers get surprised by posts or stories that blow up traffic-wise days or weeks after they first appeared, with no obvious sign of how or why they hit that invisible tipping point.
So how are publishers and media companies supposed to deal with this problem? Madrigal’s solution is an appealing one, at least for those who create content—he says the only dependable way of generating real traffic and engagement is actually to write things that people care about or are interested in. In other words, the “content is king” approach. As he describes it:
“The only real way to optimize for social spread is in the nature of the content itself. There’s no way to game e-mail or people’s instant messages. There’s no power users you can contact. There’s no algorithms to understand.”
It’s not that getting people to share content via e-mail or chat app or instant messaging is that different from trying to get them to share it on Twitter or Facebook—the same general rules apply, in the sense that it has to be engaging and interesting and shareable, and all those other things we are supposed to be doing with our content. The difficult part is that it’s hard to track in the same way publishers watch Facebook “likes” or page followers, or Twitter retweets and other metrics. So it’s difficult to tell whether it’s working and why, or what you should do differently.
In a sense, what Madrigal is describing just reinforces the fact that much of what content companies do is more of an art than a science—even though social-media gurus and analytics providers would like to make it sound like something that can be quantified and measured from every aspect. And maybe that’s not such a bad thing, even if it does make our jobs harder.
Also from GigaOM:
Social Media Advertising Realities (subscription required)