measuring the roi of social media... or not

This week I've had the pleasure of lecturing on the Master in Digital Marketing course at the IE Business School, which offers the #3 ranked MBA course in the World according to Forbes. It's a daunting task - these people know their business, and are already far more knowledgeable about social media than most people I've presented to or taught in the past. Which is why I need your help.

In session one, we did an indepth exploration of the theory behind, and practice of, social media both consumer/audience facing and inside the organisation (eg. "Enterprise 2.0"). In the second session I showed them - some of them already knew how to do much of this - how to use the whole web as their canvas.

But, as you'd expect from a top notch group of students, they rose up in unison at the end, and asked me to come up with real hard data on the ROI of social media. Now everyone in the social media industry knows that there are plenty of vendors, service providers and social media gurus who claim to know the answer, yet I get the feeling that most of these "infuencer metrics" and long tail curves just aren't going to cut it with this audience. And they shouldn't.

But there is growing evidence that, although social media spend is said to be growing, few of those spending the dosh have worked out a way to measure the ROI. Indeed, a study - and I don't put a huge amount of faith in the results of a single study, but it is noteworthy regardless - suggested as few as 14% of professionals across a range of industries claim to measure ROI for their social media activities.

For as long as I can remember, the main benefits of hosting a community on a site, or otherwise engaging via social media, have been to:

• increase brand loyalty and affinity
• increase the number of opportunities to sell to consumers (by increasing repeat visits)
• get users involved in creating fresh, new content on the site
• learn more about users - primarily demographics, but also observing behaviour
• in a few instances, it's also about crowd-sourcing customer service and/or innovating new ideas for service delivery and products
• reach out to new audiences, and better serve captive ones

Inside the enterprise, it's all about making efficiency savings by bubbling content and knowledge to the top, increasing connections between staff, and supporting useful behaviours such as group collaboration, sharing, etc.

I have over simplified, of course, but those are the main things I hear the industry saying.

Now don't get me wrong - I very strongly believe that all of these things are worthwhile. Social media can help organisations reach out to new audiences, embrace existing ones, and there's nothing at all wrong with being more open, honest and transparent no matter what business your in. And within the enterprise, I hate nothing more than beareaucratic structures that stifle, rather than assist, people who hold the knowledge, creativity and skills that a business needs to thrive.

However, when it comes to actual, concrete metrics, there are little. Social media doesn't always increase sales by x percent. Enterprise collaboration systems aren't necessarily responsible for a x percent reduction in cost base.

It might be because we're trying to measure something that can't easily be measured. It's taken for granted that word of mouth advertising is the most effective marketing tool, yet we don't, as far as I'm aware, have a way to measure it. And on the Enterprise side, we can't measure the value, or savings, of two employees dreaming up a creative solution or new product during a fag break. Probably because, in most instances, there is no cost involved in implementing it - it just happens.

But that's a tough argument to make with shit hot marketing students who are used to measuring the effectiveness of campaigns and other activities.

Which leaves me - and our whole industry - in an awkward position: 

Plan A: I can either go back to them tomorrow and tell them that, actually, it wasn't their wonderful £3 million campaign that increased sales by 20% year on year, or their new management structure that led to greater effectiveness and efficiences, it was the fact that it was sunny outside, interest rates were low, and people just generally felt happy in themselves. Implying, basically, that you simply can't measure this sort of thing effectively.

Plan B: I come up with some compelling evidence, both anecdotal and hard quantitative data, to prop up our argument - one I strongly believe in - that social media, and enterprise social media, not only genuinely make an impact, but that its ROI can be measured.

So I need your help. Help me come up with a strong, evidence based argument(s) for either Plan A or Plan B - by 15.00 London time (BST) Friday or I'm toast. In return for your efforts and insights, I'll post the results of this research, as well as output from the discussion, right here.

Comments

[twitter.com/anu]

What's wrong with: 'We spend £300k on a social media engagement programme. Sales went up by £1 million (or costs decreased by £400k)' There's your ROI there.

Oh, but that ignores all the other factors that might have been responsible for the increase/decrease. The thing is that I don't think they can be measured without an enormous amount of effort, which, most times, isn't worth it. So then you decide to not measure certain things, and then selection bias effects inevitably come into play.

The gold standard for effectiveness measurement is a well put together clinical trial. They run for years, involve thousands of people, and every effort is made to control bias. Cost millions though.

Pretty much every industry involved with information transfer between humans (marketing, advertising, PR, comms, KM) has tried and failed to come up with valid and cheap ways of calculating ROI - I don't see how social media can be different.

[gianandrea facchini]

Well, first of all, go and question the definition of Roi.
Second, ask them if there is any real, concrete, evidence that sales were propelled by a TV spot other than a graph showing that sales goes up within a TV campaign.
If not, just catch a graph showing the sales trend within a wom campaign.
Compare the cost of any activity and the conversion cost.
That should be fun.

Then, let's go over and define the goal of social media activity which is no to sell something but to prepare the ground for a sale.
It's called empathy generation.
When a friend of you talk about a computer or a car, you do not hurry up in a store and buy it, if you don't need it, but the recommendation remain and work on the brand perception and likeability.

You can measure empathy with research and with some metrics online, such as trend of positive and negative post, number of post on a product, etc.

[twitter.com/alexwoodcreates]

Great post!

I've been pondering this a lot recently. Recent developments in social media monitoring tools including Netvibes and Google widgets (within Google apps) have opened my eyes to the potential of real time intelligence.

OK, so you may not be able to measure ROI immediately, but you can see the real time effects of your campaign, whether that's online or a traditional offline campaign. And I think that's worth its weight in gold.

[twitter.com/hcdelp]

I'm with Anu.

You have to measure change in sales/savings using a benchmark created before launching/changing the social media program and the best way to do this is by consumer research - relatively expensive and time-consuming, and thus frequently avoided.

Second problem - nobody wants to take the time to figure out the metrics that apply to their company.

Coca-Cola sells products offline to massive numbers of customers. They need to keep track of multiple inputs of staffing and technology investment in SM and need to conduct(and can afford) the extensive consumer research that will tell them whether their SM campaign was influential in purchases. They can also track the cost savings from reducing the number of customer service representatives they laid-off because social media is so much more efficient for customer care :)

I bought my cat a reflective collar with his email address embroidered on it (in case he gets lost). That company only sells online. The owner is the only employee. He is using existing hardware, free software and needs only to track his time diverted to his SM campaign. He can track click-throughs from his social media profiles and promotional links to determine the effects on his sales. He can also include a quick questionnaire so customers can let him know how they were referred.

And there's every manner of business in-between that need to figure out what the cost and gain are from their individual investment in social media.

Third problem - ROI is not some magic quantitative answer to "Should we be using social media"? I think this is a big one. People are looking for ROI = 10 means we keep using it. ROI = 100 means it's a fantastic success. ROI = -10 means we ditch it and go back to sandwich boards.

Companies need to measure each of their objectives separately to determine whether it is reasonable to get a return from social media for that objective and whether the current SM strategy for that objective is working.

ROI gives companies an overview of the fiscal success of the their social media efforts. Other performance indicators must be measured to break down ROI into the components that are adding to and taking away from return on the social media program. This information is what informs action!

Hope that helps your class a bit!

Best,
Hannah Del Porto
Impactwatch

[Roger Johnson]

I'm tweeting about your blog at 4:30 ET: This blogger asks a great question: measuring the ROI of social media... or not = opensource ROI - http://ow.ly/qVJu

I wish you success in this noble inquiry.

Roger

[Kimmo Linkama]

I'm not 100% sure about this, but couldn't social media ROI be measured in much the same way PR ROI has been measured for ages? It won't be possible to attribute certain new customers or certain sales made to certain clearly-defined activities, but surely it will be possible to calculate the input in social media activities vs. brand awareness, loyalty, customer inquiries, changes in ad campaign pull percentages or other "soft" parameters over a period of time?

From my adventures in social media - aimless as they are yet - I have come to the conclusion that many people are approaching social media ROI in a somewhat erroneous way. As if we were talking about a "campaign" where you invest a certain amount of money, work-hours or whatever, launch it on day X, end it on day X+something and then sit back and let the bean-counters tell you how much extra revenue it brought in. It just doesn't work like that.

In fact, I am slightly amazed at how scattered the opinions about SM ROI are, given that social media "gurus" seem to come thirteen in a dozen. Let's accept it, the more we are dealing with peer-to-peer communication, the less accurately measurable the results will be.

[Lee Provoost]

Hi Robin,

One of the toughest talks I ever had in my life was defending the raison d'etre of a large tech community at Capgemini to an executive board member. The main question the guy had for me was: "what will this bring in for extra revenue or profit in hard euros for the company?".

At that particular moment a very frustrating experience, but he was actually exactly right. The problem is that probably every day there are bright guys with bright ideas standing at your desk with the next big thing. How do you make a choice/decision as an executive? Well with the easiest and most fundamental benchmark: what is the return on investment?

Now ROI is a pretty difficult to measure and to define concept, especially in the scene of social media. First of all, I think you can tell an amazing story about the qualitative ROI from social media like brand recognition, customer intimacy, etc to an extend where the class room will shed tears from the lovely story. Knowing that it's an executive MBA class, they'll probably just ask you: so what is the quantitative ROI? Read: how many euro will this bring extra in the pocket?

First of all looking at the cost side: this is pretty easy to calculate: cost of (dedicated?) employees, cost of software, consulting, etc.

Second looking at the benefits. Now, the interesting thing is that money is not necessarily the only hard quantitve ROI that CEOs are looking for. The following are equally important:
- Lead-to-sales time: if the introduction of a wiki for project references, or a microblogging tool, would speed up the time you need to answer for a proposal with 30 %, well that means that you can more quickly respond, but also in the same time respond to many more RFPs coming in.
- Time-to-market: if you can speed up the time you need to deliver because you can lower your time needed to develop a product (e.g. better cooperation between your suppliers and you due to better collaboration tools), that alone is already a pretty compelling reason. Now of course this varies from company to company (it has to be measured continuously) but if you could reduce time to market from 1 year to 8 months, that means 4 months of revenue/profit and you;re faster than the competitors. This business agility reason is a VERY compelling one, not only in social media but also for instance in cloud computing.
- Bring employees up to speed: one of the biggest frustrations, and cause of lost time, is the fact that new employees (new in the company or new in the project) can't find their way. I have now clients that have specific on-boarding kits on the wiki for instance where you can also ask around in Yammer for help. If that can speed up the employees onboarding from weeks to a couple of days...

So unfortunately these are not the hard ball figures you were looking for i think. It's all about measuring, measuring and measuring. That's why I am pro the fact of just starting up small initiatives, bottom up, let it grow, don't spend millions at once. You will quickly see if there are results (that can be measured).

Hope it helps you a bit. Looking forward having some ROI discussions when I'm in London in a month!

Cheers,

Lee

[Robin]

Thank you everyone for your thoughts. This is really useful, although what everyone seems to be saying is that it's difficult to measure quantitatively. I know there must be numbers out there - claims, both imagined and evidence based - of numbers that can be achieved with social media. If anyone knows where I'd find such evidence for discussion in the class this would be great.

Thanks again for taking the time to read and respond, it's much appreciated, even if I fear I'm still going to get skewered...

:-)
Robin.

[Hannah Del Porto]

http://www.readwriteweb.com/archives/social_media_roi_dells_3m_on_twitter_and_four_bett.php

[twitter.com/kuno]

It's difficult to measure "accuracy" if you are shooting at a small target with a shotgun. Same with ROI. If you have a focused campaign with specific goals and strategies, then you use a toolkit that allows you to measure each step and final outcome, you can achieve your objective.

As an example, let's say the goal is to increase leads to 1000 per month and out of that yield a conversion rate of 5%, or 50 new customers a month. We launch a Web campaign with a specific call to action for a great discount or free content on our home page leading to a lead capture form. We optimize our web site and landing pages for SEO with campaign-specific keywords. We promote the campaign using social media, measure the results daily for web traffic, seo, and leads and tweak our strategy to optimize results. We have a process in place for calling or e-mailing leads to convert them to customers, and we track that as well. At the end of the campaign we know how much labor has been expended for the tasks (our cost plus whatever software you use), how many leads we have captured and our conversion rate to sales. So we also know revenue produced by our campaign.

So now we have ROI measured very accurately. Obviously social media is not the only component of this strategy, and that's how it should be.

In case you're wondering, there are tools that do all of this. We use and resell HubSpot (www.hubspot.com).