Radiocentre and Ebiquity’s new study Re-Evaluating Media was good news for traditional (or proven, non-digital, legacy, whatever we are calling it this week) media, not least Out of Home.

The study found Out of Home is the best medium for maximising reach, the best medium for building frequency, the second-best medium for low cost audience delivery and the third best for getting ads noticed.

If you want lots of people to notice your ad lots of times at great £value – OOH is where it’s at!

I was surprised to read in Re-Evaluating Media that OOH did not score so well in Increases Campaign ROI and Brand Salience. There is ample evidence that OOH delivers ROI in abundance, so why does it not show up in the Radiocentre report?

Let’s take two examples that make the point. 

First, we can quantify ROI from work like Talon’s 2015 joint econometric study with BrandScience and seven OOH media owners. For example, OOH delivers a ROI of around £4.50 for a medium weight campaign.

One of the strengths of Talon’s study was ensuring BrandScience meta-analysis of 200+ campaigns used ROUTE data for OOH measurement (see here for why this is so crucial).

Secondly and even more recently, OOH specialist agency, Rapport released their 2018 study Standing on the Shoulder of Giants; an IPA Databank analysis by Peter Field (the doyen of media effectiveness).

Rapport found OOH strongly increases brand measures like esteem (+41%), fame (+32%), plus it increases customer acquisition (+15%), profit (+26%), and market share (+30%). (There is a lot more in the study - read it here)

I started digging into the Re-Evaluating Media methodology. I strongly commend Radiocentre and Ebiquity for the transparent and accountable detail their study provides. It was time to get nerdy.

The first thing I looked at was how the relative importance of a medium’s attributes was looked at.

Radiocentre and Ebiquity conducted 116 in-depth telephone interviews. These were split between a number of groups. 68 were with marketers and media experts in companies spending £2m+ on advertising in the last year, 30 interviews were with media buying agencies, a further 13 interviews were held with full service agencies and 5 with creative agencies.

According to the interview responses, building frequency, ads being noticed and low cost audience delivery were considerably less important attributes, compared to targeting the right people in the right place at the right time, increasing the campaign ROI and increasing brand salience.

The importance of all the attributes was calculated using MaxDiff analysis, which is the statistical version of forcing responses into either/or choices, rather than expressing strength of preference. It is commonly used to determine the specifics of a product and its pricing, where one thing can be traded against another and ultimately an optimum offer determined.

But in this case, many of the attributes are interdependent. For example, increasing brand salience and increasing ROI are not either/or. One is likely to influence the other.  MaxDiff is not really designed to be a ‘real world’ representation of how campaign strategies are executed. You don’t set out to build frequency and not to have our ads being noticed, so why use a technique that suggests this might be possible?

The problem is that this type of analysis will, at first glance, leave the reader with the impression that OOH and other media don’t do so well in ‘important’ things.

Taking the parochial path, I looked at how Re-Evaluating Media calculated OOH’s (low) scores for Increasing the Campaign ROI and Increasing Brand Salience.

Campaign ROI relied on three sources:

Firstly, research commissioned by MarketReach, The Private Life of Mail (2015). MarketReach’s partner for the ROI section was BrandScience who did an econometric analysis in 2014 that included OOH and it found OOH’s ROI was about £3.20 (6th highest).

The second source used to calculate the OOH ROI was another Radiocentre study. This time from the 2013 paper, Radio the ROI Multiplier. In this econometric analysis, OOH’s ROI was about £2.00 (5th highest). They measured 461 Radio campaigns and just 41 for OOH. The study recognises this, saying: “The project was instigated by the radio industry, so radio was always going to be central to the agencies’ replies: we did not set any quotas for reporting of other media”

Route launched in 2013 so it’s not surprising Radio the ROI Multiplier contains zero Route data – but why are they and Ebiquity relying on a 2013 study about radio for a 2018 OOH ROI?

The last source is Ebiquity’s own database (2014–17), which calculates OOH ROI at £0.57 (7th place). I don’t know how many OOH campaigns they had or how many had Route data in them (versus flat spend or client business KPI data).

Brand Salience relied on five sources, including  DCM’s 2016 Building Box Office Brands Vol II (a cinema advertising study using Millward Brown’s surveys) and two further 2016 studies from Magnetic: Why Being Different Still Makes a Difference and Metrics that Matter (both about magazine advertising). At least there is one study from OOH here: Clear Channel The Power of OOH on Consideration (2016), plus an IPA piece: Media in Focus (2016 Binet & Field).

By way of contrast, OOH’s top score in maximising reach is sourced from an independent and fresh IPA Touchpoints rolling dataset (itself using JIC-approved ROUTE data for OOH).

Pulling this together, I am not suggesting Radiocentre and Ebiquity’s Re-Evaluating Media is a bad study. The findings are encouraging for traditional media. Nor am I suggesting that OOH must be number one in every metric imaginable. Horses for courses.

But I am inclined to think that the low score in brand and ROI metrics may be predicated on less-than-ideal source material. It’s fair to say that using better sources may tell a different story and more accurately represent OOH’s contribution in these areas.