The measure of a marketer

If you’re a marketer you measure creativity. This is not a ‘should you?’ question. The alternative would be – what? – to explain to the board that you intend to invest tens of millions of corporate dollars in an activity that might, or might not, who knows, bring some kind of return. That would be like the operations director announcing the multi-million purchase of a new machine line but declining to say what function it is to serve, if any. So measure you must.

That’s the easy bit out of the way. The questions from here get tougher. When? How? With what objective in mind? With what tolerances for the inexactitudes that characterise the creative process?

Contemplating that last point, it occurs to you, if it hadn’t already, that the interface between ‘measurement’ and ‘creativity’ isn’t an interface at all – since that sounds far too clean and clinical – but a messy collision of opposites. Precision and tidiness on one side; chaos and serendipity on the other.

Bringing the mindset of metrics to the domain of free expression is like peeling off a wide city boulevard to find yourself abruptly in the Latin quarter. The straight lines and rectangle turns are now behind you, and have given way to winding alleys and knotted junctions, with the turn-offs all on the slant. It would be fun if you didn’t mind getting lost, but in the wrong clothes, with the wrong attitude and an A-to-B objective propelling your steps, it is oddly uncomfortable.

That said, your first big decision is binary enough, or as close as it gets in this marketing neighbourhood. Do you measure after the creative work has been commercially exposed? Or before? Or both?

The virtue of the post-activity option is that it assesses the actual thing – the actual pack design, the actual YouTube spot – not an early approximation of it. And the metrics can have real-time, real-market validity – perhaps even getting as crudely commercial as looking to see if sales went up after the new work was unveiled.

Any marketer with a capacity for nuance would pause here, however, and consider the downsides. There are two. The first is the notoriously tricky issue of isolation. New creative material is often accompanied by substantive changes in the product or service offer. If sales rise, how do you know which bit did the work?

And no matter what post-activity metrics you use – awareness, net promoter score, volume sales – the data will always be confounded by the crazy asymmetries of the seething marketplace, and especially by competitor activity.

The second problem with measuring after the work has appeared is that the work has appeared. It’s too late to improve it now.

Not that pre-testing is any cleaner. The first question to vex even the experienced and diligent is the level of veracity of the test materials to the final piece. Some marketers pride themselves on stripping out all ‘executional’ elements to gain a truer assessment of the deep thought within.

Yet one of the finest marketers of recent years – Unilever’s former CMO Simon Clift – maintained that marketers spend too much resource assessing the ‘concept’ and not enough on what they consider to be the ‘incidentals’. In his work on Olivio and Lynx he pioneered pre-testing of music, wardrobe and casting, since in his view they could violently polarise.

It’s your choice, but know this: whether you opt for simplified approximation or more realistic expression, you are going to have to live with imperfection. There are advantages and flaws to both, and zealots insisting you go one way should be heard with caution.

As big a test of your judgment will be the chosen methodology. It used to be simpler. Research agencies would, from a position of neutrality, report back on whether consumers understood, enjoyed and were motivated by prototype creative work.

Today, research companies are rarely agnostic. They come with their own views about what ‘works’ and assess creative content against that preconceived ideal. One of the big sexy ones out there claims to know that advertising creative needs to be ‘emotional’ to perform at the highest level. Its single-figure metric is devoted to crystallising that.

The naive marketer will be seduced; it’s a good spiel. The sceptical marketer will probe the source of their certainty – and be told that it relates to analysis of past IPA Effectiveness Awards winners. Hearing that, and checking how those winners are chosen, the sceptical marketer is likely to remain sceptical.

These ‘single-metric’ research firms usually provide ‘guidance’ on how to attain the make-or-break number that deems the work worthy to go ahead: this kind of narrative, that kind of slogan, a certain kind of track.

But once a marketer starts chasing the metric you know that the spirit of creativity has died. The logic should be drumming away inside your subconscious: if big research agencies advise all their clients how to do it, and they all comply, convergence must be the outcome. Yet that breaks the number one rule of commercial creativity: be different.

None of these measurement decisions would be beyond the skills of a normally competent practitioner were it not for another factor: tick, tock. Testing eats time, and a crunch moment will inevitably loom. Mild panic is understandable: this idea did better than that one, and it may not be perfect, but – tick, tock – let’s go with it.

The better leaders remain calm, and insist on 48 hours they don’t really have, for reflection. The very best call for reflection even when the news has been positive; like elite sportspeople, they somehow find precious moments to eschew the good and go for the great, sometimes making it all seem easy.

It’s not. This stuff is hard. It’s exposing. You start out wanting to test some work, only to find yourself being tested. You hope the research will decide, only to realise that the biggest decisions will be all yours. You embark on a process with the laudable aim of measuring creativity – only for the truth to dawn, somewhere towards the end, that creativity is measuring you.